From f058d6bc4bb9179ccdf0709ad87f46411d0ca810 Mon Sep 17 00:00:00 2001 From: Peter Hauge Date: Sat, 6 Jun 2026 14:46:26 -0700 Subject: [PATCH 1/3] feat: align filter and override config formats with APIOps Toolkit - Rename filter config keys from *Names suffix to bare camelCase plurals (e.g., apiNames -> apis, backendNames -> backends, versionSetNames -> versionSets) to match the Toolkit's configuration.extractor.yaml schema - Rename filter config filename from configuration.extract.yaml to configuration.extractor.yaml across templates, CI/CD workflows, and docs - Add graceful handling for apimServiceName in override config (Toolkit uses this for target APIM instance; CLI logs info and ignores it) - Override config format already aligned (same section names and properties wrapper structure as Toolkit) - Update all 13 test files, 7 docs, CI/CD templates, and specs Closes #114 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/ci-cd/azure-devops.md | 6 +- docs/ci-cd/github-actions.md | 2 +- docs/commands/extract.md | 52 ++++++------- docs/commands/init.md | 6 +- docs/guides/filtering-resources.md | 76 +++++++++--------- docs/guides/migration-from-v1.md | 8 +- docs/reference/configuration.md | 10 +-- specs/contracts/cli-commands.md | 2 +- src/lib/config-loader.ts | 77 +++++++++++-------- src/models/config.ts | 32 ++++---- src/services/filter-service.ts | 66 ++++++++-------- src/services/init-service.ts | 6 +- src/services/transitive-resolver.ts | 8 +- src/services/workspace-extractor.ts | 4 +- .../azure-devops/extract-pipeline.ts | 4 +- src/templates/configs/filter-config.ts | 38 ++++----- .../github-actions/extract-workflow.ts | 4 +- tests/unit/clients/artifact-store.test.ts | 10 +-- tests/unit/lib/config-loader.test.ts | 52 ++++++------- .../services/api-product-extractor.test.ts | 4 +- .../services/delete-unmatched-service.test.ts | 26 +++---- tests/unit/services/extract-service.test.ts | 6 +- tests/unit/services/filter-service.test.ts | 30 ++++---- tests/unit/services/init-service.test.ts | 6 +- .../unit/services/resource-extractor.test.ts | 2 +- .../unit/services/transitive-resolver.test.ts | 16 ++-- .../unit/services/workspace-extractor.test.ts | 14 ++-- .../azure-devops/extract-pipeline.test.ts | 2 +- .../configs/config-templates.test.ts | 40 +++++----- .../github-actions/extract-workflow.test.ts | 4 +- 30 files changed, 312 insertions(+), 301 deletions(-) diff --git a/docs/ci-cd/azure-devops.md b/docs/ci-cd/azure-devops.md index 86bd7c4..d1237e0 100644 --- a/docs/ci-cd/azure-devops.md +++ b/docs/ci-cd/azure-devops.md @@ -43,7 +43,7 @@ The extract pipeline pulls configuration from your APIM instance, publishes the | Parameter | Type | Default | Description | |-----------|------|---------|-------------| -| `CONFIGURATION_YAML_PATH` | string | `Extract All APIs` | Choose `Extract All APIs` for a full extract, or `configuration.extract.yaml` to use a [filter file](../guides/filtering-resources.md) | +| `CONFIGURATION_YAML_PATH` | string | `Extract All APIs` | Choose `Extract All APIs` for a full extract, or `configuration.extractor.yaml` to use a [filter file](../guides/filtering-resources.md) | | `resourceGroup` | string | `$(APIM_RESOURCE_GROUP)` | Azure resource group containing your APIM instance | | `serviceName` | string | `$(APIM_SERVICE_NAME)` | Name of the APIM service instance | @@ -55,7 +55,7 @@ flowchart TD B --> C[npm ci] C --> D{Configuration choice?} D -->|Extract All APIs| E[apiops extract --resource-group ... --service-name ...] - D -->|configuration.extract.yaml| F[apiops extract ... --filter configuration.extract.yaml] + D -->|configuration.extractor.yaml| F[apiops extract ... --filter configuration.extractor.yaml] E --> G[Publish pipeline artifact] F --> G G --> H[Create branch apim-extract-BuildId] @@ -92,7 +92,7 @@ The key task is `AzureCLI@2`, which authenticates using your service connection: --subscription-id $(AZURE_SUBSCRIPTION_ID) ``` -When the filter option is selected, `--filter configuration.extract.yaml` is added to the command. +When the filter option is selected, `--filter configuration.extractor.yaml` is added to the command. > **Why AzureCLI@2?** This task injects Azure credentials into the shell environment, allowing `apiops extract` to authenticate via `DefaultAzureCredential`. See [Authentication Guide](../guides/authentication.md). diff --git a/docs/ci-cd/github-actions.md b/docs/ci-cd/github-actions.md index a422973..48c993b 100644 --- a/docs/ci-cd/github-actions.md +++ b/docs/ci-cd/github-actions.md @@ -44,7 +44,7 @@ The extract workflow pulls configuration from your APIM instance and creates a P | Input | Description | Options | |-------|-------------|---------| | `ENVIRONMENT` | Which APIM instance to extract from | `dev`, `prod` | -| `CONFIGURATION_YAML_PATH` | Extract all APIs or use a filter file | `Extract All APIs`, `configuration.extract.yaml` | +| `CONFIGURATION_YAML_PATH` | Extract all APIs or use a filter file | `Extract All APIs`, `configuration.extractor.yaml` | ### What It Does diff --git a/docs/commands/extract.md b/docs/commands/extract.md index cbb0ba4..b5a3422 100644 --- a/docs/commands/extract.md +++ b/docs/commands/extract.md @@ -34,7 +34,7 @@ apiops extract \ --subscription-id 00000000-0000-0000-0000-000000000000 \ --resource-group my-rg \ --service-name my-apim \ - --filter ./configuration.extract.yaml + --filter ./configuration.extractor.yaml ``` ### Extract without transitive dependencies @@ -98,23 +98,23 @@ By default, `apiops extract` exports **all** resources from the APIM instance (3 To extract only specific resources, pass a YAML filter file with `--filter`: ```yaml -# configuration.extract.yaml -apiNames: +# configuration.extractor.yaml +apis: - echo-api - petstore-api -productNames: +products: - starter -backendNames: +backends: - backend-api -namedValueNames: +namedValues: - api-key -tagNames: +tags: - production -policyFragmentNames: +policyFragments: - rate-limit-fragment -loggerNames: +loggers: - appinsights-logger -diagnosticNames: +diagnostics: - applicationinsights ``` @@ -124,22 +124,22 @@ All 16 supported filter keys: | Filter key | Resource type | |------------|---------------| -| `apiNames` | APIs | -| `backendNames` | Backends | -| `productNames` | Products | -| `namedValueNames` | Named values | -| `loggerNames` | Loggers | -| `diagnosticNames` | Diagnostics | -| `tagNames` | Tags | -| `policyFragmentNames` | Policy fragments | -| `gatewayNames` | Gateways | -| `versionSetNames` | API version sets | -| `groupNames` | Groups | -| `subscriptionNames` | Subscriptions | -| `schemaNames` | Schemas | -| `policyRestrictionNames` | Policy restrictions | -| `documentationNames` | Documentation resources | -| `workspaceNames` | Workspaces | +| `apis` | APIs | +| `backends` | Backends | +| `products` | Products | +| `namedValues` | Named values | +| `loggers` | Loggers | +| `diagnostics` | Diagnostics | +| `tags` | Tags | +| `policyFragments` | Policy fragments | +| `gateways` | Gateways | +| `versionSets` | API version sets | +| `groups` | Groups | +| `subscriptions` | Subscriptions | +| `schemas` | Schemas | +| `policyRestrictions` | Policy restrictions | +| `documentations` | Documentation resources | +| `workspaces` | Workspaces | ### Transitive dependencies diff --git a/docs/commands/init.md b/docs/commands/init.md index a672f5a..42b5914 100644 --- a/docs/commands/init.md +++ b/docs/commands/init.md @@ -80,7 +80,7 @@ In interactive mode (the default when running in a terminal), `apiops init` prom |------|---------| | `.github/workflows/extract.yaml` | Pipeline to extract APIM artifacts | | `.github/workflows/publish.yaml` | Pipeline to publish artifacts to APIM | -| `configuration.extract.yaml` | Sample filter configuration for extraction | +| `configuration.extractor.yaml` | Sample filter configuration for extraction | | `configuration.{env}.yaml` | Override templates per environment (e.g., `configuration.dev.yaml`, `configuration.prod.yaml`) | | `IDENTITY-SETUP-GITHUB.md` | Step-by-step guide for configuring federated credentials | @@ -90,9 +90,7 @@ In interactive mode (the default when running in a terminal), `apiops init` prom |------|---------| | `pipelines/extract.yaml` | Pipeline to extract APIM artifacts | | `pipelines/publish.yaml` | Pipeline to publish artifacts to APIM | -| `configuration.extract.yaml` | Sample filter configuration for extraction | -| `configuration.{env}.yaml` | Override templates per environment | -| `IDENTITY-SETUP-AZDO.md` | Step-by-step guide for configuring service connections | +| `configuration.extractor.yaml` | Sample filter configuration for extraction | ### Both platforms diff --git a/docs/guides/filtering-resources.md b/docs/guides/filtering-resources.md index 1c01b56..fa8da99 100644 --- a/docs/guides/filtering-resources.md +++ b/docs/guides/filtering-resources.md @@ -16,8 +16,8 @@ By default, `apiops extract` pulls every resource from your APIM instance. For l 1. Create a filter file: ```yaml -# configuration.extract.yaml -apiNames: +# configuration.extractor.yaml +apis: - petstore-api - orders-api ``` @@ -29,7 +29,7 @@ apiops extract \ --resource-group my-rg \ --service-name my-apim \ --subscription-id 00000000-0000-0000-0000-000000000000 \ - --filter configuration.extract.yaml + --filter configuration.extractor.yaml ``` Only `petstore-api`, `orders-api`, and their transitive dependencies are extracted. @@ -41,29 +41,29 @@ Only `petstore-api`, `orders-api`, and their transitive dependencies are extract The filter file is a YAML document where each key is a resource type and the value is an array of resource names: ```yaml -# configuration.extract.yaml +# configuration.extractor.yaml # APIs to extract (by display name or API ID) -apiNames: +apis: - petstore-api - orders-api # Backends to include -backendNames: +backends: - orders-backend # Products to include -productNames: +products: - starter - enterprise # Named values to include -namedValueNames: +namedValues: - api-key - connection-string # Leave sections out (or comment them) to extract ALL of that type -# loggerNames: +# loggers: # - appinsights ``` @@ -79,22 +79,22 @@ namedValueNames: | Filter Field | APIM Resource | Example Values | |-------------|---------------|----------------| -| `apiNames` | APIs | `petstore-api`, `orders-v2` | -| `backendNames` | Backends | `orders-backend`, `payment-service` | -| `productNames` | Products | `starter`, `enterprise`, `internal` | -| `namedValueNames` | Named Values | `api-key`, `db-connection-string` | -| `loggerNames` | Loggers | `appinsights-logger`, `eventhub-logger` | -| `diagnosticNames` | Diagnostics | `applicationinsights`, `azuremonitor` | -| `tagNames` | Tags | `production`, `beta`, `internal` | -| `policyFragmentNames` | Policy Fragments | `rate-limit-fragment`, `cors-policy` | -| `gatewayNames` | Self-hosted Gateways | `on-prem-gateway`, `edge-gateway` | -| `versionSetNames` | API Version Sets | `orders-version-set` | -| `groupNames` | Groups | `developers`, `partners`, `admins` | -| `subscriptionNames` | Subscriptions | `team-a-subscription` | -| `schemaNames` | Global Schemas | `shared-error-schema` | -| `policyRestrictionNames` | Policy Restrictions | `no-external-calls` | -| `documentationNames` | Documentation | `getting-started`, `changelog` | -| `workspaceNames` | Workspaces | `team-a-workspace`, `team-b-workspace` | +| `apis` | APIs | `petstore-api`, `orders-v2` | +| `backends` | Backends | `orders-backend`, `payment-service` | +| `products` | Products | `starter`, `enterprise`, `internal` | +| `namedValues` | Named Values | `api-key`, `db-connection-string` | +| `loggers` | Loggers | `appinsights-logger`, `eventhub-logger` | +| `diagnostics` | Diagnostics | `applicationinsights`, `azuremonitor` | +| `tags` | Tags | `production`, `beta`, `internal` | +| `policyFragments` | Policy Fragments | `rate-limit-fragment`, `cors-policy` | +| `gateways` | Self-hosted Gateways | `on-prem-gateway`, `edge-gateway` | +| `versionSets` | API Version Sets | `orders-version-set` | +| `groups` | Groups | `developers`, `partners`, `admins` | +| `subscriptions` | Subscriptions | `team-a-subscription` | +| `schemas` | Global Schemas | `shared-error-schema` | +| `policyRestrictions` | Policy Restrictions | `no-external-calls` | +| `documentations` | Documentation | `getting-started`, `changelog` | +| `workspaces` | Workspaces | `team-a-workspace`, `team-b-workspace` | --- @@ -120,7 +120,7 @@ flowchart TD Given this filter: ```yaml -apiNames: +apis: - petstore-api ``` @@ -150,7 +150,7 @@ apiops extract \ --resource-group my-rg \ --service-name my-apim \ --subscription-id 00000000-0000-0000-0000-000000000000 \ - --filter configuration.extract.yaml \ + --filter configuration.extractor.yaml \ --no-transitive ``` @@ -170,8 +170,8 @@ apiops extract \ A team that owns one or two APIs: ```yaml -# configuration.extract.yaml -apiNames: +# configuration.extractor.yaml +apis: - orders-api - orders-admin-api ``` @@ -183,33 +183,33 @@ Transitive dependencies (backends, named values, policy fragments) are auto-incl Extract everything associated with a product: ```yaml -# configuration.extract.yaml -productNames: +# configuration.extractor.yaml +products: - enterprise ``` -> **Note:** Filtering by `productNames` extracts the product definition and its associations, but does **not** transitively include the APIs in that product. To include the APIs, add them to `apiNames` as well. +> **Note:** Filtering by `products` extracts the product definition and its associations, but does **not** transitively include the APIs in that product. To include the APIs, add them to `apis` as well. ### Shared Infrastructure Team A platform team managing cross-cutting resources: ```yaml -# configuration.extract.yaml -namedValueNames: +# configuration.extractor.yaml +namedValues: - global-api-key - rate-limit-threshold - cors-allowed-origins -policyFragmentNames: +policyFragments: - standard-rate-limit - cors-policy - auth-validation -loggerNames: +loggers: - appinsights-logger -backendNames: +backends: - identity-service ``` @@ -222,7 +222,7 @@ There is no "exclude" syntax. To extract everything except certain resources, li ## Tips - **Start broad, narrow later** — Begin with no filter to see what's in your APIM instance, then create a filter for your team's slice -- **One filter per team** — In multi-team setups, each team maintains its own `configuration.extract.yaml` +- **One filter per team** — In multi-team setups, each team maintains its own `configuration.extractor.yaml` - **Commit the filter file** — Keep it in version control alongside your artifacts so CI/CD pipelines can use it - **Case-sensitive names** — Filter values must match APIM resource names exactly (usually lowercase with hyphens) - **Validate early** — The config loader validates that each filter field is an array of strings and will throw `Failed to load filter config` on invalid YAML diff --git a/docs/guides/migration-from-v1.md b/docs/guides/migration-from-v1.md index 44b0561..9ba902e 100644 --- a/docs/guides/migration-from-v1.md +++ b/docs/guides/migration-from-v1.md @@ -148,20 +148,20 @@ Replace the v1 pipeline tasks/actions with v2 CLI commands. **v1** (`configuration.extractor.yaml`): ```yaml -apiNames: +apis: - payments-api - orders-api ``` -**v2** (filter YAML — same format, different file name convention): +**v2** (`configuration.extractor.yaml` — same format and file name): ```yaml -apiNames: +apis: - payments-api - orders-api ``` -The filter YAML format is compatible. Rename the file if you prefer the v2 convention, and pass it with `--filter`: +The filter YAML format is fully compatible with v1. You can use your existing `configuration.extractor.yaml` as-is with the `--filter` flag: ```bash apiops extract \ diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 9f8c424..1f929eb 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -17,7 +17,7 @@ flowchart TD |----------|--------|---------| | **1 (highest)** | CLI flags | `--resource-group my-rg` | | **2** | Environment variables | `AZURE_SUBSCRIPTION_ID=...` | -| **3** | YAML config files | `configuration.extract.yaml`, `configuration.dev.yaml` | +| **3** | YAML config files | `configuration.extractor.yaml`, `configuration.dev.yaml` | | **4 (lowest)** | Default values | `--output ./apim-artifacts` | For example, if `AZURE_SUBSCRIPTION_ID` is set as an environment variable but `--subscription-id` is also passed on the command line, the CLI flag wins. @@ -109,16 +109,16 @@ See [Authentication Guide](../guides/authentication.md) for details on each auth ### Filter Configuration -**File:** `configuration.extract.yaml` (or any path passed to `--filter`) +**File:** `configuration.extractor.yaml` (or any path passed to `--filter`) Controls which resources are extracted. See [Filtering Resources](../guides/filtering-resources.md) for the full reference. ```yaml -# configuration.extract.yaml -apiNames: +# configuration.extractor.yaml +apis: - petstore-api - orders-api -backendNames: +backends: - petstore-backend ``` diff --git a/specs/contracts/cli-commands.md b/specs/contracts/cli-commands.md index 1e593bf..f60ca31 100644 --- a/specs/contracts/cli-commands.md +++ b/specs/contracts/cli-commands.md @@ -97,7 +97,7 @@ Initialize repository structure and CI/CD pipeline configuration. - `.github/workflows/extract.yml` and `publish.yml` (for `github-actions`) - `.azdo/pipelines/extract.yml` and `publish.yml` (for `azure-devops`) - `apim-artifacts/` directory (empty, with `.gitkeep`) -- `configuration.extract.yaml` (sample filter file) +- `configuration.extractor.yaml` (sample filter file) - `configuration.{env}.yaml` (sample override files) --- diff --git a/src/lib/config-loader.ts b/src/lib/config-loader.ts index 8370adc..6a8f166 100644 --- a/src/lib/config-loader.ts +++ b/src/lib/config-loader.ts @@ -42,53 +42,53 @@ export async function loadFilterConfig(filePath: string): Promise): OverrideConfig { const normalized: OverrideConfig = {}; + // Log and ignore apimServiceName — Toolkit uses this for target APIM instance, + // but CLI uses --service-name flag instead. + if (parsed.apimServiceName !== undefined) { + const serviceName = typeof parsed.apimServiceName === 'string' + ? parsed.apimServiceName + : JSON.stringify(parsed.apimServiceName); + logger.info( + `Override config contains 'apimServiceName' ("${serviceName}"). ` + + `The CLI uses --service-name instead; this field will be ignored.` + ); + } + const namedValues = normalizeOverrideSection(parsed.namedValues, 'namedValues'); const backends = normalizeOverrideSection(parsed.backends, 'backends'); const apis = normalizeOverrideSection(parsed.apis, 'apis'); diff --git a/src/models/config.ts b/src/models/config.ts index e1ac2c7..d6726d9 100644 --- a/src/models/config.ts +++ b/src/models/config.ts @@ -19,22 +19,22 @@ export interface ExtractConfig { } export interface FilterConfig { - apiNames?: string[]; - backendNames?: string[]; - productNames?: string[]; - namedValueNames?: string[]; - loggerNames?: string[]; - diagnosticNames?: string[]; - tagNames?: string[]; - policyFragmentNames?: string[]; - gatewayNames?: string[]; - versionSetNames?: string[]; - groupNames?: string[]; - subscriptionNames?: string[]; - schemaNames?: string[]; - policyRestrictionNames?: string[]; - documentationNames?: string[]; - workspaceNames?: string[]; + apis?: string[]; + backends?: string[]; + products?: string[]; + namedValues?: string[]; + loggers?: string[]; + diagnostics?: string[]; + tags?: string[]; + policyFragments?: string[]; + gateways?: string[]; + versionSets?: string[]; + groups?: string[]; + subscriptions?: string[]; + schemas?: string[]; + policyRestrictions?: string[]; + documentations?: string[]; + workspaces?: string[]; } export interface PublishConfig { diff --git a/src/services/filter-service.ts b/src/services/filter-service.ts index 6dd666b..c9c9c22 100644 --- a/src/services/filter-service.ts +++ b/src/services/filter-service.ts @@ -16,21 +16,21 @@ import { getNamePart } from '../lib/resource-path.js'; * Map resource types to their corresponding FilterConfig field names. */ const FILTER_FIELD_MAP: Partial> = { - [ResourceType.Api]: 'apiNames', - [ResourceType.Backend]: 'backendNames', - [ResourceType.Product]: 'productNames', - [ResourceType.NamedValue]: 'namedValueNames', - [ResourceType.Logger]: 'loggerNames', - [ResourceType.Diagnostic]: 'diagnosticNames', - [ResourceType.Tag]: 'tagNames', - [ResourceType.PolicyFragment]: 'policyFragmentNames', - [ResourceType.Gateway]: 'gatewayNames', - [ResourceType.VersionSet]: 'versionSetNames', - [ResourceType.Group]: 'groupNames', - [ResourceType.Subscription]: 'subscriptionNames', - [ResourceType.GlobalSchema]: 'schemaNames', - [ResourceType.PolicyRestriction]: 'policyRestrictionNames', - [ResourceType.Documentation]: 'documentationNames', + [ResourceType.Api]: 'apis', + [ResourceType.Backend]: 'backends', + [ResourceType.Product]: 'products', + [ResourceType.NamedValue]: 'namedValues', + [ResourceType.Logger]: 'loggers', + [ResourceType.Diagnostic]: 'diagnostics', + [ResourceType.Tag]: 'tags', + [ResourceType.PolicyFragment]: 'policyFragments', + [ResourceType.Gateway]: 'gateways', + [ResourceType.VersionSet]: 'versionSets', + [ResourceType.Group]: 'groups', + [ResourceType.Subscription]: 'subscriptions', + [ResourceType.GlobalSchema]: 'schemas', + [ResourceType.PolicyRestriction]: 'policyRestrictions', + [ResourceType.Documentation]: 'documentations', }; /** @@ -38,23 +38,23 @@ const FILTER_FIELD_MAP: Partial> = { * If the parent (e.g., Api or Product) passes the filter, all children are included. */ const PARENT_FILTER_MAP: Partial> = { - [ResourceType.ApiPolicy]: 'apiNames', - [ResourceType.ApiTag]: 'apiNames', - [ResourceType.ApiDiagnostic]: 'apiNames', - [ResourceType.ApiOperation]: 'apiNames', - [ResourceType.ApiOperationPolicy]: 'apiNames', - [ResourceType.ApiSchema]: 'apiNames', - [ResourceType.ApiRelease]: 'apiNames', - [ResourceType.ApiTagDescription]: 'apiNames', - [ResourceType.ApiWiki]: 'apiNames', - [ResourceType.GraphQLResolver]: 'apiNames', - [ResourceType.GraphQLResolverPolicy]: 'apiNames', - [ResourceType.ProductPolicy]: 'productNames', - [ResourceType.ProductApi]: 'productNames', - [ResourceType.ProductGroup]: 'productNames', - [ResourceType.ProductTag]: 'productNames', - [ResourceType.ProductWiki]: 'productNames', - [ResourceType.GatewayApi]: 'gatewayNames', + [ResourceType.ApiPolicy]: 'apis', + [ResourceType.ApiTag]: 'apis', + [ResourceType.ApiDiagnostic]: 'apis', + [ResourceType.ApiOperation]: 'apis', + [ResourceType.ApiOperationPolicy]: 'apis', + [ResourceType.ApiSchema]: 'apis', + [ResourceType.ApiRelease]: 'apis', + [ResourceType.ApiTagDescription]: 'apis', + [ResourceType.ApiWiki]: 'apis', + [ResourceType.GraphQLResolver]: 'apis', + [ResourceType.GraphQLResolverPolicy]: 'apis', + [ResourceType.ProductPolicy]: 'products', + [ResourceType.ProductApi]: 'products', + [ResourceType.ProductGroup]: 'products', + [ResourceType.ProductTag]: 'products', + [ResourceType.ProductWiki]: 'products', + [ResourceType.GatewayApi]: 'gateways', }; /** @@ -108,7 +108,7 @@ export function shouldIncludeResource( function getParentNameForFilter(descriptor: ResourceDescriptor): string | undefined { const parentName = getNamePart(descriptor.nameParts, 0); // API children need revision suffix stripped (e.g. "my-api;rev=2" → "my-api") - return PARENT_FILTER_MAP[descriptor.type] === 'apiNames' + return PARENT_FILTER_MAP[descriptor.type] === 'apis' ? extractRootApiName(parentName) : parentName; } diff --git a/src/services/init-service.ts b/src/services/init-service.ts index b97adfb..31c8c59 100644 --- a/src/services/init-service.ts +++ b/src/services/init-service.ts @@ -198,7 +198,7 @@ class InitServiceImpl implements InitService { // Check for config files const filterConfig = path.join( config.outputDir, - 'configuration.extract.yaml' + 'configuration.extractor.yaml' ); if (await this.fileExists(filterConfig)) { conflictingFiles.push(filterConfig); @@ -375,9 +375,9 @@ class InitServiceImpl implements InitService { ): Promise { // Filter config const filterContent = generateFilterConfig(); - const filterPath = path.join(config.outputDir, 'configuration.extract.yaml'); + const filterPath = path.join(config.outputDir, 'configuration.extractor.yaml'); await fs.writeFile(filterPath, filterContent); - generatedFiles.configs.push('configuration.extract.yaml'); + generatedFiles.configs.push('configuration.extractor.yaml'); // Override configs for each environment for (const env of config.environments) { diff --git a/src/services/transitive-resolver.ts b/src/services/transitive-resolver.ts index 4abab24..4be4500 100644 --- a/src/services/transitive-resolver.ts +++ b/src/services/transitive-resolver.ts @@ -160,10 +160,10 @@ function addToFilter( dep: TransitiveDependency ): boolean { const fieldMap: Partial> = { - [ResourceType.NamedValue]: 'namedValueNames', - [ResourceType.Backend]: 'backendNames', - [ResourceType.PolicyFragment]: 'policyFragmentNames', - [ResourceType.VersionSet]: 'versionSetNames', + [ResourceType.NamedValue]: 'namedValues', + [ResourceType.Backend]: 'backends', + [ResourceType.PolicyFragment]: 'policyFragments', + [ResourceType.VersionSet]: 'versionSets', }; const field = fieldMap[dep.type]; diff --git a/src/services/workspace-extractor.ts b/src/services/workspace-extractor.ts index 1700217..b3317f3 100644 --- a/src/services/workspace-extractor.ts +++ b/src/services/workspace-extractor.ts @@ -49,8 +49,8 @@ export async function extractWorkspaces( ): Promise { const results: WorkspaceExtractionResult[] = []; let workspaceNames: string[]; - if (filter?.workspaceNames && filter.workspaceNames.length > 0) { - workspaceNames = filter.workspaceNames; + if (filter?.workspaces && filter.workspaces.length > 0) { + workspaceNames = filter.workspaces; } else { const discovered: string[] = []; for await (const item of client.listResources(context, ResourceType.Workspace)) { diff --git a/src/templates/azure-devops/extract-pipeline.ts b/src/templates/azure-devops/extract-pipeline.ts index bfc76ae..9ca9a33 100644 --- a/src/templates/azure-devops/extract-pipeline.ts +++ b/src/templates/azure-devops/extract-pipeline.ts @@ -21,7 +21,7 @@ parameters: default: 'Extract All APIs' values: - 'Extract All APIs' - - 'configuration.extract.yaml' + - 'configuration.extractor.yaml' - name: resourceGroup type: string displayName: 'Azure Resource Group' @@ -72,7 +72,7 @@ steps: --resource-group \${{ parameters.resourceGroup }} \\ --service-name \${{ parameters.serviceName }} \\ --output ${config.artifactDir} \\ - --filter configuration.extract.yaml \\ + --filter configuration.extractor.yaml \\ --subscription-id $(AZURE_SUBSCRIPTION_ID) - task: PublishPipelineArtifact@1 diff --git a/src/templates/configs/filter-config.ts b/src/templates/configs/filter-config.ts index 1615b57..3004266 100644 --- a/src/templates/configs/filter-config.ts +++ b/src/templates/configs/filter-config.ts @@ -2,7 +2,7 @@ // Licensed under the MIT license. /** * T047: Sample filter configuration template - * Generates a sample configuration.extract.yaml file + * Generates a sample configuration.extractor.yaml file */ export function generateFilterConfig(): string { @@ -10,81 +10,81 @@ export function generateFilterConfig(): string { # Customize this file to control which resources are extracted # Extract only specific APIs by name -# apiNames: +# apis: # - echo-api # - petstore-api # Extract only specific products -# productNames: +# products: # - starter # - unlimited # Extract only specific backends -# backendNames: +# backends: # - backend-api # - legacy-backend # Extract only specific named values -# namedValueNames: +# namedValues: # - api-key # - connection-string # Extract only specific loggers -# loggerNames: +# loggers: # - appinsights-logger # Extract only specific diagnostics -# diagnosticNames: +# diagnostics: # - applicationinsights # Extract only specific tags -# tagNames: +# tags: # - production # - external # Extract only specific policy fragments -# policyFragmentNames: +# policyFragments: # - rate-limit-fragment # - cors-fragment # Extract only specific gateways -# gatewayNames: +# gateways: # - default # - internal-gateway # Extract only specific version sets -# versionSetNames: +# versionSets: # - payments-v1 # Extract only specific groups -# groupNames: +# groups: # - administrators # Extract only specific subscriptions -# subscriptionNames: +# subscriptions: # - starter-subscription # Extract only specific schemas -# schemaNames: +# schemas: # - pet-schema # Extract only specific policy restrictions -# policyRestrictionNames: +# policyRestrictions: # - global-policy-restriction # Extract only specific documentations -# documentationNames: +# documentations: # - getting-started # Extract only specific workspaces -# workspaceNames: +# workspaces: # - dev-workspace # Filter behavior: # - Leave a section commented out to include ALL resources of that type # - Set a section to an empty array ([]) to exclude ALL resources of that type # Example: -# gatewayNames: [] -# subscriptionNames: [] +# gateways: [] +# subscriptions: [] `; } diff --git a/src/templates/github-actions/extract-workflow.ts b/src/templates/github-actions/extract-workflow.ts index b6c40c7..73cc7a5 100644 --- a/src/templates/github-actions/extract-workflow.ts +++ b/src/templates/github-actions/extract-workflow.ts @@ -29,7 +29,7 @@ on: type: choice options: - Extract All APIs - - configuration.extract.yaml + - configuration.extractor.yaml permissions: id-token: write @@ -96,7 +96,7 @@ jobs: --resource-group \${{ env.APIM_RESOURCE_GROUP }} \\ --service-name \${{ env.APIM_SERVICE_NAME }} \\ --output ${config.artifactDir} \\ - --filter configuration.extract.yaml + --filter configuration.extractor.yaml - name: Upload artifacts uses: actions/upload-artifact@v4 diff --git a/tests/unit/clients/artifact-store.test.ts b/tests/unit/clients/artifact-store.test.ts index f60d1d9..60c7c53 100644 --- a/tests/unit/clients/artifact-store.test.ts +++ b/tests/unit/clients/artifact-store.test.ts @@ -275,17 +275,17 @@ describe('ArtifactStore', () => { const result = await store.listResources(tmpDir); expect(result.length).toBeGreaterThanOrEqual(3); - const apiNames = result + const apis = result .filter((d) => d.type === ResourceType.Api) .map((d) => d.nameParts[0]) .sort(); - expect(apiNames).toContain('api1'); - expect(apiNames).toContain('api2'); + expect(apis).toContain('api1'); + expect(apis).toContain('api2'); - const productNames = result + const products = result .filter((d) => d.type === ResourceType.Product) .map((d) => d.nameParts[0]); - expect(productNames).toContain('prod1'); + expect(products).toContain('prod1'); }); }); diff --git a/tests/unit/lib/config-loader.test.ts b/tests/unit/lib/config-loader.test.ts index 88e7354..649d400 100644 --- a/tests/unit/lib/config-loader.test.ts +++ b/tests/unit/lib/config-loader.test.ts @@ -20,12 +20,12 @@ describe('config-loader', () => { describe('loadFilterConfig', () => { it('should load a valid filter YAML file', async () => { const content = ` -apiNames: +apis: - api1 - api2 -productNames: +products: - starter -tagNames: +tags: - v1 `; const filePath = path.join(tmpDir, 'filter.yaml'); @@ -33,9 +33,9 @@ tagNames: const config = await loadFilterConfig(filePath); expect(config).toBeDefined(); - expect(config!.apiNames).toEqual(['api1', 'api2']); - expect(config!.productNames).toEqual(['starter']); - expect(config!.tagNames).toEqual(['v1']); + expect(config!.apis).toEqual(['api1', 'api2']); + expect(config!.products).toEqual(['starter']); + expect(config!.tags).toEqual(['v1']); }); it('should return undefined for missing file', async () => { @@ -71,7 +71,7 @@ tagNames: it('should throw for invalid type (non-array field)', async () => { const content = ` -apiNames: "not-an-array" +apis: "not-an-array" `; const filePath = path.join(tmpDir, 'bad.yaml'); await fs.writeFile(filePath, content, 'utf-8'); @@ -81,7 +81,7 @@ apiNames: "not-an-array" it('should throw for array containing non-strings', async () => { const content = ` -apiNames: +apis: - 123 - true `; @@ -93,30 +93,30 @@ apiNames: it('should handle all filter fields', async () => { const content = ` -apiNames: [a] -backendNames: [b] -productNames: [c] -namedValueNames: [d] -loggerNames: [e] -diagnosticNames: [f] -tagNames: [g] -policyFragmentNames: [h] -gatewayNames: [i] -versionSetNames: [j] -groupNames: [k] -subscriptionNames: [l] -schemaNames: [m] -policyRestrictionNames: [n] -documentationNames: [o] -workspaceNames: [p] +apis: [a] +backends: [b] +products: [c] +namedValues: [d] +loggers: [e] +diagnostics: [f] +tags: [g] +policyFragments: [h] +gateways: [i] +versionSets: [j] +groups: [k] +subscriptions: [l] +schemas: [m] +policyRestrictions: [n] +documentations: [o] +workspaces: [p] `; const filePath = path.join(tmpDir, 'all-fields.yaml'); await fs.writeFile(filePath, content, 'utf-8'); const config = await loadFilterConfig(filePath); expect(config).toBeDefined(); - expect(config!.apiNames).toEqual(['a']); - expect(config!.workspaceNames).toEqual(['p']); + expect(config!.apis).toEqual(['a']); + expect(config!.workspaces).toEqual(['p']); }); }); diff --git a/tests/unit/services/api-product-extractor.test.ts b/tests/unit/services/api-product-extractor.test.ts index c70b01d..724cdfe 100644 --- a/tests/unit/services/api-product-extractor.test.ts +++ b/tests/unit/services/api-product-extractor.test.ts @@ -727,8 +727,8 @@ describe('api-extractor', () => { ); }); - it('should skip revision that does not match filter apiNames', async () => { - const filter: FilterConfig = { apiNames: ['other-api'] }; + it('should skip revision that does not match filter apis', async () => { + const filter: FilterConfig = { apis: ['other-api'] }; const client = createMockClient({ listApiRevisions: async function* () { yield { apiRevision: '2' }; diff --git a/tests/unit/services/delete-unmatched-service.test.ts b/tests/unit/services/delete-unmatched-service.test.ts index f440c29..bcf0860 100644 --- a/tests/unit/services/delete-unmatched-service.test.ts +++ b/tests/unit/services/delete-unmatched-service.test.ts @@ -114,11 +114,11 @@ describe('delete-unmatched-service', () => { const result = await computeDeleteActions(client, store, testContext, testConfig); - const groupNames = result.filter((d) => d.type === ResourceType.Group).map((d) => d.nameParts[0]); - expect(groupNames).not.toContain('administrators'); - expect(groupNames).not.toContain('developers'); - expect(groupNames).not.toContain('guests'); - expect(groupNames).toContain('custom-group'); + const groups = result.filter((d) => d.type === ResourceType.Group).map((d) => d.nameParts[0]); + expect(groups).not.toContain('administrators'); + expect(groups).not.toContain('developers'); + expect(groups).not.toContain('guests'); + expect(groups).toContain('custom-group'); }); it('should skip system products', async () => { @@ -136,11 +136,11 @@ describe('delete-unmatched-service', () => { const result = await computeDeleteActions(client, store, testContext, testConfig); - const productNames = result.filter((d) => d.type === ResourceType.Product).map((d) => d.nameParts[0]); - expect(productNames).not.toContain('master'); - expect(productNames).not.toContain('unlimited'); - expect(productNames).not.toContain('starter'); - expect(productNames).toContain('custom-product'); + const products = result.filter((d) => d.type === ResourceType.Product).map((d) => d.nameParts[0]); + expect(products).not.toContain('master'); + expect(products).not.toContain('unlimited'); + expect(products).not.toContain('starter'); + expect(products).toContain('custom-product'); }); it('should skip echo-api system API', async () => { @@ -156,9 +156,9 @@ describe('delete-unmatched-service', () => { const result = await computeDeleteActions(client, store, testContext, testConfig); - const apiNames = result.filter((d) => d.type === ResourceType.Api).map((d) => d.nameParts[0]); - expect(apiNames).not.toContain('echo-api'); - expect(apiNames).toContain('custom-api'); + const apis = result.filter((d) => d.type === ResourceType.Api).map((d) => d.nameParts[0]); + expect(apis).not.toContain('echo-api'); + expect(apis).toContain('custom-api'); }); it('should handle empty artifact store (nothing to delete)', async () => { diff --git a/tests/unit/services/extract-service.test.ts b/tests/unit/services/extract-service.test.ts index a50a2a1..6be2ca8 100644 --- a/tests/unit/services/extract-service.test.ts +++ b/tests/unit/services/extract-service.test.ts @@ -105,7 +105,7 @@ describe('extract-service', () => { }); const store = createMockStore(); - const filter: FilterConfig = { namedValueNames: ['nv-keep'] }; + const filter: FilterConfig = { namedValues: ['nv-keep'] }; const config: ExtractConfig = { service: testContext, outputDir: '/output', @@ -381,7 +381,7 @@ describe('extract-service', () => { service: testContext, outputDir: '/output', includeTransitive: true, - filter: { apiNames: [] }, // Trigger transitive resolution + filter: { apis: [] }, // Trigger transitive resolution logLevel: LogLevel.INFO, }; @@ -404,7 +404,7 @@ describe('extract-service', () => { service: testContext, outputDir: '/output', includeTransitive: true, - filter: { apiNames: [] }, + filter: { apis: [] }, logLevel: LogLevel.INFO, }; diff --git a/tests/unit/services/filter-service.test.ts b/tests/unit/services/filter-service.test.ts index 4d19bea..9911bf2 100644 --- a/tests/unit/services/filter-service.test.ts +++ b/tests/unit/services/filter-service.test.ts @@ -26,7 +26,7 @@ describe('filter-service', () => { }); it('should include resources when filter field is undefined', () => { - const filter: FilterConfig = { productNames: ['my-product'] }; + const filter: FilterConfig = { products: ['my-product'] }; const descriptor: ResourceDescriptor = { type: ResourceType.Api, nameParts: ['my-api'], @@ -35,7 +35,7 @@ describe('filter-service', () => { }); it('should exclude all resources when filter field is empty array', () => { - const filter: FilterConfig = { apiNames: [] }; + const filter: FilterConfig = { apis: [] }; const descriptor: ResourceDescriptor = { type: ResourceType.Api, nameParts: ['my-api'], @@ -44,7 +44,7 @@ describe('filter-service', () => { }); it('should include matching resources (case-insensitive)', () => { - const filter: FilterConfig = { apiNames: ['My-Api'] }; + const filter: FilterConfig = { apis: ['My-Api'] }; const descriptor: ResourceDescriptor = { type: ResourceType.Api, nameParts: ['my-api'], @@ -53,7 +53,7 @@ describe('filter-service', () => { }); it('should exclude non-matching resources', () => { - const filter: FilterConfig = { apiNames: ['other-api'] }; + const filter: FilterConfig = { apis: ['other-api'] }; const descriptor: ResourceDescriptor = { type: ResourceType.Api, nameParts: ['my-api'], @@ -62,7 +62,7 @@ describe('filter-service', () => { }); it('should match API revisions by root name', () => { - const filter: FilterConfig = { apiNames: ['my-api'] }; + const filter: FilterConfig = { apis: ['my-api'] }; const descriptor: ResourceDescriptor = { type: ResourceType.Api, nameParts: ['my-api;rev=2'], @@ -71,7 +71,7 @@ describe('filter-service', () => { }); it('should filter child resources by parent name', () => { - const filter: FilterConfig = { apiNames: ['my-api'] }; + const filter: FilterConfig = { apis: ['my-api'] }; // ApiPolicy child — nameParts: [apiName] const policyDescriptor: ResourceDescriptor = { @@ -89,7 +89,7 @@ describe('filter-service', () => { }); it('should filter grandchild resources by grandparent name', () => { - const filter: FilterConfig = { apiNames: ['my-api'] }; + const filter: FilterConfig = { apis: ['my-api'] }; // ApiOperationPolicy — nameParts: [apiName, opName], filter checks nameParts[0] const opPolicy: ResourceDescriptor = { @@ -106,7 +106,7 @@ describe('filter-service', () => { }); it('should filter product children by product name', () => { - const filter: FilterConfig = { productNames: ['starter'] }; + const filter: FilterConfig = { products: ['starter'] }; // ProductPolicy — nameParts: [productName] const productPolicy: ResourceDescriptor = { @@ -124,7 +124,7 @@ describe('filter-service', () => { it('should filter product children by parent name, not by child name (ProductApi)', () => { // nameParts[0] = productName for ProductApi - const filter: FilterConfig = { productNames: ['starter'] }; + const filter: FilterConfig = { products: ['starter'] }; // ProductApi with product='premium' should NOT match filter for 'starter' const productApi: ResourceDescriptor = { @@ -135,7 +135,7 @@ describe('filter-service', () => { }); it('should always include ServicePolicy', () => { - const filter: FilterConfig = { apiNames: [] }; + const filter: FilterConfig = { apis: [] }; const descriptor: ResourceDescriptor = { type: ResourceType.ServicePolicy, nameParts: [], @@ -144,7 +144,7 @@ describe('filter-service', () => { }); it('should filter named values', () => { - const filter: FilterConfig = { namedValueNames: ['my-secret'] }; + const filter: FilterConfig = { namedValues: ['my-secret'] }; const included: ResourceDescriptor = { type: ResourceType.NamedValue, nameParts: ['my-secret'], @@ -158,7 +158,7 @@ describe('filter-service', () => { }); it('should filter backends', () => { - const filter: FilterConfig = { backendNames: ['my-backend'] }; + const filter: FilterConfig = { backends: ['my-backend'] }; const included: ResourceDescriptor = { type: ResourceType.Backend, nameParts: ['my-backend'], @@ -167,7 +167,7 @@ describe('filter-service', () => { }); it('should filter gateways', () => { - const filter: FilterConfig = { gatewayNames: ['gw-1'] }; + const filter: FilterConfig = { gateways: ['gw-1'] }; const included: ResourceDescriptor = { type: ResourceType.Gateway, nameParts: ['gw-1'], @@ -176,7 +176,7 @@ describe('filter-service', () => { }); it('should filter gateway children by gateway name', () => { - const filter: FilterConfig = { gatewayNames: ['gw-1'] }; + const filter: FilterConfig = { gateways: ['gw-1'] }; const gwApi: ResourceDescriptor = { type: ResourceType.GatewayApi, nameParts: ['gw-1', 'my-api'], // nameParts[0]=gatewayName, nameParts[1]=apiName @@ -201,7 +201,7 @@ describe('filter-service', () => { }); it('should filter resources based on config', () => { - const filter: FilterConfig = { apiNames: ['api-1'] }; + const filter: FilterConfig = { apis: ['api-1'] }; const descriptors: ResourceDescriptor[] = [ { type: ResourceType.Api, nameParts: ['api-1'] }, { type: ResourceType.Api, nameParts: ['api-2'] }, diff --git a/tests/unit/services/init-service.test.ts b/tests/unit/services/init-service.test.ts index 2958a95..0078830 100644 --- a/tests/unit/services/init-service.test.ts +++ b/tests/unit/services/init-service.test.ts @@ -138,7 +138,7 @@ describe('init-service', () => { const result = await initService.run(config); - expect(result.configs).toContain('configuration.extract.yaml'); + expect(result.configs).toContain('configuration.extractor.yaml'); }); it('should generate override configuration for each environment', async () => { @@ -251,7 +251,7 @@ describe('init-service', () => { // Mock file exists for filter config and the CLI tarball vi.mocked(fs.access).mockImplementation(async (filePath: PathLike) => { const p = filePath.toString(); - if (p === TEST_CLI_PACKAGE_RESOLVED || p.includes('configuration.extract.yaml')) { + if (p === TEST_CLI_PACKAGE_RESOLVED || p.includes('configuration.extractor.yaml')) { return Promise.resolve(); } throw new Error('ENOENT'); @@ -276,7 +276,7 @@ describe('init-service', () => { // Mock file exists for filter config and the CLI tarball vi.mocked(fs.access).mockImplementation(async (filePath: PathLike) => { const p = filePath.toString(); - if (p === TEST_CLI_PACKAGE_RESOLVED || p.includes('configuration.extract.yaml')) { + if (p === TEST_CLI_PACKAGE_RESOLVED || p.includes('configuration.extractor.yaml')) { return Promise.resolve(); } throw new Error('ENOENT'); diff --git a/tests/unit/services/resource-extractor.test.ts b/tests/unit/services/resource-extractor.test.ts index 495daf3..5318011 100644 --- a/tests/unit/services/resource-extractor.test.ts +++ b/tests/unit/services/resource-extractor.test.ts @@ -93,7 +93,7 @@ describe('resource-extractor', () => { { name: 'nv-2', properties: {} }, ]); const store = createMockStore(); - const filter: FilterConfig = { namedValueNames: ['nv-1'] }; + const filter: FilterConfig = { namedValues: ['nv-1'] }; const result = await extractResourceType( client, store, testContext, diff --git a/tests/unit/services/transitive-resolver.test.ts b/tests/unit/services/transitive-resolver.test.ts index c4e3c96..f7a17ab 100644 --- a/tests/unit/services/transitive-resolver.test.ts +++ b/tests/unit/services/transitive-resolver.test.ts @@ -115,12 +115,12 @@ describe('transitive-resolver', () => { const apis = new Map>(); const filter: FilterConfig = { - apiNames: ['my-api'], - namedValueNames: [], // Start with empty — should be expanded + apis: ['my-api'], + namedValues: [], // Start with empty — should be expanded }; const expanded = resolveTransitiveDependencies(policies, apis, filter); - expect(expanded.namedValueNames).toContain('my-secret'); + expect(expanded.namedValues).toContain('my-secret'); }); it('should not add to undefined filter fields (unfiltered types)', () => { @@ -129,14 +129,14 @@ describe('transitive-resolver', () => { const apis = new Map>(); - // namedValueNames is undefined = all named values included + // namedValues is undefined = all named values included const filter: FilterConfig = { - apiNames: ['my-api'], + apis: ['my-api'], }; const expanded = resolveTransitiveDependencies(policies, apis, filter); // Should remain undefined (no need to add — all are already included) - expect(expanded.namedValueNames).toBeUndefined(); + expect(expanded.namedValues).toBeUndefined(); }); it('should not duplicate existing entries', () => { @@ -146,11 +146,11 @@ describe('transitive-resolver', () => { const apis = new Map>(); const filter: FilterConfig = { - namedValueNames: ['existing-secret'], + namedValues: ['existing-secret'], }; const expanded = resolveTransitiveDependencies(policies, apis, filter); - expect(expanded.namedValueNames).toEqual(['existing-secret']); + expect(expanded.namedValues).toEqual(['existing-secret']); }); }); diff --git a/tests/unit/services/workspace-extractor.test.ts b/tests/unit/services/workspace-extractor.test.ts index ed638a1..0dd91d7 100644 --- a/tests/unit/services/workspace-extractor.test.ts +++ b/tests/unit/services/workspace-extractor.test.ts @@ -52,7 +52,7 @@ describe('workspace-extractor', () => { seenTypes.push(type); }; const store = createMockStore(); - const filter: FilterConfig = { workspaceNames: ['ws-1'] }; + const filter: FilterConfig = { workspaces: ['ws-1'] }; await extractWorkspaces( client, store, testContext, '/output', filter @@ -99,7 +99,7 @@ describe('workspace-extractor', () => { it('should skip extraction when workspace names array is empty', async () => { const client = createMockClient(); const store = createMockStore(); - const filter: FilterConfig = { workspaceNames: [] }; + const filter: FilterConfig = { workspaces: [] }; const results = await extractWorkspaces( client, store, testContext, '/output', filter @@ -117,7 +117,7 @@ describe('workspace-extractor', () => { } }; const store = createMockStore(); - const filter: FilterConfig = { workspaceNames: ['ws-1'] }; + const filter: FilterConfig = { workspaces: ['ws-1'] }; const results = await extractWorkspaces( client, store, testContext, '/output', filter @@ -132,7 +132,7 @@ describe('workspace-extractor', () => { const client = createMockClient(); client.listResources = async function* () {}; const store = createMockStore(); - const filter: FilterConfig = { workspaceNames: ['ws-1', 'ws-2'] }; + const filter: FilterConfig = { workspaces: ['ws-1', 'ws-2'] }; const results = await extractWorkspaces( client, store, testContext, '/output', filter @@ -150,7 +150,7 @@ describe('workspace-extractor', () => { throw new Error('Workspace not found'); }; const store = createMockStore(); - const filter: FilterConfig = { workspaceNames: ['bad-ws'] }; + const filter: FilterConfig = { workspaces: ['bad-ws'] }; const results = await extractWorkspaces( client, store, testContext, '/output', filter @@ -170,7 +170,7 @@ describe('workspace-extractor', () => { }; client.getResource = vi.fn().mockResolvedValue(undefined); const store = createMockStore(); - const filter: FilterConfig = { workspaceNames: ['ws-1'] }; + const filter: FilterConfig = { workspaces: ['ws-1'] }; const results = await extractWorkspaces( client, store, testContext, '/output', filter @@ -191,7 +191,7 @@ describe('workspace-extractor', () => { }; client.getResource = vi.fn().mockResolvedValue(undefined); const store = createMockStore(); - const filter: FilterConfig = { workspaceNames: ['ws-1'] }; + const filter: FilterConfig = { workspaces: ['ws-1'] }; const results = await extractWorkspaces( client, store, testContext, '/output', filter diff --git a/tests/unit/templates/azure-devops/extract-pipeline.test.ts b/tests/unit/templates/azure-devops/extract-pipeline.test.ts index 6e83415..cf2c221 100644 --- a/tests/unit/templates/azure-devops/extract-pipeline.test.ts +++ b/tests/unit/templates/azure-devops/extract-pipeline.test.ts @@ -25,7 +25,7 @@ describe('azure-devops/extract-pipeline', () => { const pipeline = generateExtractPipeline({ artifactDir: './apim-artifacts' }); expect(pipeline).toContain('name: CONFIGURATION_YAML_PATH'); expect(pipeline).toContain("'Extract All APIs'"); - expect(pipeline).toContain("'configuration.extract.yaml'"); + expect(pipeline).toContain("'configuration.extractor.yaml'"); }); it('should include runtime parameters for resource group and service name', () => { diff --git a/tests/unit/templates/configs/config-templates.test.ts b/tests/unit/templates/configs/config-templates.test.ts index 3bc4ee7..03fb5dd 100644 --- a/tests/unit/templates/configs/config-templates.test.ts +++ b/tests/unit/templates/configs/config-templates.test.ts @@ -15,42 +15,42 @@ describe('configs/filter-config', () => { expect(config).toContain('# APIM Extract Filter Configuration'); }); - it('should include commented examples for apiNames', () => { + it('should include commented examples for apis', () => { const config = generateFilterConfig(); - expect(config).toContain('# apiNames:'); + expect(config).toContain('# apis:'); }); - it('should include commented examples for productNames', () => { + it('should include commented examples for products', () => { const config = generateFilterConfig(); - expect(config).toContain('# productNames:'); + expect(config).toContain('# products:'); }); - it('should include commented examples for backendNames', () => { + it('should include commented examples for backends', () => { const config = generateFilterConfig(); - expect(config).toContain('# backendNames:'); + expect(config).toContain('# backends:'); }); - it('should include commented examples for namedValueNames', () => { + it('should include commented examples for namedValues', () => { const config = generateFilterConfig(); - expect(config).toContain('# namedValueNames:'); + expect(config).toContain('# namedValues:'); }); - it('should include commented examples for policyFragmentNames', () => { + it('should include commented examples for policyFragments', () => { const config = generateFilterConfig(); - expect(config).toContain('# policyFragmentNames:'); + expect(config).toContain('# policyFragments:'); }); it('should include commented examples for all supported filter fields', () => { const config = generateFilterConfig(); const fields = [ - 'gatewayNames', - 'versionSetNames', - 'groupNames', - 'subscriptionNames', - 'schemaNames', - 'policyRestrictionNames', - 'documentationNames', - 'workspaceNames', + 'gateways', + 'versionSets', + 'groups', + 'subscriptions', + 'schemas', + 'policyRestrictions', + 'documentations', + 'workspaces', ]; fields.forEach((field) => { expect(config).toContain(`# ${field}:`); @@ -60,8 +60,8 @@ describe('configs/filter-config', () => { it('should document empty arrays as exclude-all behavior', () => { const config = generateFilterConfig(); expect(config).toContain('# - Set a section to an empty array ([]) to exclude ALL resources of that type'); - expect(config).toContain('# gatewayNames: []'); - expect(config).toContain('# subscriptionNames: []'); + expect(config).toContain('# gateways: []'); + expect(config).toContain('# subscriptions: []'); }); it('should not have any uncommented configuration by default', () => { diff --git a/tests/unit/templates/github-actions/extract-workflow.test.ts b/tests/unit/templates/github-actions/extract-workflow.test.ts index 98f8fd4..fc2340a 100644 --- a/tests/unit/templates/github-actions/extract-workflow.test.ts +++ b/tests/unit/templates/github-actions/extract-workflow.test.ts @@ -26,7 +26,7 @@ describe('github-actions/extract-workflow', () => { expect(workflow).toContain('CONFIGURATION_YAML_PATH:'); expect(workflow).toContain('type: choice'); expect(workflow).toContain('Extract All APIs'); - expect(workflow).toContain('configuration.extract.yaml'); + expect(workflow).toContain('configuration.extractor.yaml'); }); it('should include ENVIRONMENT choice input', () => { @@ -68,7 +68,7 @@ describe('github-actions/extract-workflow', () => { const lines = workflow.split('\n'); const withConfigStart = lines.findIndex((l) => l.includes('Run APIM Extract (With Configuration)')); const withConfigSection = lines.slice(withConfigStart).join('\n'); - expect(withConfigSection).toContain('--filter configuration.extract.yaml'); + expect(withConfigSection).toContain('--filter configuration.extractor.yaml'); }); it('should use custom artifact directory in extract command', () => { From 5eeb9961d342ddcd6f4250211c6cd9512843cb3b Mon Sep 17 00:00:00 2001 From: Peter Hauge Date: Sat, 6 Jun 2026 15:07:18 -0700 Subject: [PATCH 2/3] feat: accept legacy *Names filter keys as backward-compat aliases The config loader now accepts both Toolkit-style keys (e.g., apis, backends, versionSets) and legacy *Names keys (e.g., apiNames, backendNames, versionSetNames). Legacy keys emit a deprecation warning. Using both forms for the same field is an error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- src/lib/config-loader.ts | 94 ++++++++++++++-------------- tests/unit/lib/config-loader.test.ts | 57 +++++++++++++++++ 2 files changed, 103 insertions(+), 48 deletions(-) diff --git a/src/lib/config-loader.ts b/src/lib/config-loader.ts index 6a8f166..fdee928 100644 --- a/src/lib/config-loader.ts +++ b/src/lib/config-loader.ts @@ -34,61 +34,59 @@ function assertStringArray(value: unknown, fieldName: string): string[] { * Load and parse a filter configuration YAML file. * Returns undefined if file doesn't exist. */ +/** + * Mapping from FilterConfig field to its legacy alias (the old *Names key). + * Both the Toolkit-style key and the legacy alias are accepted during parsing. + */ +const FILTER_KEY_ALIASES: Record = { + apis: 'apiNames', + backends: 'backendNames', + products: 'productNames', + namedValues: 'namedValueNames', + loggers: 'loggerNames', + diagnostics: 'diagnosticNames', + tags: 'tagNames', + policyFragments: 'policyFragmentNames', + gateways: 'gatewayNames', + versionSets: 'versionSetNames', + groups: 'groupNames', + subscriptions: 'subscriptionNames', + schemas: 'schemaNames', + policyRestrictions: 'policyRestrictionNames', + documentations: 'documentationNames', + workspaces: 'workspaceNames', +}; + export async function loadFilterConfig(filePath: string): Promise { try { const content = await fs.readFile(filePath, 'utf-8'); const parsed = (yaml.load(content) ?? {}) as Record; - // Validate structure — each field must be an array of strings + // Validate structure — each field must be an array of strings. + // Accept both Toolkit-style keys (e.g. "apis") and legacy aliases (e.g. "apiNames"). const config: FilterConfig = {}; - if (parsed.apis !== undefined) { - config.apis = assertStringArray(parsed.apis, 'apis'); - } - if (parsed.backends !== undefined) { - config.backends = assertStringArray(parsed.backends, 'backends'); - } - if (parsed.products !== undefined) { - config.products = assertStringArray(parsed.products, 'products'); - } - if (parsed.namedValues !== undefined) { - config.namedValues = assertStringArray(parsed.namedValues, 'namedValues'); - } - if (parsed.loggers !== undefined) { - config.loggers = assertStringArray(parsed.loggers, 'loggers'); - } - if (parsed.diagnostics !== undefined) { - config.diagnostics = assertStringArray(parsed.diagnostics, 'diagnostics'); - } - if (parsed.tags !== undefined) { - config.tags = assertStringArray(parsed.tags, 'tags'); - } - if (parsed.policyFragments !== undefined) { - config.policyFragments = assertStringArray(parsed.policyFragments, 'policyFragments'); - } - if (parsed.gateways !== undefined) { - config.gateways = assertStringArray(parsed.gateways, 'gateways'); - } - if (parsed.versionSets !== undefined) { - config.versionSets = assertStringArray(parsed.versionSets, 'versionSets'); - } - if (parsed.groups !== undefined) { - config.groups = assertStringArray(parsed.groups, 'groups'); - } - if (parsed.subscriptions !== undefined) { - config.subscriptions = assertStringArray(parsed.subscriptions, 'subscriptions'); - } - if (parsed.schemas !== undefined) { - config.schemas = assertStringArray(parsed.schemas, 'schemas'); - } - if (parsed.policyRestrictions !== undefined) { - config.policyRestrictions = assertStringArray(parsed.policyRestrictions, 'policyRestrictions'); - } - if (parsed.documentations !== undefined) { - config.documentations = assertStringArray(parsed.documentations, 'documentations'); - } - if (parsed.workspaces !== undefined) { - config.workspaces = assertStringArray(parsed.workspaces, 'workspaces'); + for (const [field, legacyAlias] of Object.entries(FILTER_KEY_ALIASES)) { + const key = field as keyof FilterConfig; + const toolkitValue = parsed[field]; + const legacyValue = parsed[legacyAlias]; + + if (toolkitValue !== undefined && legacyValue !== undefined) { + throw new Error( + `Filter config contains both '${field}' and '${legacyAlias}'. ` + + `Use '${field}' (the APIOps Toolkit format).` + ); + } + + if (toolkitValue !== undefined) { + config[key] = assertStringArray(toolkitValue, field); + } else if (legacyValue !== undefined) { + logger.warn( + `Filter key '${legacyAlias}' is deprecated; use '${field}' instead ` + + `(APIOps Toolkit format).` + ); + config[key] = assertStringArray(legacyValue, legacyAlias); + } } logger.debug(`Loaded filter config from ${filePath}`); diff --git a/tests/unit/lib/config-loader.test.ts b/tests/unit/lib/config-loader.test.ts index 649d400..b0581cb 100644 --- a/tests/unit/lib/config-loader.test.ts +++ b/tests/unit/lib/config-loader.test.ts @@ -118,6 +118,63 @@ workspaces: [p] expect(config!.apis).toEqual(['a']); expect(config!.workspaces).toEqual(['p']); }); + + it('should accept legacy *Names keys as aliases', async () => { + const content = ` +apiNames: + - api1 + - api2 +productNames: + - starter +backendNames: + - backend1 +versionSetNames: + - vs1 +`; + const filePath = path.join(tmpDir, 'legacy.yaml'); + await fs.writeFile(filePath, content, 'utf-8'); + + const config = await loadFilterConfig(filePath); + expect(config).toBeDefined(); + expect(config!.apis).toEqual(['api1', 'api2']); + expect(config!.products).toEqual(['starter']); + expect(config!.backends).toEqual(['backend1']); + expect(config!.versionSets).toEqual(['vs1']); + }); + + it('should throw when both Toolkit and legacy keys are used for the same field', async () => { + const content = ` +apis: + - api1 +apiNames: + - api2 +`; + const filePath = path.join(tmpDir, 'conflict.yaml'); + await fs.writeFile(filePath, content, 'utf-8'); + + await expect(loadFilterConfig(filePath)).rejects.toThrow( + "contains both 'apis' and 'apiNames'" + ); + }); + + it('should accept a mix of Toolkit and legacy keys for different fields', async () => { + const content = ` +apis: + - api1 +backendNames: + - backend1 +versionSets: + - vs1 +`; + const filePath = path.join(tmpDir, 'mixed.yaml'); + await fs.writeFile(filePath, content, 'utf-8'); + + const config = await loadFilterConfig(filePath); + expect(config).toBeDefined(); + expect(config!.apis).toEqual(['api1']); + expect(config!.backends).toEqual(['backend1']); + expect(config!.versionSets).toEqual(['vs1']); + }); }); describe('loadOverrideConfig', () => { From fd1c93fd8ddd9862cc9bf4dce22529d696b93808 Mon Sep 17 00:00:00 2001 From: Peter Hauge Date: Sat, 6 Jun 2026 15:17:59 -0700 Subject: [PATCH 3/3] docs: replace v1/v2 terminology with APIOps Toolkit/apiops-cli Use "APIOps Toolkit" instead of "v1" and "apiops-cli" instead of "v2" throughout the migration guide for clarity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/guides/migration-from-v1.md | 52 ++++++++++++++++---------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/docs/guides/migration-from-v1.md b/docs/guides/migration-from-v1.md index 9ba902e..b37f7fb 100644 --- a/docs/guides/migration-from-v1.md +++ b/docs/guides/migration-from-v1.md @@ -1,23 +1,23 @@ -# Migration from v1 Toolkit +# Migration from APIOps Toolkit -Migrate from the [Azure/apiops](https://github.com/Azure/apiops) toolkit (v1) to apiops-cli (v2) — same concepts, simpler tooling, more features. +Migrate from the [APIOps Toolkit](https://github.com/Azure/apiops) to apiops-cli — same concepts, simpler tooling, more features. ## Why Migrate? -The v1 toolkit uses separate Extractor and Publisher binaries orchestrated by pipeline templates. It works, but: +The APIOps Toolkit uses separate Extractor and Publisher binaries orchestrated by pipeline templates. It works, but: - Requires Docker or the .NET SDK to run - Uses two separate configuration files and complex pipeline YAML - Supports ~20 resource types - Has no built-in dry-run, incremental publish, or scaffolding command -apiops-cli (v2) is a single Node.js CLI that covers the full workflow with less setup. +apiops-cli is a single Node.js CLI that covers the full workflow with less setup. --- ## Key Differences -| Feature | v1 (Azure/apiops) | v2 (apiops-cli) | +| Feature | APIOps Toolkit | apiops-cli | |---------|-------------------|-----------------| | **Runtime** | .NET SDK or Docker | Node.js 22+ | | **CLI** | Separate Extractor/Publisher binaries | Single `apiops` CLI | @@ -33,9 +33,9 @@ apiops-cli (v2) is a single Node.js CLI that covers the full workflow with less | **Resource types** | ~20 | 34 (see below) | | **Pipeline targets** | GitHub Actions, Azure DevOps | GitHub Actions, Azure DevOps | -### Additional resource types in v2 +### Additional resource types in apiops-cli -v2 supports all v1 resource types plus: `GlobalSchema`, `PolicyRestriction`, `Documentation`, `ApiSchema`, `ApiRelease`, `ApiTagDescription`, `ApiWiki`, `ProductWiki`, `GraphQLResolver`, `McpServer`, and more. +apiops-cli supports all APIOps Toolkit resource types plus: `GlobalSchema`, `PolicyRestriction`, `Documentation`, `ApiSchema`, `ApiRelease`, `ApiTagDescription`, `ApiWiki`, `ProductWiki`, `GraphQLResolver`, `McpServer`, and more. --- @@ -70,7 +70,7 @@ This creates: ### 3. Verify artifact compatibility -**Your existing extracted artifacts should work as-is with v2.** The artifact format is backward compatible — v2 reads the same `apiInformation.json`, `backendInformation.json`, `policy.xml`, and other files that v1 produces. +**Your existing extracted artifacts should work as-is with apiops-cli.** The artifact format is backward compatible — apiops-cli reads the same `apiInformation.json`, `backendInformation.json`, `policy.xml`, and other files that the APIOps Toolkit produces. Test by running a dry-run against your existing artifacts: @@ -86,11 +86,11 @@ If the dry-run shows the expected resources, your artifacts are compatible. ### 4. Update pipeline YAML -Replace the v1 pipeline tasks/actions with v2 CLI commands. +Replace the APIOps Toolkit pipeline tasks/actions with apiops-cli commands. #### GitHub Actions -**v1 (before):** +**APIOps Toolkit (before):** ```yaml - name: Run Publisher @@ -102,7 +102,7 @@ Replace the v1 pipeline tasks/actions with v2 CLI commands. CONFIGURATION_YAML_PATH: configuration.publisher.yaml ``` -**v2 (after):** +**apiops-cli (after):** ```yaml - name: Publish APIs @@ -116,7 +116,7 @@ Replace the v1 pipeline tasks/actions with v2 CLI commands. #### Azure DevOps -**v1 (before):** +**APIOps Toolkit (before):** ```yaml - task: AzureCLI@2 @@ -126,7 +126,7 @@ Replace the v1 pipeline tasks/actions with v2 CLI commands. --configuration-yaml-path configuration.publisher.yaml ``` -**v2 (after):** +**apiops-cli (after):** ```yaml - task: AzureCLI@2 @@ -145,7 +145,7 @@ Replace the v1 pipeline tasks/actions with v2 CLI commands. #### Extractor configuration -**v1** (`configuration.extractor.yaml`): +**APIOps Toolkit** (`configuration.extractor.yaml`): ```yaml apis: @@ -153,7 +153,7 @@ apis: - orders-api ``` -**v2** (`configuration.extractor.yaml` — same format and file name): +**apiops-cli** (`configuration.extractor.yaml` — same format and file name): ```yaml apis: @@ -161,7 +161,7 @@ apis: - orders-api ``` -The filter YAML format is fully compatible with v1. You can use your existing `configuration.extractor.yaml` as-is with the `--filter` flag: +The filter YAML format is fully compatible with the APIOps Toolkit. You can use your existing `configuration.extractor.yaml` as-is with the `--filter` flag: ```bash apiops extract \ @@ -172,9 +172,9 @@ apiops extract \ #### Publisher configuration -v1's `configuration.publisher.yaml` maps directly to v2's override files. The structure is the same: +The APIOps Toolkit's `configuration.publisher.yaml` maps directly to apiops-cli's override files. The structure is the same: -**v1:** +**APIOps Toolkit:** ```yaml namedValues: @@ -183,7 +183,7 @@ namedValues: value: "prod-value" ``` -**v2** (`overrides.prod.yaml` — same structure): +**apiops-cli** (`overrides.prod.yaml` — same structure): ```yaml namedValues: @@ -203,7 +203,7 @@ apiops publish \ ### 6. Test with dry-run -Before your first real publish with v2, always preview: +Before your first real publish with apiops-cli, always preview: ```bash apiops publish \ @@ -220,7 +220,7 @@ Review the output to confirm the correct resources would be created, updated, or ## New Features to Adopt -After migration, take advantage of v2-only capabilities: +After migration, take advantage of apiops-cli capabilities: ### Incremental publish @@ -252,7 +252,7 @@ apiops publish --format json ... | jq '.summary' ### Transitive dependency filtering -v2 automatically includes resources that your filtered APIs depend on (backends, named values, policy fragments). No need to manually list every dependency. +apiops-cli automatically includes resources that your filtered APIs depend on (backends, named values, policy fragments). No need to manually list every dependency. ```bash apiops extract --filter filter.yaml # includes deps by default @@ -276,10 +276,10 @@ apiops extract --cloud usgov ... |-------|-------|-----| | `apiops: command not found` | CLI not installed globally | Run `npm install -g @peterhauge/apiops-cli` | | Artifacts not recognized | Unexpected directory structure | Verify your artifacts follow the standard layout (`apis/{name}/apiInformation.json`, etc.) | -| Authentication fails in pipeline | v1 used service connection env vars; v2 uses `DefaultAzureCredential` | See [Authentication Guide](./authentication.md). For GitHub Actions, use `azure/login` with OIDC. For Azure DevOps, use `AzureCLI@2` task. | -| Override values not applied | Wrong override file format or path | Check YAML structure matches v2 format. Pass with `--overrides `. | -| Extra resources published | v2 supports more resource types than v1 | This is expected. v2 extracts additional resource types (e.g., `GlobalSchema`, `ApiWiki`). Review with `--dry-run`. | -| `--delete-unmatched` removes unexpected resources | v2 sees more resource types | Run `--dry-run --delete-unmatched` first. Consider using `--commit-id` for safer incremental deploys. | +| Authentication fails in pipeline | APIOps Toolkit used service connection env vars; apiops-cli uses `DefaultAzureCredential` | See [Authentication Guide](./authentication.md). For GitHub Actions, use `azure/login` with OIDC. For Azure DevOps, use `AzureCLI@2` task. | +| Override values not applied | Wrong override file format or path | Check YAML structure matches apiops-cli format. Pass with `--overrides `. | +| Extra resources published | apiops-cli supports more resource types than the APIOps Toolkit | This is expected. apiops-cli extracts additional resource types (e.g., `GlobalSchema`, `ApiWiki`). Review with `--dry-run`. | +| `--delete-unmatched` removes unexpected resources | apiops-cli sees more resource types | Run `--dry-run --delete-unmatched` first. Consider using `--commit-id` for safer incremental deploys. | ---