diff --git a/.env.template b/.env.template index 6bc94ea30..fe6f4587d 100644 --- a/.env.template +++ b/.env.template @@ -2,7 +2,7 @@ GITHUB_TOKEN= # Apigee proxy name to be used for test execution -# nhs-notify-supplier--internal-dev--nhs-notify-supplier-PR-XX +# nhs-notify-supplier--internal-dev--nhs-notify-supplier-prxxx PROXY_NAME= # APIM env to run e2e tests against, other options are: ref, int, prod @@ -25,17 +25,30 @@ TARGET_ENVIRONMENT=prxx # API Keys # ======== # In order to find out the value of an environments given API key, follow these steps -# 1. Log in to Non-Prod -# 2. Navigate to 'Publish' > 'Apps' and search for the app linked to authentication -# 3. Copy the "key" from the Credentials related to the app +# 1. Log in to the AWS NHS Notify Suppliers Dev account +# 2. Open the paramenter store and search for the parameter /dev/e2e/keys/apim/pr +# 3. Copy the decrypted "value" # Note: For INT and higher environments use developer portal https://identity.prod.api.platform.nhs.uk/ export NON_PROD_API_KEY=xxx export INTEGRATION_API_KEY=xxx export PRODUCTION_API_KEY=xxx + +# Status Endpoint API Key +# In order to find the value of the status endpoint API key, follow these steps: +# 1. Log in to the AWS NHS Notify Suppliers Dev account +# 2. Open the paramenter store and search for the parameter /dev/e2e/keys/apim/status +# 3. Copy the decrypted "value" export STATUS_ENDPOINT_API_KEY=xxx # Private Keys # ============ +# In order to set the NON_PROD_PRIVATE_KEY, follow these steps: +# 1. Log in to the AWS NHS Notify Suppliers Dev account +# 2. Open the paramenter store and search for the parameter /dev/e2e/keys/private +# 3. Copy the decrypted "value" +# 4. Create a .pem file and paste the value copied in step 3, +# save the file and provide the path to the file as the value for NON_PROD_PRIVATE_KEY +# for example /workspaces/nhs-notify-supplier-api/scripts/JWT/internal-dev-test-1.pem # private key path used to generate authentication for tests ran against the internal-dev and internal-qa export NON_PROD_PRIVATE_KEY=xxx # private key path used to generate authentication for tests ran against the int environment diff --git a/.gitignore b/.gitignore index b3fe5713f..d427e514e 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,7 @@ version.json # Please, add your custom content below! .idea .env +.devcontainer/devcontainer-lock.json # dependencies node_modules diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7ea5c619e..779967589 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,6 @@ -## Contributing to NHS Notify Supplier API + + +# Contributing to NHS Notify Supplier API ## Feature Branches @@ -45,3 +47,11 @@ GitHooks **must** be configured and run on commits before pushing to remote. Ref ## Testing Your Branch You can test your branch in a dynamic environment prior to merging to `main`. These are created as part of the `cicd-1-pull-request.yaml` workflow, triggered when a PR is created or updated. + +## Function Documentation + +Each Lambda and internal package has a `README.md` alongside the source describing its purpose, flow, integration points, and peculiarities. These are bundled into the docs site via `docs/generate-includes.sh`. + +When making changes to a Lambda or internal package, check whether the corresponding README needs updating. Function documentation is not auto-generated and can become stale if not maintained alongside code changes. + + diff --git a/README.md b/README.md index d8b7cf781..4286751ea 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,6 @@ This repository documents the Supplier API specification and provides an SDK wit - [Packages](#packages) - [Documentation](#documentation) - [SDK Assets](#sdk-assets) - - [Examples](#examples) - [API Developers](#api-developers) - [Setup](#setup) - [Prerequisites and Configuration](#prerequisites-and-configuration) @@ -64,10 +63,6 @@ If packages are unavailable the latest SDKs can be downloaded directly from: - TypeScript `sdk-ts-[Version].zip` - CSharp `sdk-csharp-[Version].zip` -### Examples - -TODO:CCM-11209 Links to example clients. - ## API Developers New developers of the NHS Notify Supplier API should understand the below. @@ -153,6 +148,14 @@ by default they will be available at [http://localhost:3050](http://localhost:30 These are generated using [https://hub.docker.com/r/openapitools/openapi-generator-cli](https://hub.docker.com/r/openapitools/openapi-generator-cli) +### Unit Testing + +Run unit tests from the repository root: + +```bash +npm run test:unit +``` + ### Documentation - You can preview the OAS locally by running `make serve-oas` diff --git a/config/suppliers/README.md b/config/suppliers/README.md new file mode 100644 index 000000000..9931f5555 --- /dev/null +++ b/config/suppliers/README.md @@ -0,0 +1,39 @@ + + +# Supplier Configuration + +## Purpose + +Static JSON configuration files that define the supplier allocation rules for the Supplier API. These are loaded into DynamoDB by infrastructure tooling and are queried at runtime by the `supplier-allocator` Lambda. Check relevant repositories (nhs-notify-internal, nhs-notify-supplier-config) as they orchestrate supplier config data ingress depending on target account, environment, etc. + +## Configuration Entities + +| Entity | Directory | Description | +| ----------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Supplier** | `supplier/` | Print supplier definitions with ID, name, channel type, daily capacity, and status (PROD/DRAFT) | +| **Letter Variant** | `letter-variant/` | Letter type definitions with physical constraints (sheets, sides, ink coverage, delivery days), associated pack specification IDs, and volume group assignment | +| **Volume Group** | `volume-group/` | Groupings of letter variants for allocation purposes, with status and date range validity | +| **Supplier Allocation** | `supplier-allocation/` | Maps a supplier to a volume group with a target `allocationPercentage` and status | +| **Pack Specification** | `pack-specification/` | Detailed print assembly specs (paper, envelope, print colour, duplex) with constraints and billing ID | +| **Supplier Pack** | `supplier-pack/` | Links a supplier to a pack specification with approval status | + +## Allocation Lookup Chain + +When the `supplier-allocator` Lambda processes a `LetterRequestPreparedEvent`: + +1. The event's `letterVariantId` identifies the **Letter Variant**. +2. The variant's `volumeGroupId` identifies the **Volume Group** (must be `PROD` status and within date range). +3. **Supplier Allocations** for that volume group determine which suppliers are eligible and their target allocation percentages (must sum to 100). +4. The variant's `packSpecificationIds` are filtered by the letter's physical constraints. +5. **Supplier Packs** confirm which eligible suppliers support the selected pack specification. +6. The supplier with the lowest weighted allocation factor (furthest below their target share) is selected. + +## Nuances and Peculiarities + +- These files are the source of truth for the supplier config DynamoDB table (`SUPPLIER_CONFIG_TABLE_NAME`). Changes here flow into DynamoDB via infrastructure deployment. +- Runtime persistence is event-driven: supplier-config events are routed through SQS to the `supplier-config-ingress` Lambda, which upserts records into the config table. +- `status: "PROD"` is required at multiple levels (supplier, volume group, allocation) for an allocation to be active. +- Volume groups have `startDate` (and optional `endDate`) fields. Allocations are only valid when the current date falls within this range (evaluated in London timezone). +- Supplier `dailyCapacity` is tracked separately in `SUPPLIER_QUOTAS_TABLE` and resets at midnight London time. It is not stored in these config files. + + diff --git a/docs/collections/_consumers/acceptance.md b/docs/collections/_consumers/acceptance.md index 2d38ae9f4..ee8d7dcd1 100644 --- a/docs/collections/_consumers/acceptance.md +++ b/docs/collections/_consumers/acceptance.md @@ -1,9 +1,15 @@ + + +# acceptance + --- + title: Acceptance Pack order: 0 nav_order: 3 has_children: false has_toc: false + --- ## Introduction and scope @@ -74,7 +80,7 @@ Validate that your system can functionally establish a secure, authenticated con **Endpoints:** -- GET /_status - Verify API health and authentication readiness +- GET /\_status - Verify API health and authentication readiness - GET /letters - Sample request to confirm authentication and API connectivity **Objectives:** @@ -83,11 +89,11 @@ Validate that your system can functionally establish a secure, authenticated con - Validate that a call to both status and letters endpoints return successful responses - Provide evidence of successful API calls -| Criteria | Description | -|---|---| -| **Steps** | 1. Authenticate with APIM
2. Send a successful request to GET /_status or any of the endpoints to confirm service availability
3. Call GET /letters to verify that the connection is working correctly
4. Confirm the API returns a successful response | -| **Acceptance** | - API connection successfully established for both endpoints
- No authentication or connectivity errors observed. | -| **Evidence** | Screenshot or API log showing successful responses from both endpoints | +| Criteria | Description | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Authenticate with APIM
2. Send a successful request to GET /\_status or any of the endpoints to confirm service availability
3. Call GET /letters to verify that the connection is working correctly
4. Confirm the API returns a successful response | +| **Acceptance** | - API connection successfully established for both endpoints
- No authentication or connectivity errors observed. | +| **Evidence** | Screenshot or API log showing successful responses from both endpoints | | **Business value** | Demonstrates that your organisation can connect securely to NHS Notify and that authentication is correctly implemented before processing any real letter data. Ensures compliance with NHS Digital security standards and protects patient information from unauthorised access. | ### AT2 - Receive and prepare letters for production @@ -115,16 +121,16 @@ Validate your system's ability to: - Retrieve the full list of 2,500 allocated letters - Ensure no skipping or missing records -| Criteria | Description | -|---|---| -| **Steps** | 1. Call GET /letters to query a list of letters that is ready to be printed. Parameter to use: limit = 2500
2. Record total count and confirm 2,500 unique letter IDs are retrieved. | -| **Acceptance** | - 2,500 unique letter IDs retrieved.
- No missing or duplicated IDs.
- Letters are retrieved in a single call | -| **Evidence** | API logs showing successful acknowledgement with total count of 2,500 unique letter IDs. | -| **Business value** | Confirms your production systems can begin each print run with all allocated letters, ensuring no delays or missed communications. | +| Criteria | Description | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Call GET /letters to query a list of letters that is ready to be printed. Parameter to use: limit = 2500
2. Record total count and confirm 2,500 unique letter IDs are retrieved. | +| **Acceptance** | - 2,500 unique letter IDs retrieved.
- No missing or duplicated IDs.
- Letters are retrieved in a single call | +| **Evidence** | API logs showing successful acknowledgement with total count of 2,500 unique letter IDs. | +| **Business value** | Confirms your production systems can begin each print run with all allocated letters, ensuring no delays or missed communications. | ### AT3 - Process letter list twice -This test validates that your system can process the same list of allocated letters multiple times without creating duplicate jobs, re-importing data or triggering re-prints. It is designed to prove that your integration correctly handles repeated API calls and that your processing logic is idempotent. +This test validates that your system can process the same list of allocated letters multiple times without creating duplicate jobs, re-importing data or triggering re-prints. It is designed to prove that your integration correctly handles repeated API calls and that your processing logic is idempotent. Please note that the steps described below are part of the test scenario and they are not meant to describe a business workflow. You may demonstrate this behaviour in any suitable way, but the steps below describe our recommended approach. @@ -150,12 +156,12 @@ Demonstrate that your system can: - Detect previously processed letter IDs and safely ignore duplicates - Maintain consistent data handling repeated calls -| Criteria | Description | -|---|---| -| **Steps** | 1. Call GET /letters?limit=2500 to retrieve the full list of allocated letters
2. Call GET /letters again immediately to retrieve the same list of letters without updating any statuses
3. Compare the second list to the first
4. Confirm that:
 a) The same 2,500 unique IDs are returned.
 b) No new processing jobs are triggered for already seen letters
 c) Your system ignores duplicates. | -| **Acceptance** | - Each letter is processed once.
- Duplicate retrievals do not cause re-printing, re-queuing, or duplicate API updates. | -| **Evidence** | - Retrieval logs showing both API calls and ID comparisons
- Processing logs showing no duplicate triggers
- Summary of total unique IDs vs. total retrieved records. | -| **Business value** | Confirms your system is idempotent — repeated API calls do not cause duplication, ensuring operational stability and preventing wasted print runs. | +| Criteria | Description | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Call GET /letters?limit=2500 to retrieve the full list of allocated letters
2. Call GET /letters again immediately to retrieve the same list of letters without updating any statuses
3. Compare the second list to the first
4. Confirm that:
 a) The same 2,500 unique IDs are returned.
 b) No new processing jobs are triggered for already seen letters
 c) Your system ignores duplicates. | +| **Acceptance** | - Each letter is processed once.
- Duplicate retrievals do not cause re-printing, re-queuing, or duplicate API updates. | +| **Evidence** | - Retrieval logs showing both API calls and ID comparisons
- Processing logs showing no duplicate triggers
- Summary of total unique IDs vs. total retrieved records. | +| **Business value** | Confirms your system is idempotent — repeated API calls do not cause duplication, ensuring operational stability and preventing wasted print runs. | ### AT4 – Acknowledge that you have accepted a list of letters @@ -174,15 +180,15 @@ Your system can mark all received letters as 'Accepted' for production, confirmi - Bulk status update using POST /letters - GET /letters/{id} to verify that the status updated - **Objective:** -Confirm that you can mark letters as ACCEPTED for production readiness. + **Objective:** + Confirm that you can mark letters as ACCEPTED for production readiness. -| Criteria | Description | -|---|---| -| **Steps** | 1. Call GET /letters?limit= 1000
2. Send POST /letters request with a single bulk payload
3. Verify via GET /letters/{id} that status updated to ACCEPTED.
4. Call GET /letters?limit=1000 and ensure that the letters are no longer in the pending queue. | -| **Acceptance** | - Targeted letters successfully updated from 'PENDING' status to 'ACCEPTED' status
- No letter is skipped
- API returns a successful response. | -| **Evidence** | - Include before and after status
- Include total update count. | -| **Business value** | Confirms that your system can inform NHS Notify of production readiness for each allocated letter. | +| Criteria | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Call GET /letters?limit= 1000
2. Send POST /letters request with a single bulk payload
3. Verify via GET /letters/{id} that status updated to ACCEPTED.
4. Call GET /letters?limit=1000 and ensure that the letters are no longer in the pending queue. | +| **Acceptance** | - Targeted letters successfully updated from 'PENDING' status to 'ACCEPTED' status
- No letter is skipped
- API returns a successful response. | +| **Evidence** | - Include before and after status
- Include total update count. | +| **Business value** | Confirms that your system can inform NHS Notify of production readiness for each allocated letter. | ### AT5 – Printed letters (Optional Status) @@ -207,12 +213,12 @@ Letters that were ACCEPTED are successfully printed and reported as PRINTED, pro **Objective** Prove your system records the point of physical print completion. -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify letters with status ACCEPTED
2. Send a PATCH /letters/{id} request to update targeted letters to PRINTED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to PRINTED. | -| **Acceptance** | - Only letters currently ACCEPTED are targeted
- All targeted letters are updated to PRINTED (no skips, no duplicates)
- API returns successful responses. | -| **Evidence** | API responses and before/after samples show correct transition. | -| **Business value** | Shows print workflow completion. | +| Criteria | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Steps** | 1. Identify letters with status ACCEPTED
2. Send a PATCH /letters/{id} request to update targeted letters to PRINTED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to PRINTED. | +| **Acceptance** | - Only letters currently ACCEPTED are targeted
- All targeted letters are updated to PRINTED (no skips, no duplicates)
- API returns successful responses. | +| **Evidence** | API responses and before/after samples show correct transition. | +| **Business value** | Shows print workflow completion. | ### AT6 – Enclosed letters (Optional Status) @@ -239,12 +245,12 @@ Letters that have been successfully printed and enclosed in envelopes are report **Objective:** Demonstrate that your system can record the completion of envelope-insertion and packaging steps by updating eligible letters to ENCLOSED and confirming the status change. -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify letters with status ACCEPTED or PRINTED
2. Send a PATCH /letters/{id} request to update targeted letters to ENCLOSED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to ENCLOSED. | -| **Acceptance** | - Only letters currently in ACCEPTED or PRINTED are targeted
- All targeted letters are updated to ENCLOSED
- API returns successful response. | -| **Evidence** | API responses and before/after samples show correct transition. | -| **Business value** | Ensures enclosures are tracked prior to dispatch. | +| Criteria | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Identify letters with status ACCEPTED or PRINTED
2. Send a PATCH /letters/{id} request to update targeted letters to ENCLOSED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to ENCLOSED. | +| **Acceptance** | - Only letters currently in ACCEPTED or PRINTED are targeted
- All targeted letters are updated to ENCLOSED
- API returns successful response. | +| **Evidence** | API responses and before/after samples show correct transition. | +| **Business value** | Ensures enclosures are tracked prior to dispatch. | ### AT7 – Dispatch letters @@ -272,12 +278,12 @@ Your system records the postal hand-off of each letter. **Objective:** Validate that your system promptly reports all dispatched letters by updating their status to DISPATCHED. -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify letters with status 'ACCEPTED', 'PRINTED', 'ENCLOSED'
2. Send a PATCH /letters/{id} request to update targeted letters to DISPATCHED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to DISPATCHED (This step is for acceptance testing and evidence, not a production requirement) | -| **Acceptance** | - Only letters currently in ACCEPTED, PRINTED, or ENCLOSED state are targeted
- All targeted letters are updated to DISPATCHED
- API returns a successful response. | -| **Evidence** | API responses and before/after samples show correct transition. | -| **Business value** | Demonstrates postal hand-off tracking. | +| Criteria | Description | +| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Identify letters with status 'ACCEPTED', 'PRINTED', 'ENCLOSED'
2. Send a PATCH /letters/{id} request to update targeted letters to DISPATCHED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to DISPATCHED (This step is for acceptance testing and evidence, not a production requirement) | +| **Acceptance** | - Only letters currently in ACCEPTED, PRINTED, or ENCLOSED state are targeted
- All targeted letters are updated to DISPATCHED
- API returns a successful response. | +| **Evidence** | API responses and before/after samples show correct transition. | +| **Business value** | Demonstrates postal hand-off tracking. | ### AT8 – Confirm Delivery (Optional Status) @@ -306,12 +312,12 @@ Letters that have completed postal delivery are updated to DELIVERED, confirming **Objectives:** Show that the printed letters have been delivered to patients -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify letters with status ACCEPTED, PRINTED, ENCLOSED, or DISPATCHED
2. Send a PATCH /letters/{id} request to update targeted letters to DELIVERED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to DELIVERED. | -| **Acceptance** | - Only letters currently in ACCEPTED, PRINTED, ENCLOSED, or DISPATCHED state are targeted
- All targeted letters are updated to DELIVERED
- API returns a successful response. | -| **Evidence** | API responses and before/after samples show correct transition. | -| **Business value** | Ensures that letters are delivered to patients. | +| Criteria | Description | +| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Identify letters with status ACCEPTED, PRINTED, ENCLOSED, or DISPATCHED
2. Send a PATCH /letters/{id} request to update targeted letters to DELIVERED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to DELIVERED. | +| **Acceptance** | - Only letters currently in ACCEPTED, PRINTED, ENCLOSED, or DISPATCHED state are targeted
- All targeted letters are updated to DELIVERED
- API returns a successful response. | +| **Evidence** | API responses and before/after samples show correct transition. | +| **Business value** | Ensures that letters are delivered to patients. | ### AT9 – Forward to a specialist printer @@ -340,12 +346,12 @@ Prove that your system can correctly: - Update their status to FORWARDED including specifying the reason why the letter was forwarded - Stop further local processing or dispatch -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify letters with status ACCEPTED and specification that require accessible format
2. Send a PATCH /letters/{id} request to update targeted letters to FORWARDED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to FORWARDED. | -| **Acceptance** | - All targeted letters are successfully updated to FORWARDED
- API returns a successful response.
- Forwarded letters are excluded from local printing, enclosing, or dispatch batches. | -| **Evidence** | API responses and before/after samples show correct transition. | -| **Business value** | Ensures that letters requiring accessible formats are correctly redirected to the relevant specialist supplier (e.g. RNIB). | +| Criteria | Description | +| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Identify letters with status ACCEPTED and specification that require accessible format
2. Send a PATCH /letters/{id} request to update targeted letters to FORWARDED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to FORWARDED. | +| **Acceptance** | - All targeted letters are successfully updated to FORWARDED
- API returns a successful response.
- Forwarded letters are excluded from local printing, enclosing, or dispatch batches. | +| **Evidence** | API responses and before/after samples show correct transition. | +| **Business value** | Ensures that letters requiring accessible formats are correctly redirected to the relevant specialist supplier (e.g. RNIB). | ### AT10 – Handle Returns @@ -374,7 +380,6 @@ Demonstrate that returned mail is correctly: - Identified and recorded as RETURNED - Associated with a valid return timestamp and reason as per this list: - - R01- Addressee gone away - R02- Address incomplete - R03- Address inaccessible @@ -388,12 +393,12 @@ Demonstrate that returned mail is correctly: - Logged for audit and address-quality feedback -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify letters with status DELIVERED OR DISPATCHED
2. Send a PATCH /letters/{id} request to update targeted letters to RETURNED or send a POST / letters for a bulk update
3. Verify via GET /letters/{id} that status updated to RETURNED. | -| **Acceptance** | - All targeted letters are successfully updated to RETURNED
- API returns a successful response. | -| **Evidence** | API logs showing state transitions, timestamps. | -| **Business value** | Supports address-quality feedback. | +| Criteria | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Identify letters with status DELIVERED OR DISPATCHED
2. Send a PATCH /letters/{id} request to update targeted letters to RETURNED or send a POST / letters for a bulk update
3. Verify via GET /letters/{id} that status updated to RETURNED. | +| **Acceptance** | - All targeted letters are successfully updated to RETURNED
- API returns a successful response. | +| **Evidence** | API logs showing state transitions, timestamps. | +| **Business value** | Supports address-quality feedback. | ### AT11 – Manage cancellations @@ -433,12 +438,12 @@ Demonstrate that your system: - Updates affected letters to CANCELLED status, including a code and a reason for cancellation - Prevents any further production, printing, or dispatch of those letters -| Criteria | Description | -|---|---| -| **Steps** | 1. Select list of letter IDs to be cancelled
2. Send a PATCH /letters/{id} request to update targeted letters to CANCELLED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to CANCELLED. | -| **Acceptance** | - Only letters not yet dispatched or delivered may be cancelled
- All targeted letters are successfully updated to CANCELLED
- API returns a successful response. | -| **Evidence** | API audit trail and before/after validation samples.
For this test, the key evidence is that you can notify us once your internal cancellation process is complete and that the API correctly reflects those cancellations. The test is not about the cancellation business process itself. | -| **Business value** | Confirms that suppliers can reliably report letter cancellations to NHS Notify once they have been actioned internally, supporting accurate lifecycle tracking and preparing the foundation for a future automated cancellation integration. | +| Criteria | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Steps** | 1. Select list of letter IDs to be cancelled
2. Send a PATCH /letters/{id} request to update targeted letters to CANCELLED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to CANCELLED. | +| **Acceptance** | - Only letters not yet dispatched or delivered may be cancelled
- All targeted letters are successfully updated to CANCELLED
- API returns a successful response. | +| **Evidence** | API audit trail and before/after validation samples.
For this test, the key evidence is that you can notify us once your internal cancellation process is complete and that the API correctly reflects those cancellations. The test is not about the cancellation business process itself. | +| **Business value** | Confirms that suppliers can reliably report letter cancellations to NHS Notify once they have been actioned internally, supporting accurate lifecycle tracking and preparing the foundation for a future automated cancellation integration. | ### AT12 – Handle failures @@ -476,12 +481,12 @@ Demonstrate that your system: - Records failure codes and reasons consistently - Communicates these to NHS Notify for transparency and incident reporting -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify the letters that failed production requirements
2. Send a PATCH /letters/{id} request to update targeted letters to FAILED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to FAILED. | -| **Acceptance** | - All targeted letters are successfully updated to FAILED
- API response confirms successful updates
- Failure code and failure reason are updated. | -| **Evidence** | API logs showing state transitions, timestamps. | -| **Business value** | Provides transparent incident reporting by informing of the reason why the letter cannot be processed. | +| Criteria | Description | +| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Identify the letters that failed production requirements
2. Send a PATCH /letters/{id} request to update targeted letters to FAILED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to FAILED. | +| **Acceptance** | - All targeted letters are successfully updated to FAILED
- API response confirms successful updates
- Failure code and failure reason are updated. | +| **Evidence** | API logs showing state transitions, timestamps. | +| **Business value** | Provides transparent incident reporting by informing of the reason why the letter cannot be processed. | ### AT13 – Reject invalid letters (Optional Status) @@ -520,12 +525,12 @@ Demonstrate that your system correctly: - Records the reason and reason code - Ensures NHS Notify is informed when a rejection occurs due to quota or compliance limits -| Criteria | Description | -|---|---| -| **Steps** | 1. Identify bad letter requests
2. Send a PATCH /letters/{id} request to update targeted letters to REJECTED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to REJECTED. | -| **Acceptance** | - All targeted letters are successfully updated to REJECTED
- API response confirms successful updates
- Rejected code and rejected reason are updated
- No rejected letter proceeds to ACCEPTED or any production status| -| **Evidence** | API logs showing state transitions, timestamps. | -| **Business value** | Ensures non-compliant inputs don't enter production.
Ensure NHS Notify is informed when supplier quotas are exhausted | +| Criteria | Description | +| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | 1. Identify bad letter requests
2. Send a PATCH /letters/{id} request to update targeted letters to REJECTED or send a POST /letters for a bulk update
3. Verify via GET /letters/{id} that status updated to REJECTED. | +| **Acceptance** | - All targeted letters are successfully updated to REJECTED
- API response confirms successful updates
- Rejected code and rejected reason are updated
- No rejected letter proceeds to ACCEPTED or any production status | +| **Evidence** | API logs showing state transitions, timestamps. | +| **Business value** | Ensures non-compliant inputs don't enter production.
Ensure NHS Notify is informed when supplier quotas are exhausted | ### AT14 – Submit and reconcile management information (MI) @@ -534,18 +539,18 @@ This test verifies that your system can submit accurate, complete MI that matche **Business outcome:** NHS Notify uses MI data for reconciliation, billing, and operational assurance. You must ensure each submission represents a unique data set and does not duplicate previous entries. -A duplicate MI entry is identified when multiple submissions are received with the same combination of reporting date and specification reference for the same groupID. +A duplicate MI entry is identified when multiple submissions are received with the same combination of reporting date and specification reference for the same groupID. **Endpoints:** - POST /mi -| Criteria | Description | -|---|---| -| **Steps** | - Submit MI for a known number of processed letters. The MI should be grouped by specification and group ID
- Reconcile counts and timestamps with your local records. | -| **Acceptance** | - MI entries appear in NHS Notify within expected timeframe
- Counts match processed letters
- No duplicates MI entries for the same date or missing records. | -| **Evidence** | MI payloads and responses. | -| **Business value** | Shows that operational and billing data are aligned and complete. | +| Criteria | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Steps** | - Submit MI for a known number of processed letters. The MI should be grouped by specification and group ID
- Reconcile counts and timestamps with your local records. | +| **Acceptance** | - MI entries appear in NHS Notify within expected timeframe
- Counts match processed letters
- No duplicates MI entries for the same date or missing records. | +| **Evidence** | MI payloads and responses. | +| **Business value** | Shows that operational and billing data are aligned and complete. | **Objectives**: @@ -586,12 +591,12 @@ To confirm that your system can within <=10 seconds: - Retrieve all letter metadata and specifications and download all associated PDFs -| Criteria | Description | -|---|---| -| **Steps** | 1. Retrieve letter list via GET /letters?limit=500
2. Mark the letters as accepted via POST /letters batch update
3. Download each letter's PDFs via GET /letters/{id}/data (signed URL expires after ~1 minute)
4. Check that file contents are complete
5. Fetch the remaining letters following steps 1-4 until no letters remain in the queue.

Execute the full flow three times and record the best result. | -| **Acceptance** | - 100% of PDFs downloaded successfully and verified
- All data retrieved within <=10 seconds under normal test load
- Complete test three times and record the best result. | -| **Evidence** | - Download logs with timestamps and total elapsed time. | -| **Business value** | Confirms your system can retrieve all required data for daily letter production quickly and reliably, meeting NHS Notify operational performance expectations and ensuring no printing delays. | +| Criteria | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Steps** | 1. Retrieve letter list via GET /letters?limit=500
2. Mark the letters as accepted via POST /letters batch update
3. Download each letter's PDFs via GET /letters/{id}/data (signed URL expires after ~1 minute)
4. Check that file contents are complete
5. Fetch the remaining letters following steps 1-4 until no letters remain in the queue.

Execute the full flow three times and record the best result. | +| **Acceptance** | - 100% of PDFs downloaded successfully and verified
- All data retrieved within <=10 seconds under normal test load
- Complete test three times and record the best result. | +| **Evidence** | - Download logs with timestamps and total elapsed time. | +| **Business value** | Confirms your system can retrieve all required data for daily letter production quickly and reliably, meeting NHS Notify operational performance expectations and ensuring no printing delays. | ## Evidence Submission @@ -615,29 +620,31 @@ Once you pass this stage, we'll confirm your readiness for going live. ### Letter Status Definitions -| Status | Description | Initiated By | Mandatory/Optional -|---|---|---|---| -| PENDING | Initial state for all new letters. Indicates that the letter has been allocated to a supplier but not yet retrieved or accepted | NHS Notify | Mandatory Starting Status | -| REJECTED | Used when a supplier determines a letter cannot be processed Occurs immediately after PENDING , before any production begins | Supplier | Conditional | -| ACCEPTED | Letter has passed validation checks and is ready for production | Supplier | Mandatory (core workflow transition) | -| PRINTED | Letter has been physically printed and is awaiting enclosure or dispatch. | Supplier | Optional (if print tracking supported) | -| ENCLOSED | The printed letter and any relevant enclosures have been inserted into the mailing envelope | Supplier | Optional (if enclosure tracking supported) | -| DISPATCHED | Letter has left the supplier's facility and been handed over to a postal service | Supplier | Mandatory for all successfully sent letters | -| DELIVERED | Letter has been delivered to the patient | Supplier | Optional (depends on delivery reporting integration) | -| RETURNED | Letter was undeliverable or returned by the postal service | Supplier | Mandatory | -| FORWARDED | Letter requires re-direction to a specialist supplier (e.g. RNIB or another accessible-format partner) | Supplier | Conditional (for accessible-format workflows) | -| CANCELLED | Letter was cancelled following a request from the NHS Notify team | Supplier on Notify request | Conditional (only via NHS Notify instruction) | -| FAILED | Letter could not complete production due to an unrecoverable issue | Supplier | Conditional (on error detection) | +| Status | Description | Initiated By | Mandatory/Optional | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | ---------------------------------------------------- | +| PENDING | Initial state for all new letters. Indicates that the letter has been allocated to a supplier but not yet retrieved or accepted | NHS Notify | Mandatory Starting Status | +| REJECTED | Used when a supplier determines a letter cannot be processed Occurs immediately after PENDING , before any production begins | Supplier | Conditional | +| ACCEPTED | Letter has passed validation checks and is ready for production | Supplier | Mandatory (core workflow transition) | +| PRINTED | Letter has been physically printed and is awaiting enclosure or dispatch. | Supplier | Optional (if print tracking supported) | +| ENCLOSED | The printed letter and any relevant enclosures have been inserted into the mailing envelope | Supplier | Optional (if enclosure tracking supported) | +| DISPATCHED | Letter has left the supplier's facility and been handed over to a postal service | Supplier | Mandatory for all successfully sent letters | +| DELIVERED | Letter has been delivered to the patient | Supplier | Optional (depends on delivery reporting integration) | +| RETURNED | Letter was undeliverable or returned by the postal service | Supplier | Mandatory | +| FORWARDED | Letter requires re-direction to a specialist supplier (e.g. RNIB or another accessible-format partner) | Supplier | Conditional (for accessible-format workflows) | +| CANCELLED | Letter was cancelled following a request from the NHS Notify team | Supplier on Notify request | Conditional (only via NHS Notify instruction) | +| FAILED | Letter could not complete production due to an unrecoverable issue | Supplier | Conditional (on error detection) | ### Possible Scenarios and Applicable Letter Statuses -| Scenario | Description | Typical Status flow | -|---|---|---| -| Standard print and delivery | Letter specification successfully provided, letter printed, dispatch and delivered to patient. | a) PENDING → ACCEPTED → PRINTED → ENCLOSED → DISPATCHED → DELIVERED
b) PENDING → ACCEPTED → PRINTED → DISPATCHED → DELIVERED
c) PENDING → ACCEPTED → ENCLOSED → DISPATCHED → DELIVERED
d) PENDING → ACCEPTED → DISPATCHED | -| Letter rejected by letter Supplier | Letter fails validation before acceptance by the letter supplier. | PENDING → REJECTED | -| Cancelled by client | Client cancels production before dispatched. | a) PENDING → ACCEPTED → CANCELLED
b) PENDING → ACCEPTED → PRINTED → CANCELLED
c) PENDING → ACCEPTED → PRINTED → ENCLOSED → CANCELLED | -| Production failure | Technical or system error events during the processing cycle. | a) PENDING → ACCEPTED → FAILED
b) PENDING → ACCEPTED → PRINTED → FAILED
c) PENDING → ACCEPTED → PRINTED → ENCLOSED → FAILED | -| Forward for accessible format | Letter sent to an alternative supplier (e.g., RNIB) for accessible format such as braille or large print. | PENDING → ACCEPTED → FORWARDED | -| Returned to production facility | Letter dispatched but returned by the postal partner. | a) PENDING → ACCEPTED → PRINTED → ENCLOSED → DISPATCHED → RETURNED (letter undeliverable)
b) PENDING → ACCEPTED → PRINTED → ENCLOSED → DISPATCHED → DELIVERED → RETURNED | +| Scenario | Description | Typical Status flow | +| ---------------------------------- | --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Standard print and delivery | Letter specification successfully provided, letter printed, dispatch and delivered to patient. | a) PENDING → ACCEPTED → PRINTED → ENCLOSED → DISPATCHED → DELIVERED
b) PENDING → ACCEPTED → PRINTED → DISPATCHED → DELIVERED
c) PENDING → ACCEPTED → ENCLOSED → DISPATCHED → DELIVERED
d) PENDING → ACCEPTED → DISPATCHED | +| Letter rejected by letter Supplier | Letter fails validation before acceptance by the letter supplier. | PENDING → REJECTED | +| Cancelled by client | Client cancels production before dispatched. | a) PENDING → ACCEPTED → CANCELLED
b) PENDING → ACCEPTED → PRINTED → CANCELLED
c) PENDING → ACCEPTED → PRINTED → ENCLOSED → CANCELLED | +| Production failure | Technical or system error events during the processing cycle. | a) PENDING → ACCEPTED → FAILED
b) PENDING → ACCEPTED → PRINTED → FAILED
c) PENDING → ACCEPTED → PRINTED → ENCLOSED → FAILED | +| Forward for accessible format | Letter sent to an alternative supplier (e.g., RNIB) for accessible format such as braille or large print. | PENDING → ACCEPTED → FORWARDED | +| Returned to production facility | Letter dispatched but returned by the postal partner. | a) PENDING → ACCEPTED → PRINTED → ENCLOSED → DISPATCHED → RETURNED (letter undeliverable)
b) PENDING → ACCEPTED → PRINTED → ENCLOSED → DISPATCHED → DELIVERED → RETURNED | PRINTED, ENCLOSED and DELIVERED are optional statuses. Suppliers that do not support these can directly progress from ACCEPTED to DISPATCHED. + + diff --git a/docs/collections/_developers/guides/function-readmes.md b/docs/collections/_developers/guides/function-readmes.md new file mode 100644 index 000000000..a945dad9c --- /dev/null +++ b/docs/collections/_developers/guides/function-readmes.md @@ -0,0 +1,132 @@ + + +# function-readmes + +--- + +title: Function Documentation +nav_order: 6 +parent: Developer Guides +has_children: false +has_toc: true + +--- + +This page bundles function-level README files from the Supplier API codebase. + +## Data Flow Overview + +Primary data flows passing through the system: + +### Inbound (letter allocation and creation) + +```text +LetterRequestPreparedEvent (SNS) + → supplier-allocator (SQS consumer) + → resolves supplier via weighted fair-share algorithm + → upsert-letter (SQS consumer) + → inserts letter record into DynamoDB (letters table) + → DynamoDB Stream → Kinesis + → update-letter-queue: adds PENDING letters to queue table + → letter-updates-transformer: publishes LetterStatusChangeEvent to SNS +``` + +### Supplier-facing (status updates) + +```text +Supplier calls GET /letters (api-handler) + → reads from pending queue table with visibility timeout +Supplier calls PATCH /letters/{id} or POST /letters (api-handler) + → enqueues UpdateLetterCommand to SQS + → transformAmendmentEvent (SQS consumer, in api-handler package) + → fetches current letter, publishes LetterStatusChangeEvent to SNS + → upsert-letter (SQS consumer) + → updates letter status in DynamoDB (letters table) + → DynamoDB Stream → Kinesis + → update-letter-queue: removes letter from pending queue + → letter-updates-transformer: publishes LetterStatusChangeEvent to SNS +``` + +### MI submission + +```text +Supplier calls POST /mi (api-handler) + → persists MI record to DynamoDB (mi table) + → DynamoDB Stream → Kinesis + → mi-updates-transformer: publishes MISubmittedEvent to SNS +``` + +### Supplier config ingestion + +```text +Supplier config event (SNS, type prefix uk.nhs.notify.supplier-config) + → supplier-config SQS queue + → supplier-config-ingress (SQS consumer) + → upserts entity into supplier-config DynamoDB table +``` + +## Lambda Packages + +### API Handler + +{% include components/generated/readmes/lambda-api-handler.md %} + +### Authorizer + +{% include components/generated/readmes/lambda-authorizer.md %} + +### Supplier Allocator + +{% include components/generated/readmes/lambda-supplier-allocator.md %} + +### Upsert Letter + +{% include components/generated/readmes/lambda-upsert-letter.md %} + +### Update Letter Queue + +{% include components/generated/readmes/lambda-update-letter-queue.md %} + +### Letter Updates Transformer + +{% include components/generated/readmes/lambda-letter-updates-transformer.md %} + +### MI Updates Transformer + +{% include components/generated/readmes/lambda-mi-updates-transformer.md %} + +### Supplier Config Ingress + +{% include components/generated/readmes/lambda-supplier-config-ingress.md %} + +## Internal Packages + +### Datastore + +{% include components/generated/readmes/internal-datastore.md %} + +### Events + +{% include components/generated/readmes/internal-events.md %} + +### Event Builders + +{% include components/generated/readmes/internal-event-builders.md %} + +### Helpers + +{% include components/generated/readmes/internal-helpers.md %} + +## Tests + +{% include components/generated/readmes/tests-overview.md %} + +## Sandbox + +{% include components/generated/readmes/sandbox.md %} + +## Supplier Configuration + +{% include components/generated/readmes/config-suppliers.md %} + + diff --git a/docs/generate-includes.sh b/docs/generate-includes.sh index c3b5649b3..75b217a97 100755 --- a/docs/generate-includes.sh +++ b/docs/generate-includes.sh @@ -7,8 +7,32 @@ cd "$(git rev-parse --show-toplevel)" mkdir -p ./docs/_includes/components/generated # Database mermaid diagrams +# NOTE: This also regenerates internal/datastore/src/types.md as a side effect. +# Review any changes to that file before committing. npm run -w internal/datastore diagrams cp ./internal/datastore/src/types.md ./docs/_includes/components/generated/types.md #Contributing file cp ./CONTRIBUTING.md ./docs/_includes/components/generated/contributing.md + +# Function documentation (lambdas) +mkdir -p ./docs/_includes/components/generated/readmes +cp ./lambdas/api-handler/README.md ./docs/_includes/components/generated/readmes/lambda-api-handler.md +cp ./lambdas/authorizer/README.md ./docs/_includes/components/generated/readmes/lambda-authorizer.md +cp ./lambdas/supplier-allocator/README.md ./docs/_includes/components/generated/readmes/lambda-supplier-allocator.md +cp ./lambdas/upsert-letter/README.md ./docs/_includes/components/generated/readmes/lambda-upsert-letter.md +cp ./lambdas/update-letter-queue/README.md ./docs/_includes/components/generated/readmes/lambda-update-letter-queue.md +cp ./lambdas/letter-updates-transformer/README.md ./docs/_includes/components/generated/readmes/lambda-letter-updates-transformer.md +cp ./lambdas/mi-updates-transformer/README.md ./docs/_includes/components/generated/readmes/lambda-mi-updates-transformer.md +cp ./lambdas/supplier-config-ingress/README.md ./docs/_includes/components/generated/readmes/lambda-supplier-config-ingress.md + +# Function documentation (internal packages) +cp ./internal/datastore/README.md ./docs/_includes/components/generated/readmes/internal-datastore.md +cp ./internal/events/README.md ./docs/_includes/components/generated/readmes/internal-events.md +cp ./internal/event-builders/README.md ./docs/_includes/components/generated/readmes/internal-event-builders.md +cp ./internal/helpers/README.md ./docs/_includes/components/generated/readmes/internal-helpers.md + +# Function documentation (other) +cp ./tests/README.md ./docs/_includes/components/generated/readmes/tests-overview.md +cp ./sandbox/README.md ./docs/_includes/components/generated/readmes/sandbox.md +cp ./config/suppliers/README.md ./docs/_includes/components/generated/readmes/config-suppliers.md diff --git a/internal/README.md b/internal/README.md new file mode 100644 index 000000000..c953721ab --- /dev/null +++ b/internal/README.md @@ -0,0 +1,22 @@ + + +# Internal Packages + +## Purpose + +This directory contains shared workspace packages that export code used by other packages in the repository (especially Lambda workspaces). + +These packages provide common schemas, datastore repositories, event builders, and helper utilities so implementation code can reuse a single source of truth. + +## What is here + +- `datastore/`: shared DynamoDB repositories, domain types, and related errors. +- `events/`: shared event schemas and TypeScript types. +- `event-builders/`: shared mappers/builders for CloudEvent payloads. +- `helpers/`: shared logging, metrics, environment, and utility helpers. + +## Usage + +Import from these packages in other workspaces rather than duplicating logic locally. + + diff --git a/internal/datastore/README.md b/internal/datastore/README.md new file mode 100644 index 000000000..ab9022baf --- /dev/null +++ b/internal/datastore/README.md @@ -0,0 +1,31 @@ + + +# @internal/datastore + +## Purpose + +Shared data-access layer providing DynamoDB repository implementations, domain types, and error classes for the supplier API. + +## General Structure + +- **`src/types.ts`**: Zod schemas and TypeScript types for `Letter`, `InsertLetter`, `UpdateLetter`, `PendingLetter`, `MI`, `Supplier`, and related entities. +- **`src/letter-repository.ts`**: `LetterRepository` — CRUD for the main letters DynamoDB table +- **`src/letter-queue-repository.ts`**: `LetterQueueRepository` — manages the pending letter queue projection table: + - `getLetters`: paginated query with visibility-timeout update to prevent duplicate dispatch. +- **`src/mi-repository.ts`**: `MIRepository` — writes MI records. +- **`src/supplier-repository.ts`**: `SupplierRepository` — looks up suppliers by APIM application ID / supplierId. +- **`src/supplier-config-repository.ts`**: `SupplierConfigRepository` — reads letter variants, volume groups, supplier allocations, pack specifications, and supplier packs from the config table. +- **`src/supplier-quotas-repository.ts`**: `SupplierQuotasRepository` — reads and writes daily and overall allocation counts used by the supplier-allocator. +- **`src/healthcheck.ts`**: `DBHealthcheck` — verifies DynamoDB table and S3 bucket connectivity. +- **`src/errors/`**: Common application errors, which determine some of the API's error responses. + +## Key Integration Points + +- **`@nhsdigital/nhs-notify-event-schemas-supplier-config`**: Zod schemas for supplier configuration entities. +- **Consumers**: every Lambda in this repository depends on this package. + +## Nuances and Peculiarities + +- The **letter queue table** is a separate DynamoDB table from the main letters table. It acts as a priority queue projection, maintained by the `update-letter-queue` Lambda from Kinesis stream events. +- `LetterQueueRepository` implements a **visibility timeout** pattern: each returned letter's `visibilityTimestamp` is updated so subsequent queries within the timeout window skip it, preventing duplicate dispatch. + diff --git a/internal/event-builders/README.md b/internal/event-builders/README.md index 6bd5e0882..9227635ce 100644 --- a/internal/event-builders/README.md +++ b/internal/event-builders/README.md @@ -1,7 +1,19 @@ # @internal/event-builders -Helper utilities to produce supplier api event types. +## Purpose -This package contains functions for constructing CloudEvent-compliant event payloads and related helpers. +Provides functions to construct CloudEvent-compliant payloads from domain entities. Centralises event-building logic so that multiple lambdas produce structurally identical events. -Independent package to allow for type imports across the project without circular dependencies. +## General Structure + +- **`src/letter-mapper.ts`**: exports `mapLetterToCloudEvent(letter, source)` which converts a `Letter` domain object into a full `LetterStatusChangeEvent` CloudEvent. + +## Key Integration Points + +- **Used by**: `letter-updates-transformer` (publish letter updates to core) and `amendment-event-transformer` (process incoming letter updates). + +## Nuances and Peculiarities + +- Each call to `mapLetterToCloudEvent` generates a fresh `randomUUID` for the event `id` and random bytes for `traceparent`, so the same letter domain object will produce a unique event every time it is called. +- The `data.origin.event` field is set to the **new event ID** (not the original event that created the letter), establishing a fresh trace link per emission. +- Independent package to allow for type imports across the project without circular dependencies. diff --git a/internal/events/README.md b/internal/events/README.md new file mode 100644 index 000000000..739b7376f --- /dev/null +++ b/internal/events/README.md @@ -0,0 +1,33 @@ + + +# @nhsdigital/nhs-notify-event-schemas-supplier-api + +## Purpose + +Defines the Zod schemas and TypeScript types for all CloudEvents produced and consumed within the supplier API domain. Published externally as `@nhsdigital/nhs-notify-event-schemas-supplier-api` and used by both internal lambdas and external consumers. + +## General Structure + +- **`src/domain/letter.ts`**: defines the `$Letter` schema (extends `DomainBase`) and `$LetterStatus`. +- **`src/domain/mi.ts`**: defines the `$MI` schema for management information records. +- **`src/events/event-envelope.ts`**: the `EventEnvelope` factory that creates the main schema for any domain entity. Generates `type` as `uk.nhs.notify.supplier-api...v1`, representing letter status change events and MI submissions. +- **`src/events/letter-events.ts`**: exports the `$LetterStatusChangeEvent` schema for letter lifecycle events. +- **`src/events/mi-events.ts`**: exports `$MISubmittedEvent` for MI submission CloudEvents. + +## Key Integration Points + +- `upsert-letter` uses `$LetterStatusChangeEvent` on its update path. The same handler uses letter-rendering schemas (`LetterRequestPreparedEvent` v1/v2) for inserts, so `LetterStatusChangeEvent` is the key discriminator for _update_ processing. +- `api-handler` (`transformAmendmentEvent`) emits `LetterStatusChangeEvent` after supplier update commands are accepted and enriched. +- `letter-updates-transformer` emits `LetterStatusChangeEvent` for downstream consumers when a letter is inserted or when `status` or `reasonCode` changes. +- Those published events are consumed through EventPub by Core (and any other downstream consumers). +- Published to npm for external consumers who need to parse or validate supplier API events. + +## Nuances and Peculiarities + +- **`LetterStatusChangeEvent` is a central domain event, not just a transport type.** It links supplier-facing updates, async persistence, and outbound publication to Core. +- **Supplier status updates become domain events.** When suppliers call `PATCH /letters/{id}` or `POST /letters`, the API handler enqueues `UpdateLetterCommand` messages. These are transformed into `LetterStatusChangeEvent` payloads and ultimately persisted by `upsert-letter`. +- **Outbound lifecycle publication uses the same event shape.** `letter-updates-transformer` emits `LetterStatusChangeEvent` on letter INSERT and on `status`/`reasonCode` updates, which are then routed to downstream subscribers. +- **Must remain free of internal dependencies.** Since this package is published externally, it cannot import from `@internal/datastore`, `@internal/helpers`, or any other internal workspace package. All types are self-contained. +- **Schema version alignment**: the `dataschemaversion` in CloudEvents is derived from this package's `package.json` version, so bumping the package version directly affects the schema version in all emitted events. There's a GH workflow related to this. + + diff --git a/internal/events/package.json b/internal/events/package.json index a2d9a8ba0..85531264f 100644 --- a/internal/events/package.json +++ b/internal/events/package.json @@ -36,5 +36,5 @@ "typecheck": "tsc --noEmit" }, "types": "dist/index.d.ts", - "version": "1.0.19" + "version": "1.0.20" } diff --git a/internal/helpers/README.md b/internal/helpers/README.md new file mode 100644 index 000000000..8475b0c90 --- /dev/null +++ b/internal/helpers/README.md @@ -0,0 +1,27 @@ + + +# @internal/helpers + +## Purpose + +Central shared utility package providing logging, CloudWatch metrics, environment validation, and common primitives used across all lambdas and internal packages. + +## General Structure highlights + +- **`src/logger.ts`**: `createLogger(options?)` — creates a `pino` logger with uppercase level labels and ISO timestamps. Used by every Lambda's dependency container. +- **`src/metrics.ts`**: CloudWatch Embedded Metric Format (EMF) utilities: + - `emitForSingleSupplier(metrics, functionName, supplierId, count, message, dimensions?)` — emits a metric dimensioned by supplier ID using `aws-embedded-metrics`. + - `buildEMFObject(functionName, dimensions, metric)` — builds a raw EMF JSON object for direct logging (used by stream-processing lambdas). + - `MetricEntry` interface and `MetricStatus` enum (`Success`/`Failure`). + +## Key Integration Points + +- Imported by every Lambda and by `@internal/datastore` and `@internal/event-builders`. +- `pino` is the only logging library used across the codebase. +- `aws-embedded-metrics` is used for CloudWatch metric emission in Lambda context. + +## Nuances and Peculiarities + +- Changes to this package affect the runtime behaviour of all lambdas. Keep interfaces stable and changes backward-compatible. + + diff --git a/lambdas/api-handler/README.md b/lambdas/api-handler/README.md new file mode 100644 index 000000000..8c88f9951 --- /dev/null +++ b/lambdas/api-handler/README.md @@ -0,0 +1,48 @@ + + +# API Handler Lambdas + +## Purpose + +The primary supplier-facing Lambdas behind API Gateway. Contains handlers with logic to process print suppliers' requests to retrieve pending letters, update letter statuses, download letter data, and submit management information (MI). + +## Exported Handlers + +Each export in `src/index.ts` maps to a separate API Gateway route: + +| Export | Endpoint | Description | +| ------------------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `getLetters` | `GET /letters` | Returns pending letters from the letter queue table with visibility-timeout locking | +| `getLetter` | `GET /letters/{id}` | Fetches a single letter by supplier and letter ID | +| `getLetterData` | `GET /letters/{id}/data` | Returns a 303 redirect to a time-limited S3 pre-signed URL for the letter PDF | +| `patchLetter` | `PATCH /letters/{id}` | Accepts a single letter status update, enqueues it to SQS | +| `postLetters` | `POST /letters` | Accepts a batch of letter status updates (up to `MAX_LIMIT`), enqueues them to SQS | +| `postMI` | `POST /mi` | Accepts a management information submission, persists it via `MIRepository` | +| `transformAmendmentEvent` | Supports patchLetters and postLetters | Processes queued `UpdateLetterCommand` messages, looks up the current letter, and publishes a `LetterStatusChangeEvent` to SNS | +| `getStatus` | `GET /_status` | Healthcheck that verifies DynamoDB and S3 connectivity | + +## General Flow + +1. A shared dependency container is created once at cold-start in `src/config/deps.ts` (DynamoDB, S3, SQS, SNS clients, repositories, env config). +2. Each handler extracts and validates common request identifiers (`nhsd-supplier-id`, `nhsd-correlation-id`, `x-request-id`) via `extractCommonIds`. +3. Request bodies are validated against Zod schemas in `src/contracts/letters.ts` and `src/contracts/mi.ts`. +4. For status updates (`PATCH`/`POST /letters`), the handler maps the request into `UpdateLetterCommand` objects and enqueues them. +5. The `transformAmendmentEvent` handler consumes those SQS messages, fetches the full letter from DynamoDB, builds a `LetterStatusChangeEvent` CloudEvent via `mapLetterToCloudEvent`, and publishes it to SNS for further processing (upsert). + +## Key Integration Points + +- **API Gateway**: Authorizer event model: supplier ID is injected by the authorizer Lambda as `principalId`. +- **DynamoDB**: `LetterRepository` for letter CRUD, `LetterQueueRepository` for pending letter queue reads with visibility timeout, `MIRepository` for MI writes. +- **S3**: pre-signed URL generation for letter PDF download via `getLetterDataUrl`. +- **SQS**: letter status update commands are batched and enqueued; `transformAmendmentEvent` reads from the same queue. +- **SNS**: `transformAmendmentEvent` publishes `LetterStatusChangeEvent` CloudEvents. + +## Nuances and Peculiarities + +- `GET /letters` reads from the **letter queue table** (not the main letters table) and updates each returned letter's `visibilityTimestamp`. Subsequent calls within the timeout window return an empty set, preventing duplicate dispatch to the same supplier. +- `GET /letters/{id}/data` returns HTTP 303 (not 200) with the pre-signed URL in the `Location` header. +- `POST /letters` enforces a duplicate letter ID check within a single request and a configurable max batch size (`MAX_LIMIT`). +- `transformAmendmentEvent` is an SQS-triggered handler bundled in the same Lambda package, not an API Gateway route. It bridges the async status update flow by looking up current letter state before publishing the event. +- Error responses are centralised through `processError` in the error mapper to produce a consistent JSON:API error shape across all endpoints. + + diff --git a/lambdas/authorizer/README.md b/lambdas/authorizer/README.md new file mode 100644 index 000000000..c0dd3bb44 --- /dev/null +++ b/lambdas/authorizer/README.md @@ -0,0 +1,32 @@ + + +# Authorizer Lambda + +## Purpose + +API Gateway "REQUEST authorizer" that maps an APIM application identity to an internal supplier ID and returns an IAM Allow/Deny policy. Also monitors client certificate expiry. + +## General Flow + +1. API Gateway invokes the authorizer with the full request context including headers and mTLS client certificate. +2. The handler checks the client certificate expiry against `CLIENT_CERTIFICATE_EXPIRATION_ALERT_DAYS`. If the certificate is near expiry, a CloudWatch metric (`apim-client-certificate-near-expiry`) is emitted and a warning is logged. +3. The APIM supplier ID header (configured via `APIM_SUPPLIER_ID_HEADER`) is extracted with a **case-insensitive** lookup across all request headers. +4. `SupplierRepository.getSupplierByApimId()` resolves the APIM application ID to a `Supplier` record from DynamoDB. +5. If the supplier is found and status is not `DISABLED`, an Allow policy is returned with `principalId` set to the internal supplier ID, making it available to all downstream Lambda handlers. +6. If the header is missing, the supplier is not found, or the supplier is `DISABLED`, a Deny policy is returned. + +## Key Integration Points + +- **API Gateway**: REQUEST authorizer event model with callback-style response (not async return). +- **`SupplierRepository`** from `@internal/datastore`: resolves APIM application IDs to supplier records. +- **CloudWatch Embedded Metrics**: certificate expiry alerts via `metricScope`. + +## Nuances and Peculiarities + +- The handler uses the **callback pattern** because API Gateway REQUEST authorizers require it, not async return. +- Header matching is case-insensitive (`headerName.toLowerCase()`) to handle inconsistencies in header casing. +- Disabled suppliers are explicitly denied even if the APIM ID lookup succeeds. +- Certificate expiry checking is fire-and-forget; it does not affect the Allow/Deny decision. +- The `principalId` in the Allow policy is the **internal supplier ID** (not the APIM application supplier ID), so all downstream Lambdas receive the resolved identity via `event.requestContext.authorizer.principalId`. + + diff --git a/lambdas/letter-updates-transformer/README.md b/lambdas/letter-updates-transformer/README.md new file mode 100644 index 000000000..d701e3ca5 --- /dev/null +++ b/lambdas/letter-updates-transformer/README.md @@ -0,0 +1,32 @@ + + +# Letter Updates Transformer Lambda + +## Purpose + +Publishes `LetterStatusChangeEvent` CloudEvents to SNS whenever a letter's status or reason code changes in the letters DynamoDB table. This is the primary mechanism by which downstream NHS Notify Bounded Contexts (Core) learn about letter lifecycle transitions. + +## General Flow + +1. The Lambda receives a Kinesis batch of DynamoDB stream images from the letters table. +2. A filter determines which records warrant an event: + - **INSERT** events always pass (a new letter was created). + - **MODIFY** events pass only if `status` **or** `reasonCode` changed between old and new images. + - REMOVE events and status-unchanged MODIFYs are discarded. +3. Qualifying letters are mapped to `LetterStatusChangeEvent`. +4. Events are published to SNS - eventPub. +5. Metrics are emitted per event type (e.g., `letter.ACCEPTED`, `letter.DISPATCHED`) to track the volume of each status transition. + +## Key Integration Points + +- **Kinesis**: DynamoDB Streams from the letters table, providing ordered change records per partition. +- **SNS**: The eventPub is the target topic for `LetterStatusChangeEvent` CloudEvents which are then consumed by Core. + +## Nuances and Peculiarities + +- The filter checks both `status` and `reasonCode` for changes. A MODIFY that only updates `reasonCode` (e.g., adding a reason to an existing status) will still emit an event. +- **REJECTED letters do emit events.** A letter inserted with status REJECTED will match the INSERT filter and produce a `letter.REJECTED` CloudEvent. Downstream consumers will see it immediately. +- The `dataschemaversion` in each CloudEvent is dynamically set from the `@nhsdigital/nhs-notify-event-schemas-supplier-api` package version, ensuring schema version alignment with the published package. +- Metrics are aggregated by event type string (e.g., `letter.PENDING`) so operational dashboards can monitor the frequency of each transition. + + diff --git a/lambdas/mi-updates-transformer/README.md b/lambdas/mi-updates-transformer/README.md new file mode 100644 index 000000000..c6efb0312 --- /dev/null +++ b/lambdas/mi-updates-transformer/README.md @@ -0,0 +1,30 @@ + + +# MI Updates Transformer Lambda + +## Purpose + +Publishes `MISubmittedEvent` CloudEvents to SNS whenever new management information records are inserted into the MI DynamoDB table via supplier PUT requests. + +## General Flow + +1. The Lambda receives a Kinesis batch of DynamoDB stream images from the MI table. +2. Only `INSERT` events are processed. MODIFY and REMOVE events are filtered out. +3. Each new MI record is unmarshalled against `MISchema` from `@internal/datastore` and mapped to an `MISubmittedEvent` CloudEvent via `mapMIToCloudEvent`. +4. Events are published to SNS. +5. Metrics are emitted per event type. + +## Key Integration Points + +- **Kinesis**: DynamoDB Streams from the MI table. +- **SNS**: EventPub being the target topic for `MISubmittedEvent` CloudEvents. +- **`MISubmittedEvent`** from `@nhsdigital/nhs-notify-event-schemas-supplier-api`: defines the CloudEvents envelope for MI submissions. +- **`@internal/helpers`**: `buildEMFObject` for CloudWatch EMF metrics. + +## Nuances and Peculiarities + +- Only INSERT events produce CloudEvents. MI records are immutable once written, so MODIFY and REMOVE are not expected in normal operation and are silently discarded. +- The handler structure mirrors `letter-updates-transformer` closely (same batching, decoding, and metrics patterns), but the filter is simpler (INSERT-only vs. status-change detection). +- The `mapMIToCloudEvent` mapper is local to this package (`src/mappers/mi-mapper.ts`) rather than shared in `@internal/event-builders`, since MI events have a different domain structure. + + diff --git a/lambdas/supplier-allocator/README.md b/lambdas/supplier-allocator/README.md new file mode 100644 index 000000000..f33e1517c --- /dev/null +++ b/lambdas/supplier-allocator/README.md @@ -0,0 +1,34 @@ + + +# Supplier Allocator Lambda + +## Purpose + +Consumes `LetterRequestPrepared` events (v1 and v2) from an SQS queue, chooses a supplier using configuration and quota data, and forwards the allocation result to the upsert-letter queue. + +## General Flow + +1. The handler receives an SQS batch of letter-request events. +2. Each record body is parsed and validated as either `$LetterRequestPreparedEventV2` or `$LetterRequestPreparedEvent` (v1 fallback). +3. The allocator loads the relevant supplier configuration from `SUPPLIER_CONFIG_TABLE`, including the letter variant, active volume group, candidate suppliers, and compatible pack details. +4. Candidate suppliers are filtered using pack support and daily capacity, then ranked using quota data from `SUPPLIER_QUOTAS_TABLE`. +5. On success, the handler produces an allocation with `allocationStatus.status = "PENDING"`. If allocation cannot be completed, it produces a REJECTED allocation with a failure reason instead of dropping the message. +6. Each record produces a `{ letterEvent, allocationDetails }` message sent to `UPSERT_LETTERS_QUEUE_URL`. +7. After the batch completes, allocation counters are written back to `SUPPLIER_QUOTAS_TABLE`, and only genuine processing failures are returned as `batchItemFailures`. + +## Key Integration Points + +- **SQS**: Input from EventSub, output to the upsert-letter queue (`UPSERT_LETTERS_QUEUE`). +- **`SupplierConfigRepository`** from `@internal/datastore` (`SUPPLIER_CONFIG_TABLE`): reads letter variants, volume groups, supplier allocations, pack specifications, and supplier packs. +- **`SupplierQuotasRepository`** from `@internal/datastore` (`SUPPLIER_QUOTAS_TABLE`): reads and writes daily and overall allocation counts per volume group and supplier. +- **Event schemas**: `@nhsdigital/nhs-notify-event-schemas-letter-rendering` (v2) and `@nhsdigital/nhs-notify-event-schemas-letter-rendering-v1` (v1). +- **Downstream consumer**: `upsert-letter` receives `{ letterEvent, allocationDetails }` and persists either a PENDING or REJECTED letter. + +## Nuances and Peculiarities + +- **Failed allocations produce REJECTED letters, not dropped messages.** If the config lookup chain fails for any reason, the handler still sends a message to the upsert queue with `allocationStatus.status = "REJECTED"` and `supplierId = "unknown"`. No letters are silently lost. +- **The factor algorithm is a running weighted average across the lifetime of the system, not per-batch.** The `overallAllocation` table accumulates counts since deployment. A supplier that handled a disproportionate share yesterday will have a high factor today and be deprioritised, allowing others to catch up to their target percentage. +- **Daily capacity check uses London timezone.** `format(toZonedTime(new Date(), "Europe/London"), "yyyy-MM-dd")` determines the date key. Capacity resets at midnight London time, not UTC. +- **Quota updates happen after the entire batch completes**, not per-record. This optimisation means concurrent Lambda invocations can transiently over-allocate a supplier before quotas are reconciled. + + diff --git a/lambdas/supplier-config-ingress/README.md b/lambdas/supplier-config-ingress/README.md new file mode 100644 index 000000000..b8827390a --- /dev/null +++ b/lambdas/supplier-config-ingress/README.md @@ -0,0 +1,41 @@ + + +# Supplier Config Ingress Lambda + +## Purpose + +Consumes supplier-config events from SQS and upserts supplier configuration entities into the supplier config DynamoDB table. + +## General Flow + +1. The handler receives a batch of SQS messages from the supplier-config queue. +2. The entity is extracted from the event type (for example `uk.nhs.notify.supplier-config.supplier-api.letter-variant.updated`). +3. The payload is validated using the schema mapped to that entity type. +4. The entity is upserted into `SUPPLIER_CONFIG_TABLE` via `SupplierConfigRepository.upsertSupplierConfig`. +5. EMF metrics are emitted for success (`result=CREATED|UPDATED`) and failure. +6. Failed records are returned in `batchItemFailures` so SQS retries only those messages. + +## Supported Entities + +- `letter-variant` +- `volume-group` +- `supplier-allocation` +- `supplier` +- `pack-specification` +- `supplier-pack` + +## Key Integration Points + +- **Input queue**: SQS queue subscribed to EventSub SNS topic with a message-body filter (`type` prefix `uk.nhs.notify.supplier-config`). +- **`SupplierConfigRepository`** from `@internal/datastore`: performs DynamoDB upserts by entity and ID. +- **Entity schemas**: `@nhsdigital/nhs-notify-event-schemas-supplier-config` validates each supported entity payload. +- **Observability**: `@internal/helpers` EMF metrics under namespace `supplier-config-ingress`. + +## Nuances and Peculiarities + +- The handler derives the entity from the event type by splitting on `.` and reading element index 4, so the event type format is significant. +- Upsert uses DynamoDB `UpdateItem` and returns whether a record was created or updated based on previous item existence. +- Only records that fail parsing or upsert are retried because of `ReportBatchItemFailures` semantics. +- Unknown or malformed event types are reported with failure metrics using `entity=unknown`. + + diff --git a/lambdas/update-letter-queue/README.md b/lambdas/update-letter-queue/README.md new file mode 100644 index 000000000..3871d26dd --- /dev/null +++ b/lambdas/update-letter-queue/README.md @@ -0,0 +1,32 @@ + + +# Update Letter Queue Lambda + +## Purpose + +Maintains the pending letter queue DynamoDB table as a projection of the main letters table. Processes Kinesis stream records from the letters table's DynamoDB Streams and adds or removes letters from the queue table based on status transitions. + +## General Flow + +1. The Lambda receives a Kinesis batch. Each record's `data` field is a base64-encoded DynamoDB stream image containing `eventName` (INSERT, MODIFY, REMOVE) and the old/new images. +2. For each record, two checks are applied: + - **`isNewPendingLetter`**: the record is an INSERT with `NewImage.status = PENDING`. The letter is added to the queue table via `LetterQueueRepository.putLetter`. + - **`isNoLongerPending`**: the record is a MODIFY where `OldImage.status = PENDING` and `NewImage.status != PENDING`. The letter is removed from the queue table via `LetterQueueRepository.deleteLetter`. +3. Records that match neither condition are skipped (REMOVE events, MODIFY with no PENDING transition, inserts with non-PENDING status such as REJECTED). +4. Success/failure delta counts are tracked per `supplierId` and emitted as CloudWatch EMF metrics. +5. On any hard error, the handler returns immediately with `batchItemFailures` for the failing Kinesis sequence number so the shard retries from that point. + +## Key Integration Points + +- **Kinesis**: letter table DynamoDB Streams piped through Kinesis for ordered, replay-safe delivery. +- **`LetterQueueRepository`** from `@internal/datastore`: `putLetter` and `deleteLetter` on the queue projection table. +- **`@internal/helpers`**: EMF metrics via `emitForSingleSupplier` and `buildEMFObject`. + +## Nuances and Peculiarities + +- **Replay safety**: `LetterAlreadyExistsError` on insert and `LetterNotFoundError` on delete are treated as success (return 0, not added to failures). Kinesis replays after a checkpoint failure will not produce false alarms. +- **REJECTED letters are never queued.** Letters inserted with status REJECTED (from a failed supplier allocation) are INSERT events but with `status != PENDING`, so `isNewPendingLetter` returns false and they are not added to the queue. +- **Immediate failure return**: on any unexpected error, the handler stops processing remaining records and returns the failing sequence number in `batchItemFailures`, maintaining Kinesis ordering guarantees. +- Queue entries use a composite `queueSortOrderSk` (zero-padded priority + ISO timestamp) to enable priority-ordered retrieval by the API handler's `GET /letters` endpoint. + + diff --git a/lambdas/upsert-letter/README.md b/lambdas/upsert-letter/README.md new file mode 100644 index 000000000..e7a2998c0 --- /dev/null +++ b/lambdas/upsert-letter/README.md @@ -0,0 +1,32 @@ + + +# Upsert Letter Lambda + +## Purpose + +Consumes allocation and status-change messages from SQS and persists them to the letters DynamoDB table as inserts or updates. + +## General Flow + +1. Each SQS record body is parsed as a `QueueMessage`. +2. The event `type` determines whether the record is handled as an insert (`LetterRequestPreparedEvent` v1/v2) or an update (`LetterStatusChangeEvent`). +3. Insert records create new letters from allocation output; update records apply status and reason changes to existing letters. +4. Records are processed with idempotency (`IDEMPOTENCY_TABLE`) so retries do not duplicate work. +5. Success and failure metrics are emitted, and only true failures are returned as `batchItemFailures` for partial retry. + +## Key Integration Points + +- **SQS**: input from the supplier-allocator output queue. +- **`LetterRepository`** from `@internal/datastore`: `putLetter` for inserts, `updateLetterStatus` for updates. +- **`@aws-lambda-powertools/idempotency`**: backed by a DynamoDB persistence layer (`IDEMPOTENCY_TABLE`). +- **Event schemas**: `@nhsdigital/nhs-notify-event-schemas-letter-rendering` (v1 and v2) for insert operations; `@nhsdigital/nhs-notify-event-schemas-supplier-api` for update operations. + +## Nuances and Peculiarities + +- **Letters can be inserted with status REJECTED.** If the supplier-allocator could not resolve a supplier (e.g., no active allocations), it sets `allocationStatus.status = "REJECTED"`. The upsert handler respects this and inserts the letter with status REJECTED rather than PENDING. +- **Duplicate inserts are not failures.** If `putLetter` throws `LetterAlreadyExistsError`, the handler logs a warning and moves on without adding the record to `batchItemFailures`. +- **Update deduplication** uses a separate mechanism: `LetterRepository.updateLetterStatus` includes a DynamoDB condition expression checking `eventId` to skip already-processed updates (returns `undefined` rather than throwing). +- Idempotency keys on event `id` mean SQS retries for the same event are safe for both insert and update paths. +- The `billingRef` field in the letter record is set to `allocationDetails.supplierSpec.specId` (the pack specification ID), while `specificationBillingId` is set to `allocationDetails.supplierSpec.billingId`. + + diff --git a/package-lock.json b/package-lock.json index b72b4e10d..3cb3b39f7 100644 --- a/package-lock.json +++ b/package-lock.json @@ -116,7 +116,7 @@ }, "internal/events": { "name": "@nhsdigital/nhs-notify-event-schemas-supplier-api", - "version": "1.0.19", + "version": "1.0.20", "license": "MIT", "dependencies": { "@asyncapi/bundler": "^0.6.4", diff --git a/sandbox/README.md b/sandbox/README.md index 757657ba6..de58bedd5 100644 --- a/sandbox/README.md +++ b/sandbox/README.md @@ -1,90 +1,39 @@ +# Sandbox -# OpenAPI Generated JavaScript/Express Server +## Purpose -## Overview -This server was generated using the [OpenAPI Generator](https://openapi-generator.tech) project. The code generator, and it's generated code allows you to develop your system with an API-First attitude, where the API contract is the anchor and definer of your project, and your code and business-logic aims to complete and comply to the terms in the API contract. +A standalone Express mock server for Supplier API integration testing, allowing suppliers to validate their integrations without AWS access. -### prerequisites -- NodeJS >= 10.6 -- NPM >= 6.10.0 +## General Flow -The code was written on a mac, so assuming all should work smoothly on Linux-based computers. However, there is no reason not to run this library on Windows-based machines. If you find an OS-related problem, please open an issue and it will be resolved. +1. The server starts from `index.js`/`expressServer.js` and loads `api/openapi.yaml`. +2. Incoming requests are validated against the OpenAPI contract before reaching handlers. +3. Valid requests are routed to controller/service handlers that return canned responses from `data/examples/`. +4. The sandbox runs on port 9000 by default (configurable in `config.js`). -### Running the server -#### This is a long read, but there's a lot to understand. Please take the time to go through this. -1. Use the OpenAPI Generator to generate your application: -Assuming you have Java (1.8+), and [have the jar](https://github.com/openapitools/openapi-generator#13---download-jar) to generate the application, run: -```java -jar {path_to_jar_file} generate -g nodejs-express-server -i {openapi yaml/json file} -o {target_directory_where_the_app_will_be_installed} ``` -If you do not have the jar, or do not want to run Java from your local machine, follow instructions on the [OpenAPITools page](https://github.com/openapitools/openapi-generator). You can run the script online, on docker, and various other ways. -2. Go to the generated directory you defined. There's a fully working NodeJS-ExpressJs server waiting for you. This is important - the code is yours to change and update! Look at config.js and see that the settings there are ok with you - the server will run on port 8080, and files will be uploaded to a new directory 'uploaded_files'. -3. The server will base itself on an openapi.yaml file which is located under /api/openapi.yaml. This is not exactly the same file that you used to generate the app: -I. If you have `application/json` contentBody that was defined inside the path object - the generate will have moved it to the components/schemas section of the openapi document. -II. Every process has a new element added to it - `x-eov-operation-handler: controllers/PetController` which directs the call to that file. -III. We have a Java application that translates the operationId to a method, and a nodeJS script that does the same process to call that method. Both are converting the method to `camelCase`, but might have discrepancy. Please pay attention to the operationID names, and see that they are represented in the `controllers` and `services` directories. -4. Take the time to understand the structure of the application. There might be bugs, and there might be settings and business-logic that does not meet your expectation. Instead of dumping this solution and looking for something else - see if you can make the generated code work for you. -To keep the explanation short (a more detailed explanation will follow): Application starts with a call to index.js (this is where you will plug in the db later). It calls expressServer.js which is where the express.js and openapi-validator kick in. This is an important file. Learn it. All calls to endpoints that were configured in the openapi.yaml document go to `controllers/{name_of_tag_which_the_operation_was_associated_with}.js`, which is a very small method. All the business-logic lies in `controllers/Controller.js`, and from there - to `services/{name_of_tag_which_the_operation_was_associated_with}.js`. +## Key Integration Points -5. Once you've understood what is *going* to happen, launch the app and ensure everything is working as expected: -``` -npm start -``` -### Tests -Unfortunately, I have not written any unit-tests. Those will come in the future. However, the package does come with all that is needed to write and run tests - mocha and sinon and the related libraries are included in the package.js and will be installed upon npm install command - -### View and test the API -(Assuming no changes were made to config.js) - -1. API documentation, and to check the available endpoints: -http://localhost:8080/api-docs/. To -2. Download the openapi.yaml document: http://localhost:8080/openapi. -3. Every call to an endpoint that was defined in the openapi document will return a 200 and a list of all the parameters and objects that were sent in the request. -4. Endpoints that require security need to have security handlers configured before they can return a successful response. At this point they will return [ a response code of 401](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401). -5. ##### At this stage the server does not support document body sent in xml format. - -### Node version and guidelines -The code was written using Node version 10.6, and complies to the [Airbnb .eslint guiding rules](https://github.com/airbnb/javascript). - -### Project Files -#### Root Directory: -In the root directory we have (besides package.json, config.js, and log files): -- **logger.js** - where we define the logger for the project. The project uses winston, but the purpose of this file is to enable users to change and modify their own logger behavior. -- **index.js** - This is the project's 'main' file, and from here we launch the application. This is a very short and concise file, and the idea behind launching from this short file is to allow use-cases of launching the server with different parameters (changing config and/or logger) without affecting the rest of the code. -- **expressServer.js** - The core of the Express.js server. This is where the express server is initialized, together with the OpenAPI validator, OpenAPI UI, and other libraries needed to start our server. If we want to add external links, that's where they would go. Our project uses the [express-openapi-validator](https://www.npmjs.com/package/express-openapi-validator) library that acts as a first step in the routing process - requests that are directed to paths defined in the `openapi.yaml` file are caught by this process, and it's parameters and bodyContent are validated against the schema. A successful result of this validation will be a new 'openapi' object added to the request. If the path requested is not part of the openapi.yaml file, the validator ignores the request and passes it on, as is, down the flow of the Express server. - -#### api/ -- **openapi.yaml** - This is the OpenAPI contract to which this server will comply. The file was generated using the codegen, and should contain everything needed to run the API Gateway - no references to external models/schemas. - -#### utils/ -Currently a single file: - -- **openapiRouter.js** - This is where the routing to our back-end code happens. If the request object includes an ```openapi``` object, it picks up the following values (that are part of the ```openapi.yaml``` file): 'x-openapi-router-controller', and 'x-openapi-router-service'. These variables are names of files/classes in the controllers and services directories respectively. The operationId of the request is also extracted. The operationId is a method in the controller and the service that was generated as part of the codegen process. The routing process sends the request and response objects to the controller, which will extract the expected variables from the request, and send it to be processed by the service, returning the response from the service to the caller. +- **OpenAPI spec**: the sandbox uses `api/openapi.yaml` to enforce request/response contract validation. +- **Swagger UI**: available at `/api-docs/` for interactive endpoint exploration. +- **Canned responses**: JSON files in `data/examples/` provide mock API data. +- **Docker**: `Dockerfile` enables containerised deployment for CI or supplier testing environments. -#### controllers/ -After validating the request, and ensuring this belongs to our API gateway, we send the request to a `controller`, where the variables and parameters are extracted from the request and sent to the relevant `service` for processing. The `controller` handles the response from the `service` and builds the appropriate HTTP response to be sent back to the user. +## Nuances and Peculiarities -- **index.js** - load all the controllers that were generated for this project, and export them to be used dynamically by the `openapiRouter.js`. If you would like to customize your controller, it is advised that you link to your controller here, and ensure that the codegen does not rewrite this file. +- `LetterService.listLetters` respects the `limit` query parameter by truncating the canned response array to simulate pagination. +- This is an OpenAPI Generator-scaffolded project (CommonJS, not TypeScript). It does not share code with the Lambda implementations. +- No authentication is enforced; the authorization behaviour is not simulated. +- The OpenAPI spec in `sandbox/api/` may diverge from `specification/api/` if not kept in sync during spec updates. -- **Controller.js** - The core processor of the generated controllers. The generated controllers are designed to be as slim and generic as possible, referencing to the `Controller.js` for the business logic of parsing the needed variables and arguments from the request, and for building the HTTP response which will be sent back. The `Controller.js` is a class with static methods. +## Running -- **.js** - auto-generated code, processing all the operations. The Controller is a class that is constructed with the service class it will be sending the request to. Every request defined by the `openapi.yaml` has an operationId. The operationId is the name of the method that will be called. Every method receives the request and response, and calls the `Controller.js` to process the request and response, adding the service method that should be called for the actual business-logic processing. - -#### services/ -This is where the API Gateway ends, and the unique business-logic of your application kicks in. Every endpoint in the `openapi.yaml` has a variable 'x-openapi-router-service', which is the name of the service class that is generated. The operationID of the endpoint is the name of the method that will be called. The generated code provides a simple promise with a try/catch clause. A successful operation ends with a call to the generic `Service.js` to build a successful response (payload and response code), and a failure will call the generic `Service.js` to build a response with an error object and the relevant response code. It is recommended to have the services be generated automatically once, and after the initial build add methods manually. - -- **index.js** - load all the services that were generated for this project, and export them to be used dynamically by the `openapiRouter.js`. If you would like to customize your service, it is advised that you link to your controller here, and ensure that the codegen does not rewrite this file. - -- **Service.js** - A utility class, very simple and thin at this point, with two static methods for building a response object for successful and failed results in the service operation. The default response code is 200 for success and 500 for failure. It is recommended to send more accurate response codes and override these defaults when relevant. - -- **.js** - auto-generated code, providing a stub Promise for each operationId defined in the `openapi.yaml`. Each method receives the variables that were defined in the `openapi.yaml` file, and wraps a Promise in a try/catch clause. The Promise resolves both success and failure in a call to the `Service.js` utility class for building the appropriate response that will be sent back to the Controller and then to the caller of this endpoint. - -#### tests/ -- **serverTests.js** - basic server validation tests, checking that the server is up, that a call to an endpoint within the scope of the `openapi.yaml` file returns 200, that a call to a path outside that scope returns 200 if it exists and a 404 if not. -- **routingTests.js** - Runs through all the endpoints defined in the `openapi.yaml`, and constructs a dummy request to send to the server. Confirms that the response code is 200. At this point requests containing xml or formData fail - currently they are not supported in the router. -- **additionalEndpointsTests.js** - A test file for all the endpoints that are defined outside the openapi.yaml scope. Confirms that these endpoints return a successful 200 response. - - -Future tests should be written to ensure that the response of every request sent should conform to the structure defined in the `openapi.yaml`. This test will fail 100% initially, and the job of the development team will be to clear these tests. +```bash +cd sandbox && npm install && npm start +``` +Or via Docker: -#### models/ -Currently a concept awaiting feedback. The idea is to have the objects defined in the openapi.yaml act as models which are passed between the different modules. This will conform the programmers to interact using defined objects, rather than loosely-defined JSON objects. Given the nature of JavaScript programmers, who want to work with their own bootstrapped parameters, this concept might not work. Keeping this here for future discussion and feedback. +```bash +docker build -t supplier-api-sandbox sandbox/ +docker run -p 9000:9000 supplier-api-sandbox +``` diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 000000000..cf34ec667 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,66 @@ + + +# Tests + +## Purpose + +Test suites that validate the supplier API beyond individual package unit tests. Each suite targets a different layer of the system. + +## Test Suites + +| Suite | Location | Framework | Description | +| --------------- | ------------------------------------------------ | ----------- | ----------------------------------------------------------------------------------------------------- | +| **Component** | `tests/component-tests/` | Playwright | API Gateway integration tests that exercise endpoints directly in an AWS environment with seeded data | +| **Sandbox** | `tests/sandbox/` | Playwright | Tests against the sandbox Express server to validate mock API behaviour | +| **Performance** | `tests/performance/` | Playwright | Load and latency tests against deployed environments | +| **Pact** | `tests/pact-tests/`, `tests/contracts/provider/` | Pact + Jest | Consumer-driven contract tests ensuring API compatibility with known consumers | +| **E2E** | `tests/e2e-tests/` | pytest | End-to-end tests targeting generated proxies and AWS environments | + +## Prerequisites + +- **Unit and Pact tests**: There are no prerequisites as they run locally with no external dependencies. +- **Component tests** require a running AWS login and a deployed environment: + 1. Deploy a dynamic environment (can be achieved by creating a Pull Request). Take a note of the environment e.g. pr1234 + 2. In the root level create an `.env` file and setup the `GITHUB_TOKEN` and `TARGET_ENVIRONMENT` variables (use `.env.template` as a guide) + 3. Source the env file by running `set -a` -> `source .env` -> `set +a` + 4. Login to your AWS account by running `aws sso login` in the terminal +- **Sandbox tests** require a sandbox server. +- **Performance and E2E tests** require AWS credentials, deployed infrastructure, and seeded test data. See `scripts/test-data/` for test data generation and `tests/e2e-tests/README.md` for environment-specific setup. + 1. Deploy a dynamic environment (can be achieved by creating a Pull Request). Take a note of the environment e.g. pr1234. + 2. Build proxies in the dynamic environment by setting the label `deploy_proxy` (ask a member of the team if you need help) + 3. In the root level create an `.env` file and setup the `GITHUB_TOKEN`, `TARGET_ENVIRONMENT`, `TARGET_ACCOUNT_GROUP`, `PROXY_NAME`, `API_ENVIRONMENT`, `NON_PROD_API_KEY`, `STATUS_ENDPOINT_API_KEY` & `NON_PROD_PRIVATE_KEY` variables. (use `.env.template` as a guide) + 4. Source the env file by running `set -a` -> `source .env` -> `set +a` + 5. Login to your AWS account by running `aws sso login` in the terminal + +## Running Tests + +Unit tests: + +```bash +npm run test:unit +``` + +Exceptionally, this workspace doesn't contain any unit tests. See top package file for other workspaces. + +Component, sandbox, and performance tests (from `tests` workspace): + +```bash +npm run test:component +npm run test:sandbox +npm run test:performance +``` + +Pact tests: + +```bash +npm run test:pact +make test-contract (from root level Makefile) +``` + +E2E tests (require AWS credentials and environment configuration): + +```bash +make .internal-dev-test (from root level Makefile) +``` + + diff --git a/tests/e2e-tests/README.md b/tests/e2e-tests/README.md index 4b6683d86..32ebde418 100644 --- a/tests/e2e-tests/README.md +++ b/tests/e2e-tests/README.md @@ -10,8 +10,8 @@ export PROXY_NAME=nhs-notify-supplier--internal-dev--nhs-notify-supplier Available values for `PROXY_NAME` include: -* `nhs-notify-supplier--internal-dev--nhs-notify-supplier` -* `nhs-notify-supplier--internal-dev--nhs-notify-supplier-pr` +- `nhs-notify-supplier--internal-dev--nhs-notify-supplier` +- `nhs-notify-supplier--internal-dev--nhs-notify-supplier-PR-` ## Set Up API Keys @@ -22,4 +22,4 @@ export NON_PROD_API_KEY=****** export STATUS_ENDPOINT_API_KEY=****** ``` -The values have been redacted here but you can obtain them from another team member. +The values have been redacted here but you can obtain them from another team member, or check [.env.template](/.env.template) for more information on how to set them up.