Developer setup

To develop using the CTC Traders API v2.1, you must:

• be familiar with HTTP, RESTful services, XML and OAuth2
• be registered as a developer on the HMRC Developer Hub
• add at least one sandbox application on the Developer Hub

Each application you register will be assigned an HMRC ApplicationId.

You can view all the applications you have registered on the Developer Hub Applications page where you can administer API subscriptions and application credentials.

Applications can be created in one of 2 possible environments:

• sandbox environments - where you start development and testing of your software
• production environment - when you are ready to go live you can request ‘Production Credentials’ that will create you a new application (with a new ApplicationId) in the production environment

The main differences between these 2 environments are that:

• in the sandbox environment you will create and use test users to call the endpoints
• the release version of an API in the sandbox will generally be ahead of the version of the API in production

The purpose of the sandbox (also known as External Test) is to provide an HMRC platform for all external partners to complete their required assurance activities so they are confident in their readiness for migration to NCTS5.

For more information about using our sandbox environments, see Testing in the sandbox.

Trader Test

Trader Test is a functional test platform where you can assure your software integration. Trader Test simulates both the automated responses and also the real-life experience where NCTS support staff perform the tasks of Border Force personnel.

When your testing requires a manual response, NCTS support staff will perform the live manual steps of the process. This simulates and tests a full real-life journey from start to finish for you.

You can do this by performing a series of compatibility tests on your software using our sandbox environment, which is a replica of the live production environment. You’ll also need to do some sample scenarios and test data for both Great Britain and Northern Ireland.

API subscription model

You must subscribe each application you register to the CTC Traders API v2.1. Subscribing allows an application access to all of the endpoints of the API.

API access control

The CTC Traders API v2.1 is a public API, so you should be able to gain access to it automatically and subscribe to it easily.

API versioning

For your application to be able to call a particular version of one of our APIs, you must:

• ensure that your application has access to it (by checking that the version is visible to you at the top of the API’s documentation page)
• subscribe your application to it (in addition to any other subscriptions you may already have for other versions)
• modify your application’s Accept request header value when making a call to the API to reflect the version you wish to use (see the specific API reference for more details about request header values)

For more information about versioning, see our Reference guide.

API rate limiting

Each software house should register a single application with HMRC. This application will be used to identify the software house during the OAuth 2.0 grant flow and will also be used in subsequent per user API calls. We limit the number of requests that each application can make. This protects our backend service against excessive load and encourages real-time API calls over batch processing.

We set limits based on anticipated loads and peaks. Our standard limit is 3 requests per second per application. If you believe that your application will sustain traffic load above this value, contact the SDS Team.

If your software reaches the allocated limit, some of your requests will get a HTTP 429 response code. If your application receives a 429 response, it should back off temporarily before attempting to make the same request again.

In most cases, simply following software design best practices will deliver a good customer experience. API requests should not be unnecessarily batched and rate limited responses (HTTP 429) should be captured by the application and be automatically retried without generating user errors.

Retries should not be too quick (delay the next attempt by a few seconds) and should include some randomisation to minimise the risk of different application instances conflicting. Where an application instance is generating high volumes of API calls, the software design should consider periodically pausing or locally rate limiting those API calls to minimise the potential for a spike in traffic from one instance impacting another.

For more information about versioning, see our Reference guide.

Authentication

You must use your Government Gateway account to authenticate directly with us. This grants authority to your application for specific scopes. We then issue an OAuth 2.0 access token that is specific to you. Your application passes the access token in subsequent API requests to user-restricted API endpoints. A declarant access token expires after 4 hours and will need to be refreshed.

For more information about user-restricted endpoints, see our Authorisation guide.

Error handling

No system guarantees 100% error-free performance, so services calling our APIs need to be designed with failure in mind. There are several strategies to mitigate the errors that do occur.

5XX errors

If an API returns an HTTP response code in the 5XX range, the recommended response is to retry. Typically, a 500 (Internal Server Error) or 503 (Service Unavailable) response will be returned if there is an issue with an API. The issue may be transient, hence the recommendation to retry. The one exception in this error range is when a 501 (Not Implemented) is returned. This indicates that the endpoint is incorrect. Retrying will not help in this situation. Instead, the URL should be checked and fixed.

After multiple attempts, perhaps with an exponential backoff strategy, the expectation is that this will resolve the vast majority of errors. As an example, 3-5 retries over 1-2 minutes would be a reasonable retry strategy. Consideration should also be given to using the circuit breaker pattern.

4XX errors

If a response code in the 4XX range is received, the API consumer should examine the response payload for more detail on the error before attempting any retry. This error range usually indicates an error with the request. The request should be modified before attempting a retry.

For example, a 400 (Bad Request) with a payload response message stating ‘Payload is not valid according to schema’ should have its request payload modified so that it validates against the API schemas and then be resent. Similarly, a 406 (Not Acceptable) indicates that the Accept header is missing or invalid. The request should be amended accordingly.

If you are receiving these errors consistently, you should report the problem to the SDS Team.

NCTS error codes

For information about error codes returned by the NCTS, see NCTS phase 5 technical interface specification.

Best practices recommendations

There are many best practices recommendations for handling API errors and exceptions, including the following:

• The 6 Pillars of the AWS Well-Architected Framework (AWS Partner Network blog)
• Design for Failure (excerpt from Amazon white paper ‘Running Containerized Microservices on AWS’)
• Implement resilient applications (excerpt from Microsoft e-book ‘.NET Microservices Architecture for Containerized .NET Applications’)

Date-time parameters

When writing code to use date filters in request URLs, you must always use percent-encoding to avoid getting 400 Bad Request errors. This is because some common characters used in dates and timestamps cannot be used in URLs.

Format

When formatting query parameters into a request URL for date and time filtering functionality, you must use only the ISO 8601 standard for the date and time. For example, the timestamp 2021-06-21T09:00+00:00 should be encoded as 2021-06-21T09%3A00%2B00%3A00.

For more information about the formats we use for common data types, see our Reference guide.

You should also note the following:

• some common data types described in our Reference guide contain characters that are not valid for use in URLs
• some software libraries and frameworks do percent-encoding for you automatically

Important: Your messages to the NCTS service must conform to the XSD files used by CTC Traders API v2.1. The XSD files do not necessarily conform to the ISO 8601 standard, so your XML payloads might have date-time parameters that do not conform to ISO 8601.

Examples

Below are examples in different programming languages.

Language	Example
C#	Uri.EscapeDataString("2021-04-30T16:08:31+00:00");
Java	java.net.URLEncoder.encode("2021-04-30T16:08:31+00:00");
Python	from urllib.parse import quotequote('2021-04-30T16:08:31+00:00')

Find out more

For background information about percent-encoding, we recommend the following:

• RFC
• Wikipedia
• MDN

Message sender and recipient guidelines

For general guidelines about entering message sender and recipient details in messages for NCTS5 transit movements, see NCTS5 technical interface specification.

For guidelines about entering message recipient details in messages for GB to XI or XI to GB transit movements in the NCTS5 Trader Test environment, see CTC Traders API phase 5 testing guide.

Reuse of Local Reference Numbers

In NCTS5, a ‘Declaration Data’ E_DEC_DAT (IE015) message starts a transit movement with a given Local Reference Number (LRN) when you submit the message successfully, not when the message is accepted. This means that you can use the LRN to query the movement before it is accepted.

Ordinarily, if you try to submit an IE015 message with a previously used LRN to the NCTS5 service, a ‘Rejection From Office Of Departure‘ E_DEP_REJ (IE056) message will be returned. However, there are 2 scenarios where you can reuse an LRN.

Scenario	Description
After initial rejection	If you send an invalid IE015 message that is rejected after a first submission attempt and a ‘Rejection From Office Of Departure‘ E_DEP_REJ (IE056) message is received, you can reuse the LRN from the first submission attempt if you amend the IE015 message to make it valid and resubmit it.
After a cancellation request is accepted	If a cancellation request is accepted and a positive ‘Invalidation Decision‘ E_INV_DEC (IE009) message is sent to the trader at any point during departure, including a customs-instigated cancellation post-movement release, you can reuse the LRN for the declaration that was cancelled.

Push-pull notifications

You can use our Push-Pull-Notification Service (PPNS) to receive notifications of new messages from the NCTS as follows:

• if your endpoint is hosted by Amazon Web Services (AWS), you must use either edge-optimised custom domain names or regional custom domain names
• you will receive a push notification each time when there is a new message for you to read
• for messages less than 80KB, a push notification will contain the message body
• a push notification will have a field containing the message URI
• you can use this URI to download the XML message from the CTC Traders API

Important: If a movement is created in CTC Traders API version 2.0, all notifications will be directed to the 2.0 boxId. It is not possible to redirect these messages to the 2.1 boxId, as it does not exist in this case.

Using this functionality means that you can avoid polling for new messages and thus save time and resources.

Message received push notifications

If you are using our PPNS service and you submit a small message (up to 5MB in size) with an XML payload to the NCTS, you will receive a push notification with:

• the notificationType field set to MESSAGE_RECEIVED
• a URI that you can use with the Get a cached message related to an arrival and message ID and Get a cached message related to a departure and message ID endpoints to get the message

If the size of the message sent to you is less than 80kB, the message will be included in the push notification as escaped XML in the messageBody field.

Message received push notifications schema

Message received push notifications must conform to the following JSON schema.

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "additionalProperties": false,
  "id": "https://developer.service.hmrc.gov.uk/common-transit-convention-traders/push-notification.json",
  "required": [
    "messageUri",
    "messageId",
    "notificationType"
  ],
  "properties": {
    "messageUri": {
      "type": "string",
      "description": "URI of the message."
    },
    "departureId": {
      "type": "string",
      "description": "ID of the movement if it is a departure."
    },
    "arrivalId": {
      "type": "string",
      "description": "ID of the movement if it is an arrival."
    },
    "messageId": {
      "type": "string",
      "description": "ID of the message that has been received."
    },
    "messageType": {
      "type": "string",
      "description": "The type of message being received."
    },
    "enrollmentEORINumber": {
      "type": "string",
      "description": "The enrollment EORI number that owns the movement. This parameter replaces customerId in CTC Traders API v1.0."
    },
    "notificationType": {
      "type": "string",
      "enum": [
        "MESSAGE_RECEIVED"
      ],
      "description": "The notification type. Always MESSAGE_RECEIVED when the notification informs the trader of a new message."
    },
    "messageBody": {
      "type": "string",
      "description": "Escaped XML that represents the returned message. Will only be present if the message is small."
    }
  },
  "type": [
    "object"
  ]
}

Note: This schema is subject to change. You should monitor the CTC Traders API v2.1 changelog (GitHub) and the changelog for this document (GitHub) to ensure that you are aware of any changes.

Example message received push notifications

Below are examples of message received push notifications for different small message sizes.

Below 80kB threshold

Example push notification where the message body is included because the message sent to you is less than 80kB in size.

{
            "messageUri": "/customs/transits/movements/departures/0123456789abcdedf/messages/fedcba9876543210",
            "enrollmentEORINumber": "GB0123456789012",
            "departureId": "0123456789abcdedf",
            "messageId": "fedcba9876543210",
            "nessageType": "IE928",
            "notificationType": "MESSAGE_RECEIVED",
            "messageBody": "

<ncts:CC928C
    xmlns:ncts=\"http://ncts.dgtaxud.ec\" PhaseID=\"NCTS5.1\">
    <messageSender>NTA.XI</messageSender>
    <messageRecipient>swp.transit.agent</messageRecipient>
    <preparationDateAndTime>2023-05-22T16:00:17</preparationDateAndTime>
    <messageIdentification>MR3MAJP2BB6890</messageIdentification>
    <messageType>CC928C</messageType>
    <correlationIdentifier>messageIdentification0</correlationIdentifier>
    <TransitOperation>
        <LRN>MDTP-12</LRN>
    </TransitOperation>
    <CustomsOfficeOfDeparture>
        <referenceNumber>XI000081</referenceNumber>
    </CustomsOfficeOfDeparture>
    <HolderOfTheTransitProcedure>
        <identificationNumber>XI985524247819</identificationNumber>
        <name>Trader 1</name>
        <Address>
            <streetAndNumber>1 TRADER STREET</streetAndNumber>
            <postcode>BT12 5EE</postcode>
            <city>Belfast</city>
            <country>BG</country>
        </Address>
    </HolderOfTheTransitProcedure>
</ncts:CC928C>"
}

Above 80kB threshold

Push notification where the message body is not included because the message sent to you is greater than 80kB in size.

{
            "messageUri": "/customs/transits/movements/arrivals/0123456789abcdedf/messages/fedcba9876543210",
            "enrollmentEORINumber": "GB0123456789012",
            "arrivalId": "0123456789abcdedf",
            "messageId": "fedcba9876543210",
            "messageType": "IE025",
            "notificationType": "MESSAGE_RECEIVED"
} 

Submission notification push notifications

If you are using our PPNS service and you submit a message by the large message route to the NCTS, you will receive a push notification with:

• the notificationType field set to SUBMISSION_NOTIFICATION
• a URI that you can use with the Get a cached message related to an arrival and message ID and Get a cached message related to a departure and message ID endpoints to get the submitted message back
• the response indicating whether the submission succeeded or failed

Submission notification push notifications schema

Submission notification push notifications must conform to the following JSON schema.

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "additionalProperties": false,
  "id": "https://developer.service.hmrc.gov.uk/common-transit-convention-traders/push-notification-submission-notification.json",
  "required": [
    "messageUri",
    "messageId",
    "notificationType",
    "response"
  ],
  "properties": {
    "messageUri": {
      "type": "string",
      "description": "URI of the message."
    },
    "departureId": {
      "type": "string",
      "description": "ID of the movement if it is a departure."
    },
    "arrivalId": {
      "type": "string",
      "description": "ID of the movement if it is an arrival."
    },
    "messageId": {
      "type": "string",
      "description": "ID of the message that is being submitted."
    },
    "enrollmentEORINumber": {
      "type": "string",
      "description": "The enrollment EORI number that owns the movement. This parameter replaces customerId in CTC Traders API v1.0."
    },
    "notificationType": {
      "type": "string",
      "enum": [
        "SUBMISSION_NOTIFICATION"
      ],
      "description": "The notification type. Always SUBMISSION_NOTIFICATION when the notification informs the trader of submission status."
    },
    "response": {
      "type": "object",
      "description": "The response object. In the case of failure, this will be identical to faliure cases in the small message route.",
      "additionalProperties": true,
      "properties": {
        "code": {
          "type": "string",
          "description": "The response code. Will be 'SUCCESS' if submission was successful.",
          "message": "A human readable message describing the status."
        }
      }
    }
  },
  "type": [
    "object"
  ]
}

Note: This schema is subject to change. You should monitor the CTC Traders API v2.1 changelog (GitHub) and the changelog for this document (GitHub) to ensure that you are aware of any changes.

Example submission notifications push notifications

Below are examples of submission notification push notifications for different scenarios.

Success notification

{
   "messageUri": "/customs/transits/movements/departures/6482f76c21a8596e/messages/6482f76c564b0f5f",
   "notificationType": "SUBMISSION_NOTIFICATION",
   "enrollementEORINumber": "GB0123456789012",
   "departureId": "6482f76c21a8596e",
   "messageId": "6482f76c564b0f5f",
   "response": {
     "message": "The message 6482f76c564b0f5f for movement 6482f76c21a8596e was successfully processed",
     "code": "SUCCESS"
   }
}

Schema validation error notification

{
   "messageUri": "/customs/transits/movements/arrivals/6482f76c21a8596e/messages/6482f76c564b0f5f",
   "notificationType": "SUBMISSION_NOTIFICATION",
   "enrollementEORINumber": "GB0123456789012",
   "arrivalId": "6482f76c21a8596e",
   "messageId": "6482f76c564b0f5f",
   "response": {
     "message": "Request failed schema validation",
     "code": "SCHEMA_VALIDATION",
     "validationErrors": [
         {
          "lineNumber": 1,
          "columnNumber": 90,
          "message": "cvc-complex-type.2.4.a: Invalid content was found starting with element 'messageRecipient'. One of '{messageSender}' is expected."
         },
         {
          "lineNumber": 1,
          "columnNumber": 202,
          "message": "cvc-pattern-valid: Value 'abcdefg' is not facet-valid with respect to pattern '\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}' for type 'PreparationDateAndTimeContentType'."
         },
                  {
          "lineNumber": 1,
          "columnNumber": 202,
          "message": "cvc-type.3.1.3: The value 'abcdefg' of element 'preparationDateAndTime' is not valid."
         }
     ]
   }
}