Skip to main content

Test Packs

Use testPacks to sync test packs and test cases for a service.

Supported types​

Test pack type:

  • smoke
  • regression
  • manual

Test case type:

  • api
  • manual

Config shape​

testPacks:
  - name: checkout-smoke
    type: smoke
    environment: production
    releaseLabel: 2026.03
    testCases:
      - type: api
        title: create-order returns 201
        order: 1
        apiGroupName: checkout-api
        operationId: createOrder
        expectedStatusCode: 201
        requestTemplate: '{"amount":100}'
        requiresEvidence: false
        isCritical: true

      - type: manual
        title: checkout success path
        order: 2
        stepsList:
          - action: Open checkout and submit a valid card
            expectedResult: Payment is accepted
        expectedOutcome: Order is placed and confirmation is shown
        requiresEvidence: true
        isCritical: true

Required fields​

Pack level:

  • name
  • type

Case level:

  • type
  • title

Supported pack fields​

Pack fields currently supported by the CLI include:

  • name
  • type
  • environment
  • releaseLabel
  • testCases
  • testCasesPath

Reference screenshots​

Any test case — API or manual — can carry reference screenshots. They are the expected visual result a reviewer checks against, and they are attached to the case in UIGraph.

testPacks:
  - name: checkout-smoke
    type: smoke
    testCases:
      - type: manual
        title: checkout success path
        order: 1
        stepsList:
          - action: Open checkout and submit a valid card
            expectedResult: Payment is accepted
        screenshots:
          - ./.uigraph/tests/screenshots/checkout-confirmation.png
          - ./.uigraph/tests/screenshots/checkout-receipt.png

Rules:

  • Each path must point to an existing file, resolved relative to the working directory where the CLI runs.
  • A directory is rejected; point at the image itself.
  • The extension must be one of .png, .jpg, .jpeg, .gif, .webp, .svg.

Uploads are content-addressed: the CLI hashes each file and reuses the stored asset when the hash already exists, so re-running sync on unchanged screenshots uploads nothing. The summary reports both halves:

Screenshots: 2 uploaded, 5 skipped

Editing the list replaces the case's screenshot set, so dropping one of several entries detaches that image on the next sync. Removing the screenshots key entirely — or leaving it empty — sends nothing at all and leaves whatever is already attached in place. To clear a case completely, detach the images in the UI.

External test case files​

Use testCasesPath to keep a pack's test cases in a separate file instead of inline. The referenced file contains a testCases: list.

Inline testCases and an external testCasesPath may be combined — both are merged into the same pack.

.uigraph.yaml:

testPacks:
  - name: Adapter Smoke
    type: smoke
    testCases:
      - title: Register user returns 201
        type: api
        order: 1
        apiGroupName: uigraph-cli-openapi-spec
        operationId: registerUser
        expectedStatusCode: 201
    testCasesPath: .uigraph/tests/adapter-smoke.yaml

.uigraph/tests/adapter-smoke.yaml:

testCases:
  - title: Login returns 200
    type: api
    order: 2
    apiGroupName: uigraph-cli-openapi-spec
    operationId: loginUser
    expectedStatusCode: 200
  - title: Verify login flow in API docs
    type: manual
    order: 3
    stepsList:
      - action: Open the synced API group in UiGraph
        expectedResult: The `uigraph-cli-openapi-spec` API group is visible
    expectedOutcome: The login endpoint is easy to inspect in UiGraph
    requiresEvidence: false

Paths are resolved relative to the working directory where the CLI runs.

Supported test case fields​

All test cases support:

  • type
  • title
  • order
  • description
  • priority
  • tags
  • linkedTicket
  • estimatedDurationMins
  • testOwner
  • requiresEvidence
  • isCritical
  • screenshots

API test cases can also include:

  • apiGroupName
  • operationId
  • expectedStatusCode
  • requestTemplate
  • responseTimeMs
  • responseBody
  • assertions
  • mapName
  • frameName
  • focalPointName

Manual test cases can also include:

  • stepsList
  • expectedOutcome
  • preconditions
  • testData
  • postconditions

Behavior​

When running uigraph-cli sync:

  1. CLI syncs each test pack first.
  2. CLI syncs all test cases inside that pack.
  3. Each case's screenshots are hashed and uploaded (or skipped) before the case itself is written, so one call carries the case and its images.
  4. CLI includes commit hash metadata when available.

Dry run prints test pack and test case summaries without sending data.

Notes​

  • stepsList is the supported field for manual step sequences.
  • apiGroupName and operationId are most useful when your synced API spec includes stable OpenAPI operationId values.
  • The CLI validates pack and case types, but it does not fully validate cross-references such as whether a named API group already exists in UIGraph.