Skip to main content

Test runner packaging and licensing changes in 3.1.6

· 4 min read
Jeremy Scott
Co-founder

BREAKING CHANGE: The startup arguments for the test runner agent runtime have changed. Testing against production agent runtimes requires a production license.

Opscotch 3.1.6 updates the test runner to exercise the same packaging and licensing path used by current Opscotch deployments.

The test runner now injects configuration through the secrets endpoint, includes production license keys for production test coverage, and can deploy the packager and licenser Opscotch apps into the running Opscotch agent as part of the test flow.

What changed​

The 3.1.6 test runner changes focus on making test execution match the current runtime model more closely.

The main changes are:

  • configuration is injected through the secrets endpoint
  • production license keys are available for production test cases
  • the packager app is deployed into the running Opscotch agent during the test flow
  • the licenser app is deployed into the running Opscotch agent during the production test flow
  • tests use the current packaging and licensing flow instead of depending on the legacy v1 licensing path

This moves the test runner away from a special-case test setup and closer to the way real deployments are packaged, licensed, and started.

Configuration through secrets (breaking change)​

The test runner now injects configuration through the secrets endpoint.

You will see a change to the agent startup command which adds:

Start the opscotch agent...
With environment variable: OPSCOTCH_SECRETS_FROM=http://localhost:9999/secrets

That matters because test setup can now follow the same pattern used by deployments that source sensitive configuration externally. Instead of treating test configuration as a separate path, the test runner can provide configuration through the runtime mechanism designed for secret-backed startup data.

This is especially useful for tests that need to cover:

  • bootstrap configuration
  • encrypted values
  • license material
  • environment-specific settings
  • production-like startup behavior

Production license coverage (breaking change)​

3.1.6 also adds production license keys for test runner production tests.

Opscotch licenses are environment-specific. A production runtime should be tested with production licensing behavior, not only with development or non-production assumptions. Adding production license keys gives the test runner a cleaner way to validate production cases without bypassing the licensing model being tested.

When running tests against a production agent runtime, a production license still needs to be supplied. Set OPSCOTCH_TEST_PROD_LICENSE and OPSCOTCH_TEST_PROD_LICENSE_KEY when starting the test runner. This follows the Opscotch policy that the production runtime should not be weakened for testing convenience. Tests that target production behavior should use the same licensing expectations as production deployments.

Packager and licenser in the running agent​

The test runner now uses the new packaging and licensing flow by deploying the packager app and licenser app to the running Opscotch agent.

This is the important architectural change: The test runner is no longer just validating a workflow in isolation. It can bring up the supporting Opscotch apps needed to package and license the workflow in the same runtime environment under test.

This helps catch issues around:

  • package generation
  • package verification
  • runtime licensing
  • licensing host availability
  • package and license compatibility
  • production and non-production license separation

Why this matters​

These changes make tests more representative.

The older path was useful while packaging and licensing were changing, but it left too much distance between test execution and real deployment behavior. In 3.1.6, the test runner follows the current packaging and licensing model, so passing tests give stronger confidence that the same app will behave correctly when deployed through the supported runtime flow.

For the surrounding concepts, see: