Skip to main content
Version: 2.2.x

Testing

opscotch was built to be reliable, and that includes workflow authoring.

opscotch has built-in support for authoring workflow test cases which is essential for bullet proof upgrades and workflow development.

When authoring a workflow ensure that you also author a test case. This has the following benefits:

The test case:

  • documents the expected flow of the workflow.
  • can detect faults caused by a change of agent behavior.
  • is portable and can be given to opscotch consultants to diagnose a problem.
  • mocks out services used in the workflow and during development: prevents bad requests sent to the services; prevents rate limiting from services; prevents additional load on the services.

How opscotch workflow testing works​

opscotch workflow tests approximate the concept of "mocked integration tests", whereby the agent has no indication that it is interacting with a testing harness: as far as the agent is concerned, it is interacting with real services.

The opscotch testing harness is a Java program that pretends to be a HTTP server (web/api). It writes out a bootstrap file that an agent is observing, with all the host records for the tests pointing to the test harness server. The testing harness loads a set of test configurations and writes out workflow configurations, which the agent detects and executes. The testing harness listens for HTTP requests from the agent and sends replies for the workflow to consume. It also listens for metrics and logs sent from the agent. Each of these interactions (URLs, metics, logs) are observed and can be asserted on.

A full example is available here

Test case configuration​

Test cases are defined using a JSON file (schema reference). The general nature of these tests will be:

  • trigger a step to run.
  • expect some URLs to be called by the agent with appropriate responses sent back to the agent.
  • expect some metrics or logs to be sent.

Test harness configuration​

To select the test cases to run, a configuration is passed to the Test harness. See the schema here

This configuration requires:

  • the path to the opscotch resource directories for javascript to include.
  • the path to the test directories for tests to run.
  • the path to your license file.

These are entered into the Test Harness configuration.

Running tests​

The test harness configuration file is passed as an argument to the test harness Java program:

java -jar testrunner.jar testrunner.config.json

The Test harness will scan the test directories configured in the configuration file for files ending in .test.json and will schedule them for execution. It will output a message like this:

Start the opscotch agent with this argument: <your working directory>/test.bootstrap.json

Start your agent with the argument to the test harness bootstrap file:

opscotch-agent <your working directory>/test.bootstrap.json

The tests will run and give details on the results.