Skip to main content

How can I prevent a step executing concurrently?

· 2 min read

In opscotch workflows, by default, a step can be run at any time, even if there is already a run on that step — meaning this step can be executed in multiple threads. But what if you want to enforce single-threaded execution for a step?

This situation might arise if you have a step triggered by a timer, say every second. Most of the time, it completes running within that timeframe, but occasionally it may take longer. If the step is triggered every second and takes longer than a second to complete, there will be two runs of the step occurring simultaneously. If it takes more than five seconds to complete, there could be five runs of the step at the same time.

Depending on what the step does, this might be acceptable; however, in some cases, it may not be. For instance, if you're consuming data from a queue (whether from opscotch or an external service), and in this situation it’s crucial that the data is processed in order. If multiple runs are pulling data from the queue, it is almost certain that the data will be processed out of order.

How does this work?

Version 3 of opscotch introduces a step property called singleThreaded — think of this as a declaration of single-threaded behavior. When this property is not set or is set to "none" (the default behavior), the step will not exhibit single-threaded behavior. However, when this property is set, the step will enforce single-threaded execution.

As of version 3.0.0, there are two available behaviors:

  • none: No single-threaded behavior is enforced.
  • return: When a step is currently running, any subsequent start requests will be ignored without error. The step will not run.

This property is designed to be expanded upon; the following behaviors are planned for future implementation:

  • error: When a step is currently running, any subsequent start requests will be halted with an error.
  • queue: When a step is currently running, any subsequent start requests will be delayed until the current run completes.