Opscotch Architect Skill

Design-focused guidance for engineers deciding whether and how to shape a solution in Opscotch terms.

This skill complements docs/docs/llm/opscotch-llm-skill.md.

Use this file for:

• deciding whether Opscotch is a good fit
• shaping solution boundaries before writing configuration
• choosing bootstrap versus workflow responsibilities
• designing step flow, trigger model, data placement, packaging, trust, and licensing shape
• explaining design decisions in Opscotch terms

Do not use this file as the full operational reference or API catalog. For exact contracts and signatures, consult:

• /llm/apireference-index.json
• /llm/apireference.json
• /llm/apireference-authenticated-index.json
• /llm/apireference-authenticated.json

Core architectural stance

Think about Opscotch as a constrained runtime for deployable workflow products, not as a general-purpose application host.

• Bootstrap is the installed boundary of trust and permission.
• Workflow is the remotely updateable logic that uses those allowed capabilities.
• Steps are the architectural execution boundaries.
• Processors and resources are implementation units inside those boundaries.
• Packaging, signatures, keys, and licensing are part of solution architecture, not afterthoughts.

Architecturally, the first question is not "how do I code this?" but:

1. Can this outcome be expressed through supported triggers, step flow, and bootstrap permissions?
2. Where is the correct boundary between operator-controlled installation concerns and updateable business logic?
3. What is the smallest, clearest workflow shape that achieves the business outcome without violating runtime constraints?

Primary provenance:

• docs/src/llm/processed/intro.md
• docs/src/llm/processed/workflow.md
• docs/docs/llm/opscotch-llm-skill.md
• docs/src/llm/preprocessed/bootstrap-schema.md
• docs/src/llm/preprocessed/workflow-schema.md

1. Recognize the right problem class

Opscotch is strongest when the problem is about collecting or enforcing something operationally awkward rather than serving as a general application platform.

It is a strong fit when the solution needs one or more of these shapes:

• multi-step collection across systems
• authentication chains before the real work can start
• reduction, filtering, or correlation at collection time
• adding time context to cross-sectional facts
• deployment of the same logic across many targets with different local permissions or data
• commercial or trust controls that should travel with the delivered artifact

Prefer an Opscotch-shaped solution when the key business value comes from capability composition:

• controlled access to hosts, files, listeners, and deployments
• reusable workflow logic deployed into many environments
• verifiable packaged distribution
• runtime-enforced entitlement or licensing

Do not reject a solution merely because marketing pages never named that exact use case. Suitability should be judged against documented capability and constraints, not only prior examples.

Opscotch is a weak fit when the problem fundamentally requires:

• arbitrary outbound access with no installer-defined boundary
• arbitrary filesystem or runtime library access
• async JavaScript or external package ecosystems inside processors
• a rich interactive application surface rather than an execution and integration engine
• long-lived complex in-memory state machines better served by another platform

Primary provenance:

• docs/src/llm/processed/intro.md
• docs/src/llm/processed/introduction/problem-to-plan.md
• docs/src/llm/processed/core-capabilities/runtime.md
• docs/src/llm/processed/security/workflow-firewall.md

2. Frame the problem in Opscotch terms

Translate the requirement into five architectural questions:

1. What starts the work?
2. What data enters, from where, and under which permission boundary?
3. What sequence of decisions, calls, splits, or enrichments is required?
4. What state must survive across executions?
5. What leaves the system, and under which trust or commercial controls?

A good Opscotch design usually reads like:

• one business outcome
• one or more workflows that represent reusable logic units
• a small number of clearly named steps with distinct responsibilities
• bootstrap-defined boundaries that are broad enough to support the use case but no broader

Work from the manual process first. If a human would need to:

• receive an event
• authenticate
• call one service
• iterate a returned list
• enrich each item
• reduce the result
• send a decision onward

then the workflow should usually mirror that sequence rather than collapsing it into one opaque processor.

Primary provenance:

• docs/src/llm/processed/introduction/problem-to-plan.md
• docs/src/llm/processed/workflow.md
• docs/src/llm/processed/docs/patterns.md

3. Draw the bootstrap versus workflow boundary

This is the most important architectural decision in Opscotch.

Put something in bootstrap when it is installer-controlled, environment-specific, security-relevant, or defines what the runtime is allowed to do.

Bootstrap responsibilities:

• where workflow configuration comes from
• allowed outbound hosts
• allowed file roots and patterns
• allowed HTTP listeners
• allowed cross-deployment calls
• persistence root
• licensing source and pool selection
• trusted signing identities and package expectations
• preloaded keys
• deployment-specific static data

Put something in workflow when it is remotely updateable logic or reusable business behavior.

Workflow responsibilities:

• workflow and step structure
• trigger wiring inside the allowed bootstrap boundary
• processor composition
• reusable logic flow
• workflow, step, and processor data that shape behavior within the installed boundary
• persistence usage and state transitions

Use this rule:

• bootstrap answers "what may this deployment do?"
• workflow answers "how should this deployment use what it may do?"

If changing a value should require installer intent or runtime restart, it belongs closer to bootstrap. If changing a value should be part of shipping new logic, it belongs closer to workflow.

Primary provenance:

• docs/docs/llm/opscotch-llm-skill.md
• docs/src/llm/processed/workflow.md
• docs/src/llm/preprocessed/bootstrap-schema.md
• docs/src/llm/preprocessed/workflow-schema.md

4. Shape workflows, steps, processors, and resources

Architect by responsibility, not by syntax.

Workflow

A workflow should represent a coherent business or integration outcome, not just a file of related snippets.

Good workflow boundaries:

• all steps contribute to one operational outcome
• the same workflow can reasonably be reused across multiple deployments
• deployment-specific variation can be handled through bootstrap and merged data

Split into separate workflows when:

• trigger models differ substantially
• permissions differ materially
• lifecycle or deployment cadence differs
• failure isolation matters

Step

A step is the execution boundary. Use steps to separate concerns such as:

• entry and validation
• authentication preparation
• outbound call and result handling
• controller/orchestration decisions
• split and aggregate work
• persistence reads or writes
• terminal delivery or response shaping

Make a new step when one of these changes:

• trigger responsibility
• state boundary
• retry or failure-handling intent
• authentication context
• concurrency requirement
• reuse value

Processor

Processors are not the architectural boundary; they are responsibility slots within a step.

Think of processor slots by purpose:

• urlGenerator: where the target call is determined
• payloadGenerator: where outbound request content is formed
• authenticationProcessor: where request-local secret mutation happens
• resultsProcessor: where returned data is interpreted and the next move is decided
• splitGenerator and itemResultProcessor: where fan-out and item-level reduction are shaped

Resource

A resource should encapsulate reusable behavior, not hide architecture.

Use resources when:

• the same behavior appears in multiple steps or workflows
• the behavior is generic and parameterized by data
• central maintenance matters

Do not use a resource to hide a step boundary that should remain visible in architecture.

Primary provenance:

• docs/src/llm/processed/workflow.md
• docs/src/llm/processed/docs/patterns.md
• docs/src/llm/preprocessed/workflow-schema.md

5. Choose the trigger model deliberately

Trigger choice is an architectural statement about how the solution enters the runtime.

Use:

• http when an external actor pushes work to the runtime
• timer when the runtime must poll or evaluate on a schedule
• tcp when the input is socket-based
• fileWatcher when filesystem change is the business event
• runOnce when initialization or one-time startup behavior is the goal
• deploymentAccess when another Opscotch deployment should be the caller

Trigger selection heuristics:

• choose http when the external system already knows when work should start
• choose timer when the runtime must discover change by checking
• choose fileWatcher only when the real event is local file mutation and bootstrap file permissions can support it
• choose deploymentAccess when you want a deployment boundary to stay explicit instead of exposing an HTTP listener
• choose runOnce for initialization, seeding, or startup-only support behavior, not for recurring work

Avoid using a trigger only because it is technically possible. Pick the trigger that makes ownership and event origin clear to operators.

Design note:

• author conservatively against the schema for HTTP triggers and include trigger.http.method explicitly, even where older prose implies a default

Primary provenance:

• docs/src/llm/processed/introduction/problem-to-plan.md
• docs/src/llm/processed/workflow.md
• docs/src/llm/preprocessed/workflow-schema.md
• docs/src/llm/operations/unknowns/apireference-unknowns.md

6. Design data flow, state, and inheritance

Opscotch architecture is heavily shaped by where data lives.

Use merged data for configuration and parameterization. Use trigger input for invocation-specific payload. Use step-to-step context for execution flow. Use persistence for state that must survive across runs.

Think in these layers:

• bootstrap data: deployment defaults, local identities, environment-specific constants
• host data: host-specific configuration, especially authentication-adjacent values
• workflow data: logic-level defaults for a reusable business flow
• step data: role-specific parameters
• processor data: fine-grained tuning of a reusable processor/resource

Resolved merge guidance:

• broader to more specific merge order
• last merged wins for primitives
• objects and arrays merge additively

Architectural use:

• place stable defaults high in the hierarchy
• override only where the business reason is local
• keep sensitive values in the most restricted place that still supports the design

Persistence guidance:

• treat persistence as step-scoped durable state, not as a global application database
• use it for cursors, checkpoints, cache-like data, queue coordination, or last-seen markers
• prefer bootstrap persistenceRoot so persistence remains an installation concern, not scattered workflow detail

Primary provenance:

• docs/src/llm/processed/introduction/problem-to-plan.md
• docs/src/llm/processed/docs/patterns.md
• docs/src/llm/processed/workflow.md

7. Treat authentication as a boundary, not a header tweak

Authentication architecture in Opscotch is intentionally separated from ordinary processor logic.

Model authentication as one of three cases:

1. Non-sensitive static headers

• place them in bootstrap host headers

2. Request-local secret mutation

• use authenticationProcessor
• keep sensitive values in host data
• mark the host authenticationHost: true when restricted host data is needed

3. Multi-step or stateful authentication flow

• have the authenticationProcessor call dedicated scripted-auth helper steps

Architectural rule:

• normal business steps should not own secret reasoning unless the request itself requires it
• authentication steps should exist to establish or mutate auth state for a pending request, not to absorb unrelated logic

Use separate authentication helpers when:

• token acquisition is multi-stage
• cookies or headers must be derived from prior calls
• auth logic is reused across multiple request steps

Do not place sensitive authorization values in ordinary host headers.

Primary provenance:

• docs/src/llm/processed/workflow.md
• docs/src/llm/processed/docs/patterns.md
• docs/docs/workflow.md

8. Compose for maintainability

Maintainable Opscotch architecture depends on explicit roles.

Useful patterns:

• controller and worker separation when orchestration logic would otherwise dominate a worker step
• split-aggregate when the business problem is naturally itemized and reduced
• reusable resources with parameterized data
• multiple deployments when operational boundaries differ more than logic does
• packaged delivery when reuse, trust, or controlled rollout matter

Keep these separations clear:

• controller logic decides
• worker logic does
• authentication logic authenticates
• persistence logic stores or restores execution state
• delivery logic emits results or responses

Prefer small explicit steps over one large processor when:

• the flow includes multiple external dependencies
• different failure modes deserve different treatment
• future operators need to reason about the path quickly

Primary provenance:

• docs/src/llm/processed/docs/patterns.md
• docs/src/llm/processed/workflow.md
• docs/src/llm/processed/introduction/problem-to-plan.md

9. Choose raw config versus packaged delivery

This decision changes how the solution is operated and trusted.

Prefer raw workflow JSON when:

• iteration speed matters more than artifact control
• the deployment is local or internal enough that packaging controls add little value
• resource resolution and signing are not yet part of the delivery need

Prefer packaged .oapp delivery when:

• you are delivering a productized workflow artifact
• resources should be resolved and embedded at package time
• signer identity or package integrity must be enforced
• embedded commercial controls or protected distribution matter

Architectural tradeoff:

• raw config optimizes for fast change
• packaged delivery optimizes for repeatable distribution, trust, and artifact-level control

For packaged designs, include in the architecture:

• package identity
• signer requirements
• recipient encryption strategy if needed
• resourceDirs and resource ownership
• versioning and rollout expectations

Primary provenance:

• docs/src/llm/processed/administrating/packaging.md
• docs/src/llm/processed/administrating/cryptography.md
• docs/docs/workflow.md

10. Design trust, keys, and encryption as first-class concerns

In Opscotch, cryptography affects system shape.

Choose key purpose based on responsibility:

• sign for artifact integrity and signer identity
• authenticated for sender-and-recipient asserted encryption
• anonymous for recipient-only secrecy where sender identity is not part of the contract
• symmetric for shared-secret encryption cases

Architectural implications:

• package trust requires the runtime to know which public identities it accepts
• authenticated package encryption requires sender and recipient identity placement to be planned across packager and bootstrap
• bootstrap encryption and string encryption affect installation workflow, secret handling, and operator runbooks

BYOK matters architecturally because it separates:

• who creates keys
• who holds secret material
• who verifies artifacts
• who can rotate which trust boundary without rebuilding everything

Primary provenance:

• docs/src/llm/processed/administrating/cryptography.md
• docs/src/llm/processed/administrating/packaging.md
• docs/src/llm/preprocessed/bootstrap-schema.md

11. Design the licensing model, not just the license field

Licensing choices affect topology and lifecycle.

Use remote or centralized licensing when:

• entitlements must change without rebuilding packages
• multiple runtimes should draw from a controlled platform-level pool
• the commercial model is part of operating the solution, not just shipping it once

Use embedded-license packaging only when that legacy delivery shape is still intentionally required and version applicability is confirmed.

Architectural questions:

• where will PLATFORM licenses live?
• should runtimes obtain licensing from a licensing app?
• does the solution need different environments or organizations to have separate entitlement boundaries?
• what is the renewal and expiry handling story?

Treat licensing as complementary to runtime policy:

• licensing governs entitlement to execute or consume the product
• bootstrap policy governs what the runtime may access while doing so

Primary provenance:

• docs/src/llm/processed/administrating/licensing.md
• docs/src/llm/processed/security/licensing.md
• docs/src/llm/processed/core-capabilities/commercial-control.md
• docs/src/llm/processed/security/workflow-firewall.md

12. Test at the solution-shape level

Testing in Opscotch is most valuable when it validates the designed workflow shape, not just isolated snippets.

Architectural testing goals:

• verify trigger entry assumptions
• verify outbound host interactions and authentication flow
• verify step-to-step orchestration and error paths
• verify persistence assumptions
• verify emitted logs, metrics, or responses

Design the solution so it is testable:

• keep external dependencies explicit through bootstrap and hosts
• keep resources parameterized
• make controller and worker responsibilities observable
• define stable test scenarios around business outcomes

The built-in test harness is best treated as mocked integration testing for the workflow product.

Primary provenance:

• docs/src/llm/processed/administrating/testing.md
• docs/src/llm/processed/introduction/getting-started-full.md

Architectural decision checklist

Use this checklist before implementation:

1. Suitability

• Does the solution fit supported triggers, runtime boundaries, and processor constraints?
• Is the business value in multi-step collection, reduction, correlation, or controlled distribution?

2. Boundary

• Which concerns are installer-controlled and belong in bootstrap?
• Which concerns are updateable logic and belong in workflow?

3. Topology

• How many deployments are needed?
• How many workflows are needed?
• Which steps are controllers, workers, auth helpers, or terminal steps?

4. Data

• What belongs in bootstrap data, workflow data, step data, processor data, trigger input, or persistence?

5. Trust

• Is raw JSON enough, or is a packaged artifact the right delivery form?
• What signer, encryption, and key-ownership model is required?

6. Commercial model

• Does the solution require centralized licensing, embedded licensing, or no special licensing architecture?

7. Operability

• Can operators explain what starts it, what it may access, what state it keeps, and how to update it?

Unresolved design-impacting unknowns

The reconciled corpus still leaves some architectural gaps visible. Design conservatively around them.

• Persistence lifecycle details are incomplete. Treat persisted state as durable step-scoped state, but do not assume undocumented retention, cleanup, or expiration behavior.
• Data merge behavior for conflicting value types is not fully documented. Avoid depending on surprising type changes across merge layers.
• Authentication-flow behavior across multiple requests or credential-caching patterns is not fully documented. Keep auth logic explicit and request-scoped.
• A unified runtime error contract for capability-boundary violations is not fully documented. Assume deny-by-default and design observable failure handling.
• Split-aggregate partial-failure strategy is still underdocumented. If fan-out failure handling is business-critical, make the aggregation and fallback strategy explicit in step design.
• The exact removal version for the legacy embedded-license path is not named in the corpus. Confirm version applicability before choosing that architecture.

Primary provenance:

• docs/src/llm/operations/unknowns/docs/patterns-unknowns.md
• docs/src/llm/operations/unknowns/security/workflow-firewall-unknowns.md
• docs/src/llm/operations/unknowns/security/licensing-unknowns.md
• docs/src/llm/operations/unknowns/apireference-unknowns.md

Short stakeholder explanation template

When explaining an Opscotch design, use this structure:

1. Why Opscotch fits

• The problem requires [trigger shape], [data-access shape], and [transformation/correlation shape], all of which match documented Opscotch capability.

2. Why the boundary is drawn this way

• Installer-controlled permissions, trust, and environment data are in bootstrap.
• Updateable business logic and step flow are in workflow.

3. Why the step design looks this way

• Each step owns one responsibility boundary: entry, auth, collection, enrichment, aggregation, persistence, or delivery.

4. Why the delivery and trust model looks this way

• The solution uses [raw config or packaged artifact] because the main priority is [iteration speed or controlled distribution/trust/commercial control].