Skip to main content

How can I source Opscotch environment variables from an external secrets source

· 4 min read
Jeremy Scott
Co-founder

Opscotch 3.1.1 adds OPSCOTCH_SECRETS_FROM, a startup-time way to load environment-variable values from an external source.

This is intentionally much narrower than a full secret-manager integration. The runtime reads a single blob once at startup, parses it as environment-style properties, and uses matching keys to override environment-variable lookups and bootstrap ${VAR} substitutions.

If OPSCOTCH_SECRETS_FROM is set, the external-secrets loader initializes before the rest of startup environment access happens.

The loader then:

  1. Reads one source.
  2. Parses the content as a properties file such as KEY=value.
  3. Records the available keys.
  4. Uses those values when Opscotch later asks for environment variables.

This means you can keep sensitive values out of the process environment while still letting Opscotch resolve them through its normal environment-variable paths.

Supported Source Formats

The source must be a URI-like string with one of these protocols:

  • file:/path/to/secrets.properties
  • device:/path/to/secrets.properties
  • http://host/path
  • https://host/path

In the current implementation:

  • file: and device: are both treated as file reads.
  • http: and https: are fetched with a simple HTTP GET.
  • Unsupported protocols fail startup.

Example:

export OPSCOTCH_SECRETS_FROM='file:/run/secrets/opscotch.env'

Related Environment Variables in 3.1.1

The 3.1.1 release also added or expanded several startup environment-variable paths that interact with bootstrap loading:

  • OPSCOTCH_BOOTSTRAP: supply the bootstrap payload from an environment variable instead of a positional argument
  • OPSCOTCH_BOOTSTRAP_PRIVATEKEY: supply the bootstrap private key from an environment variable
  • OPSCOTCH_BOOTSTRAP_SECRETKEY: supply the decryption key material for encrypted bootstrap payloads
  • OPSCOTCH_STRING_SECRETKEYS: provide string decryption keys used by runtime string decryption paths

Those are separate features from OPSCOTCH_SECRETS_FROM, but they can also be supplied through the external-secrets blob when you want Opscotch to resolve them at startup.

Blob Format

The blob must contain environment variable like properties in KEY=VALUE format such as:

OPSCOTCH_BOOTSTRAP_SECRETKEY=senderPublicKeyHex/recipientPrivateKeyHex
OPSCOTCH_STRING_SECRETKEYS=key1,key2,key3
...

This is not a JSON API contract. The loader does not expect Vault JSON, AWS Secrets Manager JSON, or any other provider-specific response shape. If you use HTTP(S), the response body still needs to be properties-format content.

What Gets Overridden

There are two important places where the loaded values are used.

1. Environment-variable lookup​

When Opscotch reads an environment variable through, an external secret with the same key wins over the direct environment value.

2. Bootstrap ${VAR} substitution​

When bootstrap content is processed, environment values are merged with external-secret values and substitutes ${NAME} placeholders.

External secrets take precedence here too.

Practical Example

Suppose you want to keep the bootstrap decryption key outside the shell environment:

OPSCOTCH_BOOTSTRAP_SECRETKEY=senderPublicKeyHex/recipientPrivateKeyHex
export OPSCOTCH_SECRETS_FROM='file:/run/secrets/opscotch.env'

When Opscotch later asks for OPSCOTCH_BOOTSTRAP_SECRETKEY, it will receive the value from the external secrets blob.

The same pattern works for other Opscotch environment variables, including values used during bootstrap substitution.

Limits and Timeouts

Two optional environment variables control how the source is read:

  • OPSCOTCH_SECRETS_FROM_MAX_BLOB_SIZE
  • OPSCOTCH_SECRETS_FROM_READ_TIMEOUT

Defaults in the current implementation:

  • max blob size: 1048576 bytes
  • read timeout: 10000 milliseconds

Example:

export OPSCOTCH_SECRETS_FROM='https://secrets.internal/opscotch.env'
export OPSCOTCH_SECRETS_FROM_MAX_BLOB_SIZE='65536'
export OPSCOTCH_SECRETS_FROM_READ_TIMEOUT='5000'

What It Does Not Do

OPSCOTCH_SECRETS_FROM does not currently provide:

  • provider-specific authentication configuration
  • custom HTTP headers
  • POST requests
  • automatic refresh or rotation
  • polling or reloading after startup
  • secret selection from nested JSON payloads

For HTTP(S), the runtime performs a plain GET and expects a properties-format response body.

Failure Modes

The implementation defines four external-secret error paths:

  • ES1: invalid config such as unsupported protocol, malformed source, or invalid numeric limits
  • ES2: read failure, missing file, I/O failure, or non-2xx HTTP status
  • ES3: read timeout
  • ES4: invalid properties blob or blob exceeding the configured max size

If the source cannot be loaded successfully, startup fails rather than silently continuing with a partially loaded blob.

Related Bootstrap Errors

The same 3.1.1 startup path also introduced bootstrap errors:

  • BL10: missing environment variables during bootstrap substitution
  • BL11: encrypted bootstrap decryption key missing, typically OPSCOTCH_BOOTSTRAP_SECRETKEY

These are not OPSCOTCH_SECRETS_FROM errors themselves, but they are part of the same startup/bootstrap configuration area covered by the 3.1.1 release notes.

When to Use It

Use OPSCOTCH_SECRETS_FROM when you want Opscotch to resolve sensitive environment-variable values from a file, device path, or simple HTTP(S) endpoint at startup.

Do not describe it as a general secret-manager integration layer. In the current codebase it is a single-source, startup-only properties loader that overrides environment-variable resolution for matching keys.