Skip to content

Processors

Processors are explicit executable bindings, not installed tools. A configuration may use the existing runner.command contract or declare [processors], but never both. All commands run without a shell. Managed toolchains supply the exact executable paths through environment bindings; Schepherd contains no dependency version pins.

config_version = 1

[runner]
processors = ["native", "python", "javascript"]
unsupported = "error"

[processors.native]
type = "sourcemeta"
command = "${JSONSCHEMA_BIN}"

[processors.python]
type = "check-jsonschema"
command = "${CHECK_JSONSCHEMA_FRONTEND}"

[processors.javascript]
type = "ajv"
command = "${AJV_FRONTEND}"

A complete catalog-backed example is examples/processors.schepherd.toml. The Python and AJV commands are Schepherd's owned frontends, not the bare upstream CLIs. The frontend sources and dependency manifests live in tools/check-jsonschema/ and tools/ajv/. Bind their executable entrypoints in the same managed environment that provides their dependencies; do not rely on a system interpreter or adjacent executables being discovered automatically.

Selection

If runner.processors is omitted, declared processors are ordered by type: sourcemeta, check-jsonschema, ajv, custom; ties sort by processor name. An explicit list replaces the order and can restrict the candidates. It must be nonempty, contain no duplicates and reference only declared processors.

A local schema can set processor = "python"; a matching [[mappings]] entry can set its own processor and takes precedence. An explicit selection tries only that processor. Conflicting processor selections on matching mappings are errors, never skips.

Before running any document validator, Schepherd checks input format and prepares the schema with each candidate's actual executable. Verified compiler artifacts and capability decisions are reused from the schema cache. An unsupported capability response or schema compilation rejection permits trying the next candidate, before any document is validated. Schema files must first be readable, bounded, well-formed strict JSON; malformed JSON fails immediately, even under the skip policy. Missing binaries, crashes, malformed probe responses and timeouts fail the run. Once document validation starts, its result is final: an invalid document is not retried with another validator.

runner.unsupported defaults to "error" (usage error, exit 2). "skip" records each unsupported file with its reason and unsupported-processor category in the version 3 machine report; raw mode prints the skip. It does not hide invalid documents, ambiguous mappings or infrastructure errors.

Fields

Processor field Meaning
type Required after extends merging: sourcemeta, check-jsonschema, ajv, custom.
command Required after merging. Executable template, resolved like runner.command.
cwd Consumer working directory; defaults to runner.cwd. Relative to its declaring file.
env Extra consumer environment; merges over runner.env. Values are runner templates.
inherit_env Defaults to runner.inherit_env.
output sourcemeta-json, check-jsonschema-json, ajv-json or raw; defaults to the typed JSON format, raw for custom.
args Required for custom; arbitrary runner argument templates. Typed presets reject overrides.
mode Custom runner mode: batch, per-file, stdin; default per-file. Typed presets reject overrides.
convert Optional explicitly configured converter executable for source formats listed in formats.
convert_args Required argv templates when convert is set; passed literally without a shell.
formats With convert, source formats to convert. For custom without conversion, an optional input allowlist.

runner.timeout, jobs, fail_fast and max_args_bytes remain common execution settings. runner.args and runner.mode apply only to the legacy command and are rejected alongside processors. Typed processors always invoke one input per task; custom batch processors retain grouping by schema. Mixed custom modes in one run are rejected instead of reporting an incorrect common mode.

Native and frontend contracts

Type Capability probe Document invocation Native formats
sourcemeta jsonschema compile <schema> jsonschema validate <schema> <input> --json JSON, YAML
check-jsonschema frontend probe <schema> frontend validate <schema> <input> JSON, YAML, TOML, JSON5, JSONC
ajv frontend compile <schema> frontend validate <schema> <input> --compiled <template> JSON
custom None; compatibility is the configuration author's assertion Configured arguments and mode All, or explicit formats allowlist

Sourcemeta uses its documented exit classes: compilation exit 3 is unsupported; exit 4 rejects the schema for that processor. Both permit selection to continue before document validation. Exit 4 also covers unknown dialects, vocabularies and compiler limitations, so it is not proof that every validator rejects the schema. If all candidates reject it, selection fails (or records an explicit unsupported skip), retaining the rejection reasons. Other nonzero probe exits are infrastructure/configuration failures. Compilation stdout is retained as a verified template and passed to validation with --template. Sourcemeta receives an explicit empty tool configuration for both steps: a consuming project's ambient jsonschema.json must not silently change a managed compilation. No HTTP-enabling flags are added.

The owned frontends return exactly one capability JSON object on stdout: {"supported":true} or {"supported":false,"reason":"..."}, with exit 0. A schema rejection or infrastructure failure exits nonzero instead of pretending to be an unsupported capability response. Frontends deny remote reference retrieval. Their document command emits structured JSON, exiting 0 for valid and nonzero for invalid. A frontend's missing dependency must fail, never request another processor. Probe exit 4 is a schema rejection eligible for prevalidation selection; exit 2 is a fatal usage/configuration error, and other nonzero frontend probe exits are infrastructure failures.

AJV's compile returns the capability object plus a template string containing standalone compiler output. The frontend resolves that artifact's dependencies from its own installation, not the input project. Its identity and the Python frontend's identity report source, runtime and dependency versions for the compilation recipe. Python has no portable serialized validator here: only its capability check is cached; the document process still constructs a validator. No pickle or input-generated code is loaded as a substitute.

Catalog compatibility profiles require input-independent processor settings. The publisher rejects processor environment templates containing {file}, {schema}, {schema-id} or {schema-ref}. At runtime those processors bypass recorded profiles and probe each input separately, so one file's compatibility cannot determine another's.

Prepared compiler artifacts

materialize --compile strictly prepares the configured processors while the source catalog is being installed. --compile-schema can restrict this to reviewed catalog IDs. materialize --compile-persistable prepares every compilation whose schema and processor settings can safely be persisted; invocation-local schemas are reported in the JSON summary with per-reason counts and are not treated as missing artifacts. The stored recipe includes schema SHA-256, executable SHA-256, frontend/runtime identity, explicit processor settings, the compiler recipe version and platform. Changing any of them creates a new entry. Artifact size and SHA-256 are verified before use; writes are atomic and locked across processes. Known incompatibilities are cached, but a compiler crash, timeout or unavailable executable never becomes one. Sourcemeta's Draft 7-or-earlier top-level $ref sibling incompatibility is recognized only by its exact native compiler diagnostic; other exit-1 failures stay fatal. An unresolved internal reference is a negative capability, not a successful validation: the catalog entry stays visible, and a run for it fails with each selected processor's reason if none supports it. Publication rechecks a recipe after acquiring its lock and serializes shared artifact bytes by their digest before installing metadata. Waiting writers keep the first verified result; cancellation never leaves a partial compilation record.

Sourcemeta's generated file URI for an anonymous schema snapshot is replaced with a content-derived urn:schepherd:schema:<sha256> diagnostic base. This preserves instance locations and messages without embedding temporary or installation paths in a relocated compiled artifact. Explicit schema identifiers remain unchanged.

Run against a writable cache can compile an absent persistable catalog entry locally without network. A complete cache can be relocated and read-only; materialize --offline --compile and materialize --offline --compile-persistable only verify persisted entries. Local schemas always compile in invocation-local temporary storage so edits remain visible; offline local validation still invokes the compiler but never reads or writes a persistent compiler cache. Catalog schemas with external references or a custom $schema, including one nested in another schema, remain invocation-local until their dependency graph is part of the recipe. Original schema files stay in place. Every document is still validated on every run; these are compilation caches, never cached validation results.

A strict catalog preparation with --compile refuses invocation-local schemas; use --compile-persistable to prepare the safe subset and review the reported reasons. A literal property named $schema remains ordinary instance data, not a dialect declaration.

Probes and converters have a 30-second timeout. Probe/conversion output is bounded to 1 MiB per stream; serialized compiler output is bounded to 64 MiB. Compilation is not document validation and never uses the user's document as a capability test.

Typed presets intentionally have fixed arguments: accepting arbitrary arguments while probing a different command would falsely claim compatibility. Use custom for a validator with project-specific flags; its compatibility and output declaration are then explicitly the author's responsibility. The legacy runner.command path remains unchanged and is reported as custom/raw; executable names are not guessed.

Explicit conversion

There is no automatic conversion. Native formats are preferred unless conversion is explicitly configured for a listed source format. Configure the exact converter executable and argument vector; the example uses yq:

[processors.native]
type = "sourcemeta"
command = "${JSONSCHEMA_BIN}"
convert = "${YQ_BIN}"
convert_args = ["-p={format}", "-o=json", "-I=0", "{file}"]
formats = ["toml"]

convert_args is passed literally without a shell and supports {file} and {format} templates. The converter must actually support each requested source format; a conversion failure is not permission to skip validation. The output must be exactly one strict UTF-8 JSON document, without duplicate keys. Multiple documents are rejected rather than silently selecting one.

Converted inputs are private temporary files, removed after the run. Machine tasks retain original input names in files and the actual validator inputs in paths. Diagnostics refer to original files but cannot claim source line/column coordinates from converted JSON. Native YAML inputs can retain multiple documents without conversion. Environment templates using {file} always name the original source document, including during compilation and after conversion. Argument templates still name the actual JSON input the validator must read. This keeps input-dependent compiler settings identical to the validation settings without losing source attribution.

Catalog sourceFormats metadata can identify exact filenames, including extensionless ones; known JSONC filenames and recognized extensions are also detected conservatively. Conflicting metadata and unknown extensionless inputs remain unsupported instead of being guessed as JSON. Use a custom processor if its own parser intentionally owns format detection.