Pipeline registration¶

A pipeline is a deterministic, multi-step control flow written in the pipeline DSL. A DSL file may hold one or more pipeline: documents (private helper pipelines co-located with an entry point). Once registered, an agent can launch a pipeline by its fully-qualified name with run_pipeline, and one pipeline can call another. (Before #3429 a per-pipeline pipeline__<name> catalog verb also resolved to the same launch — see "Adding a pipeline" below for the current, only form.)

Namespacing is always on. Every pipeline registers under a global name of the form {entry-key}.{pipeline-name} — where {entry-key} is the config entry key and {pipeline-name} is the document's declared pipeline: name. The entry key is a pure namespace label; it does not need to equal any declared name.

Registration: explicit entries, no directory scan¶

Pipelines are registered purely via pipelines.entries declarations in config — the same explicit-registration model as skills.entries / mcp.servers. There is no directory scan; a pipeline DSL file sitting on disk with no config entry is invisible to every session.

# reyn.yaml
pipelines:
  entries:
    greetings:                 # the entry KEY is the namespace label
      path: pipelines/hello.yaml
      description: "Minimal greeting pipeline"
      enabled: true

Adding a pipeline¶

Write one or more Appendix-B DSL documents (----separated) per *.yaml file. Each declares its name with a top-level pipeline: key:

# pipelines/hello.yaml
pipeline: hello
description: Minimal greeting pipeline.
steps:
  - transform: {value: "'Hello, ' + ctx.name + '!'", output: greeting}

Declare a pipelines.entries.<key> entry pointing at the file (see above). The key is a pure namespace label — it can be anything (it need not match any declared pipeline: name).
Start (or restart) the session. The pipeline registers under its fully-qualified {key}.{name} global name, and an agent can launch it:

run_pipeline(name="greetings.hello", input={name: "Reyn"})

That is the only form. (Before #3429 a per-pipeline pipeline__<name> spelling also resolved; it was a second name for the same call.)

Namespacing: `{entry-key}.{pipeline-name}`¶

Every pipeline registers under the global name {entry-key}.{pipeline-name} — uniformly, regardless of how many pipeline: documents the file holds. The greetings entry above pointing at a pipeline: hello document registers as greetings.hello. The config entry key is a pure namespace label; unlike the old model it does not need to equal any declared name (this brings pipelines in line with the looser skills.entries / mcp.servers precedent).

Multiple pipelines in one file¶

A file may co-locate a helper pipeline with the entry point that uses it — each ----separated pipeline: document registers under the same namespace:

# pipelines/order_flow.yaml
pipeline: main
steps:
  - call: {pipeline: interrogate, output: verdict}   # dot-less → sibling
---
pipeline: interrogate
steps:
  - agent: {prompt: "Assess the suspect: {pipe}"}

Under entries: {orders: {path: pipelines/order_flow.yaml}} this registers orders.main and orders.interrogate.

`call`/`match` target resolution — the dot/no-dot rule¶

A call (or match) step's pipeline: target resolves by whether it contains a .:

No dot (call: {pipeline: interrogate}) — a same-file sibling reference; resolves to {entry-key}.interrogate. A dot-less target with no matching sibling in the same file is a load-time error (fail-loud; there is no silent fallback to some unrelated global pipeline).
Has a dot (call: {pipeline: other.helper}) — a global reference, resolved against the whole registry (other.helper), unchanged.

. is reserved as the namespace separator — it is forbidden in both a declared pipeline: name and a config entry key (a load-time error). This is what makes the dot/no-dot rule unambiguous: a local name has zero dots, a global name has exactly one.

Config cascade¶

pipelines.entries merges across the same tiers as every other config section, later tiers winning on name collision:

~/.reyn/config.yaml — user-global
reyn.yaml — project
reyn.local.yaml — project-local (gitignored)
.reyn/config/pipelines.yaml — runtime-dynamic, written by the pipeline_install_local / pipeline_install_source tools

Hand-editing any of the first three is a normal way to register a pipeline; the fourth is written automatically by the install tools below and reflects what a session installed for itself.

Failure behavior — per-entry isolated, visible but non-fatal¶

Loading is per-entry isolated: a broken declaration is never silently dropped, but it also never takes down the rest of the session's pipelines (or the session itself). At session-factory time (every reyn chat / reyn web startup), a broken entry is caught, logged as a warning, durably recorded as a pipeline_load_failed event (readable via scripts/dogfood_trace.py / the raw .reyn/events/direct/cli/*.jsonl files), and skipped — every other declared entry still loads and registers normally:

Condition	Behavior
Malformed DSL file	That entry is skipped; logged + durably recorded, naming the offending file. Other entries still load.
Entry key contains a `.`	That entry is skipped; logged + durably recorded (`.` is the reserved namespace separator). Other entries still load.
A declared `pipeline:` name contains a `.`	That entry is skipped (malformed file); logged + durably recorded.
Two `pipeline:` documents in one file declaring the same name	That entry is skipped (malformed file); logged + durably recorded.
A dot-less `call`/`match` target with no matching same-file sibling	That entry is skipped; logged + durably recorded, naming the unresolved target. Other entries still load.
Two entries producing the same global `{key}.{name}`	The FIRST-registered (config declaration order) wins; the later, colliding entry is skipped and logged + durably recorded.
An entry's `path` does not exist	That entry is skipped; logged + durably recorded, naming the path. Other entries still load.
No `pipelines.entries` declared	No pipelines registered (empty registry) — not a failure, nothing logged.

This is a deliberate middle ground between two failure postures neither of which fit: fully silent (a typo could vanish a pipeline the operator meant to ship, with zero trace — the earlier design's own stated reason for fail-loud) and fully fatal (the original fail-loud design — the first broken entry anywhere in pipelines.entries used to crash the ENTIRE session, which meant one unrelated pipeline's typo could take down reyn chat / reyn web startup entirely). Per-entry isolation keeps a broken entry visible to the operator (warning + durable event) while letting every healthy entry — and the session itself — start normally.

The hot-reload seam (/reload, Session._reapply_pipelines) is the one exception: it opts back into the OLD atomic, fail-loud posture (any broken entry aborts the WHOLE rebuild, leaving the previously-loaded registry fully intact) — a live session's already-running pipeline registry should never have an entry silently vanish out from under it mid-reload, so a broken edit at reload time is rejected wholesale rather than partially applied.

Installing pipelines¶

Two chat-callable tools under the pipeline_management category write pipelines.yaml entries — there is no reyn pipeline CLI equivalent (pipeline management is a chat-driven, in-conversation flow, mirroring skill_management).

`pipeline_install_local`¶

Registers a local pipeline DSL file into .reyn/config/pipelines.yaml:

Parses the DSL file at the given path (one or more pipeline: documents — validation step; a malformed file is refused).
Resolves the namespace key — the optional name argument, or the DSL file stem when omitted. The key is a pure label (. reserved); every pipeline: document in the file registers as {key}.{declared-name}, and the full set of registered names is enumerated in the result and the audit event.
Threat-scans every pipeline's description (strict scope) — blocks on a blocking-severity match.
Gates the pipelines.yaml write through the standard require_file_write permission flow.
Writes the entry, records a config generation (crash-recovery — survives WAL truncation), emits a pipeline_installed P6 event, and requests a hot-reload.

`pipeline_install_source`¶

Fetches a pipeline from a git/GitHub URL and installs the clone:

Gates require_http_get for the source host.
Shallow-clones the repo (--depth 1) to .reyn/pipelines/<name>/. A //subdir suffix on the URL (mirroring Terraform's module-subdir convention) selects a subdirectory of the clone instead of its root.
Locates the DSL file in the clone — an explicit path argument selects it when the repo/subdir contains more than one *.yaml file — then proceeds through the same parse → namespace-key → threat-scan → gate → write → hot-reload pipeline as the local path, with the registered path pointing at the installed copy.

Path-safety hardening (both tools, since the namespace key feeds a filesystem path under .reyn/pipelines/): the key — from the name argument or the source/file basename — is rejected outright unless it is a single safe path component ([A-Za-z0-9_-]+, no ., no .., no separators; . is reserved as the namespace separator). A belt-and-suspenders containment check (resolve() + relative_to()) additionally refuses any install destination that would resolve outside .reyn/pipelines/, guarding against a gap in the name check itself. Neither check silently rewrites an unsafe name — installation is refused with an explicit error instead.

Hot-reload¶

Edits to .reyn/config/pipelines.yaml (or to pipelines.entries in reyn.yaml / reyn.local.yaml) take effect at the next turn boundary via the "pipelines" reload seam — no session restart needed. See Concepts: Config hot-reload.

Spawned pipeline-driver registry — no explicit hand-off (#3097)¶

When a hook's pipeline_launch action starts a pipeline run (start_pipeline_run → _spawn_pipeline_driver_session), the calling session does NOT pass its own live PipelineRegistry object to the spawned driver session.

Why this is safe. The spawned driver session goes through AgentRegistry.spawn_session_recorded, which calls refresh_config_projections — this fires the driver session's OWN _reapply_pipelines seam (the same hot-reload seam described above) fires uniformly at spawn time, rebuilding the driver's PipelineRegistry directly from the current on-disk cascade.

What this replaced. An earlier design (#3094) forwarded the CALLING session's live in-memory PipelineRegistry object to the spawned driver as a spawn-local override. That is redundant with the reapply-seam rebuild above, and worse: it hands the driver a registry snapshot that can already be stale relative to disk (e.g. a concurrent pipeline_install_* landed between the calling session's own last reload and the spawn). #3097 folded that override out — the family gate (spawn → reapply seam → fresh disk read) achieves the same effect without depending on the caller having a fresh in-memory copy to hand off.

Re-adding an explicit hand-off here would reintroduce that staleness risk.

Security — launching a pipeline stays gated¶

Registering a pipeline does not loosen the capability floor. Launching a pipeline (run_pipeline, whether name= or definition=, and whether collect="attached" or collect="async") is on the same restricted floor as spawning a sub-session or re-delegating: a pipeline step can itself write, execute, or delegate, so a pipeline launch is a cost-bound multi-step dispatch.

As a result, a context narrowed by the _untrusted floor (untrusted external content is live AND safety.threat_scan.capability_narrowing is enabled — it is off by default) or the _delegate floor (an unbound delegate under delegation.capability_default=deny) cannot launch a pipeline, whether or not one is registered. Loading a pipeline definition makes it available to authorized agents; it never creates a bypass of those floors. See Capability profiles and Delegation policy.

The pipeline_install_local / pipeline_install_source verbs (the REGISTRATION action itself, distinct from launching) sit on the same untrusted-content / unbound-delegate floor as the skill_install_* and mcp_install_* verbs — no registering a pipeline from untrusted content either.