Cazi: A Deterministic Harness for AI Agents

Cazi turns “run this tool” into a durable, auditable, replayable record. An AI agent declares an intent. Cazi runs it as an ephemeral Docker container on a real execution node, stores the full spec, events, logs and exit code, and routes the output wherever the manifest says. It is a single Rust binary on Axum, PostgreSQL and Bollard, with a first-class MCP server so an agent can drive it directly.

The problem

An agent that can run shell commands is an agent with no paper trail. It invents flags, retries with different ones, and produces real side effects that nobody can replay. Intelligence can afford to guess. Consequence cannot.

The usual answer is to sandbox the agent harder. Cazi takes the other side of the line: let the agent decide, and make the decision land on something deterministic. Three constraints follow.

• Intents over instructions. Tools are declared as YAML workload manifests, not free shell commands. The agent picks which tool fits. The manifest defines exactly how it runs.
• Intents persist, outputs are ephemeral. The manifest is the source of truth. Every task is stored and replayable. Container output lives only as long as it is useful.
• The agent never shells out. It submits a task over MCP and gets back a task id. Everything after that is Cazi’s, and recorded.

Architecture

Cazi is one binary in three modes. The control plane owns the API and the database. Node agents execute containers against a Docker endpoint. The MCP server proxies an agent’s intents into the control-plane API over stdio.

Every entry lands in the control plane before anything runs. Drawn as text, rendered by Vizo.

The control plane can run an embedded node, which is the single-machine default, or attach standalone nodes on other hosts. Model turns never run in the control-plane process: agent workers run them with no database and no provider key, and reach the control plane only through the API, which is where allowlist and scope are enforced.

A task’s life

The unit of intent is a WorkloadSpec: image, command, environment, mounts, resources, network, plus the controls that matter to a harness. timeout_seconds, max_retries, priority, ttl_seconds, pull_policy, an output_format, output_sinks, and a free-form rationale the UI renders so a reader knows why the task exists. Submit it as YAML over MCP or JSON over REST.

image: alpine:latest
command: [sh, -c, "echo hello from cazi"]
timeout_seconds: 60
output_format: text
rationale: |
    Smoke-test that the node can pull and run a container.

From there the task is event-sourced. Every transition writes an event, and the events are the status.

1. Submitted. The spec is stored and checked against active learnings. A matching error learning blocks it here.
2. Scheduled. The scheduler picks a node in a fixed order: a node named in the selector, then nodes whose labels contain the selector’s, then any node whose tags cover the task’s, then any online node the tenant may use. A task whose tags no node satisfies stays pending rather than running somewhere wrong.
3. Claimed. Node agents poll to claim work and heartbeat back. A node silent for 30 seconds is offline. On startup a node reconciles what the control plane thinks is running against what Docker actually has.
4. Running. The node pulls the image, creates the container through Bollard and streams stdout and stderr back as log entries. Before the container starts, the node injects CAZI_WORKLOAD_TOKEN, an Ed25519-signed JWT naming the task and the person it acts for, so a service the task calls can verify who is knocking against the control plane’s JWKS endpoint.
5. Terminal. succeeded, failed or cancelled. Stdout is parsed per output_format (json, ndjson, markdown or raw text). Output sinks fire: a webhook, an S3 upload, a file, or a downstream task. Files the container published to its artifact URL stay hosted for seven days by default.
6. Replayed, or disposed. Replay clones the spec into a fresh execution and records replay:<id> as the source. When ttl_seconds passes, the record, logs and events are hard-deleted.

The container is disposable. The record is not.

Pipelines are Unix pipes between containers

One tool does one thing. A bigger job is a chain of small tools, not one container running a && b && c. Cazi takes that literally: a pipeline’s step declares stdin_from_prev: true, and the previous step’s stdout is mounted at /pipe/stdin in the next container. Tools read the file explicitly, because container stdin is unreliable across images.

name: recon
steps:
  - name: subdomains
    image: projectdiscovery/subfinder:latest
    command: ["-d", "example.com", "-silent"]
  - name: resolve
    image: projectdiscovery/dnsx:latest
    stdin_from_prev: true
    command: ["-l", "/pipe/stdin", "-silent", "-a", "-resp-only"]
  - name: probe
    image: projectdiscovery/httpx:latest
    stdin_from_prev: true
    command: ["-l", "/pipe/stdin", "-silent", "-json"]

The engine is a DAG, not a list. When a step ends, on_task_terminal marks it and calls advance_pipeline, which launches every step whose dependencies are satisfied, in parallel. That call holds a Postgres advisory lock per pipeline, so two sibling steps finishing at the same moment cannot double-launch a shared fan-in child. A failed step fails the pipeline and fires its webhook once.

The discipline that makes the pipes work is on the tool side. Stdout is data, ideally JSON Lines. Logs go to stderr. A step that prints a progress bar into stdout poisons the next step’s input.

Templates close the loop. A spec that worked once is saved with a TARGET placeholder, and the next run substitutes a new target into it instead of rewriting flags from memory. The substitution is whole-word, so a Metasploit option called TARGETURI survives. The agent instructions say it plainly: before writing a spec, check the template catalog. A template already has the right image and the tool’s real CLI flags, verified once.

Trust is enforced at the API, not in the prompt

An agent in Cazi is a persona with a reviewed allowlist and a scope. The allowlist starts empty, and empty means no tools. * must be written explicitly, and background runs refuse it. Every edit is a new immutable version, reviewed by a tenant admin, and a run pins the version it started with. Nothing the agent does during a run can widen what it may do.

The enforcement point is one interceptor, governance::decide, in front of every tool call. It checks, in order: is this a management tool reserved for people, is the agent allowed this tool at all, does the agent’s access level permit a write or a destructive action, and are the arguments inside scope for images, nodes, connections and threads. Any failure is a denial with a reason, written to the activity log. The model’s tool loop cannot route around it, because the HTTP call underneath carries a credential the API checks against the same scope.

Goals the agent cannot write. A person sets the objective, success criteria, constraints and budget. The agent must propose a plan before it changes anything, and each planned step is checked against the pinned allowlist before anything runs. A call that serves no planned step is flagged as drift.

Circuit breakers. Eleven ceilings guard a run: tokens, turns per stretch, lifetime turns, tool calls per turn, wall clock, repetition, consecutive failures, drift, the shared budget of a delegation tree, daily spend, and provider refusals. A denial counts as a failure, because an agent repeatedly hitting its own walls is stuck. When a breaker trips, the run pauses and asks a person instead of burning more.

Credentials that die with the turn. A run acts under a self-issued HMAC token minted per turn and checked against the live run on every use. It is revoked the moment the run stops, with no OpenBao round trip in the hot path. OpenBao holds the long-lived secrets, and those never reach a run at all. Model turns can run on separate workers that hold no database URL and no provider key; a worker’s job token is good for its own turn’s three endpoints and nothing else.

Delegation only narrows. A child run gets the intersection of its own allowlist and scope with its parent’s. Depth is capped at three, children at five per run, runs at twenty per tree. Budgets count across the whole tree, and no child outlives its parent.

Nothing anonymous, nothing unrecorded. Every task, run, schedule and catalog item carries a who record resolved server-side: the actor, the accountable person when an agent or schedule acted for them, and the route in. The activity log is hash-chained per tenant by a BEFORE INSERT trigger, SHA-256 over the previous hash and every field, so no writer can skip it. The chain head is exposed for copying somewhere Cazi cannot write, and pruning records how far it went so a verifier can tell retention from tampering.

The design document puts it in four words: verifiable, not “deterministic”. Model output is not deterministic. The record of what the agent did is.

Memory that compounds, without gaining authority

Three catalogs let Cazi get better at a job without a human retyping anything. Each is versioned, each names who wrote it, and none of them can widen what an agent may do.

Catalog	What it holds	Who may activate it
Learnings	A failure pattern, its root cause and fix, matched against incoming specs	A person. Agents only propose, capped at warning
Skills	A named playbook for a recurring class of work, loaded with use_skill	A person reviews each version
Agents	A persona: role, the skills it works from, the tools it keeps to	A tenant admin reviews each version

Learnings are the loop that turns failure into prevention. Every task failure is captured automatically as a candidate. A failed run or a tripped breaker becomes one too. A person promotes a candidate by adding the cause and fix and raising its severity: warning surfaces in the next matching submission’s response, error blocks it before it runs. Only active learnings match. A learning nobody has triggered for 90 days goes stale and needs re-review, so the catalog does not fill with advice about tools that no longer exist.

Reviewed lessons about a specific agent reach that agent’s context, marked as data rather than instructions and naming who reviewed them. That distinction matters. An agent that could write its own lessons into its own prompt would, in effect, be editing its own instructions.

The same shape governs templates: a working spec is saved once with a TARGET placeholder, and the next run substitutes rather than rewrites. Learnings remember what went wrong. Templates and skills remember what went right. Agents bundle both into a role. Semantic search over all of it runs on pgvector, so an agent asking “is there something for this?” gets an answer before it invents one.