Architecture

This page maps the binary, request path, and deployment interfaces.

One binary, two roles, three shapes

The three deployment shapes

Everything is one Go binary. It has two roles, and a deployment picks a shape by which listener it opens.

The gateway role owns listen. Clients point an OpenAI or Anthropic SDK, an MCP client, an A2A client or a realtime WebSocket session at it. It decides whether the request may proceed, inspects it, swaps the credential and forwards it. As the front door it also terminates TLS and its replicas form a raft cluster, so a key minted on one node is valid on the next. Behind a gateway it is the same process minus TLS. A Gateway API HTTPRoute sends it the traffic and it does the rest.

The inspector role owns extproc.inspect_listen. It applies the same detectors and the same policy to traffic it does not route, an ext_proc filter on a gateway you already run. It is a separate listener rather than a flag because a filter pointed at it must have no code path to a routing decision. The inspector tutorial builds this shape from nothing.

A request, end to end

A request in three bands, admit, send and deliver, with the provider beside send and deliver; steps 3 and 7, request guardrails and the credential swap, are the two that change the bytes

The request follows the order below. Early checks avoid provider calls and leave the request unchanged when they refuse it.

Admit validates the request before selecting a provider. The client authenticates with a pistra_ virtual key, found by its SHA-256 hash. The token itself is never stored (1). CEL access rules run over the key, the model, the provider and the route (2). Request guardrails run next, so that every later stage sees the redacted text. The fidelity probe, the cost estimate and the upstream request all work from the redacted bytes (3). Then the channel is chosen, native passthrough if the client and provider share a dialect, a provider’s compatibility endpoint if not, translation only when nothing else matches. The fidelity guard refuses a channel that would silently drop a capability the request uses (4). Budget is reserved only after the provider has admitted the request, so a refusal above costs no hold (5). Channels and fidelity and translation describe steps 3 and 4. Schedules explains how a slow detector can start here and finish later.

Send makes one attempt against one provider. A model alias is patched in place with sjson, the one field a passthrough request changes, and a pool picks which of its model servers serves this request (6). The client’s credential leaves the headers and the provider’s replaces it (7). The attempt is then sent. A failure the route’s plan acts on, a timeout, a 429, a 5xx, is abandoned for what the plan says comes next, the same provider after a wait or the next admissible one, carrying the same body and the same budget hold (8). Nothing has reached the client yet, so the route can retry. See Passthrough for the unchanged request bytes.

Deliver returns the response. Response guardrails inspect it, and a stream is held only as far as a match could still reach, the commit horizon, rather than buffered (9). The token counts the provider reported are read from the response and the hold is settled against them. The gateway never tokenizes anything itself (metering) (10). Finally an audit record naming the principal, the decision and which detectors had seen the text is signed onto this node’s chain. The request’s metrics and trace span close (11). See the audit trail for the record shape.

The planes

What is in each of the four planes

A deployment has four kinds of traffic, and they never share a listener.

The data plane is listen and, for the inspector role, inspect_listen. It is the only plane clients ever see.

Cluster state is one raft log per cluster. It holds the things that must be one set for the cluster rather than one per node: virtual keys and their revocations, budget spend, ACME certificates, the configuration sources, secrets, sealed before they are proposed, so the log never holds a plaintext credential, and the signed heads of each node’s audit chain. A node reads and spends against it. It does not hold a private copy of any of it. The threat model lists what is sealed and how.

The admin plane is the admin API: admin.listen for the network, which serves nothing until it is configured, and admin.local_socket for a break-glass door that only the host can open. Everything that writes configuration writes through it, the console, the pistra CLI, the Terraform provider, the Kubernetes operator. Each writes under an OIDC identity and a Cedar policy, and each into its own named source document, so an operator and a human editing at the same time never contend for one file. Its writes land in the raft log, and the data plane applies them from there. The file on disk is a seed on first boot and a source like any other afterwards. Manage the configuration through the API is the how-to.

An AI governance platform writes rules to its own source. It cannot change a provider, credential, or another team’s document. See Connect your AI governance platform for the integration contract.

Evidence leaves the node and is never read back by it. Metrics go to Prometheus from metrics_listen and to a collector as OTLP from the same registry. Traces follow the GenAI semantic conventions, and audit records go out on their own OpenTelemetry scope through a synchronous exporter. A collector’s copy of the audit trail is verifiable by pistra audit verify against the heads the cluster witnessed. That check makes someone else’s storage evidence rather than a log.

The same platform reads the decisions here. Each record carries the rule and its controls, linking the platform’s policy to runtime evidence.

The four planes touch in four places, and nowhere else.

The admin plane writes to the raft log; the data plane reads it and spends against it; both send evidence outward

What is deliberately not here

• No tokenizer in the request path. Usage is the count the provider reported, because a local count would disagree with the invoice silently. Metering.
• No prompt retention. Content is in flight only. The audit record names the decision and the detectors, never the text. Threat model.
• No Python sidecar for detection. The pattern tier and the ML tier run in-process. A remote Presidio is a detector type, not a requirement. The guardrail pipeline.
• No GatewayClass. On Kubernetes pistra is the front door behind a LoadBalancer, or an HTTPRoute target behind yours, or an ext_proc beside yours, never the thing that programs an Envoy. Deploy the front door with Helm.

Related

• Passthrough, channels and fidelity and translation, the three choices in step 4.
• Schedules and the commit horizon, how inspection avoids paying with latency.
• The audit trail, what step 11 writes, and why it is separate from the log.
• Every figure here is written by make gen-figures, and a stale one fails the build. Most are rendered from docs/figures/*.d2. The request figure is drawn by tools/genfigures/draw, for the reason recorded beside it there.