Skip to content
pistra

The audit trail

The audit trail is a separate signal from the operational log, and the separation is the point.

An operational log line may be sampled, batched and dropped, and a reader treats a gap in it as noise. An audit record may not be. Unless its absence means the event did not happen, the trail is not evidence of anything.

Delivery

Section titled “Delivery”

Audit records travel on their own OpenTelemetry instrumentation scope, pistra/audit, through a synchronous exporter. Operational logs keep the batch processor, which is right for them. It drops on a full queue and on a crash, and nobody minds.

Three consequences:

  • Export happens on the emitting goroutine. By the time an admin mutation returns, its record has either left the process or been counted as lost.
  • A failure is counted, not swallowed. Every record the collector refuses increments a counter and is reported to the text log. It is served as pistra_audit_export_failures_total, the one series on /metrics to alert on at any value above zero.
  • The local copy is always written. Every record goes to the text output as one JSON line whether or not OTLP is configured, so a collector outage costs you the fan-out, not the trail.

Nothing here is enabled by a new flag. The trail follows the same OTEL_* environment variables as every other signal. With none set, it is the local copy alone.

Records are emitted on the OpenTelemetry Logs API directly rather than through the operational logger. The event name is the record’s event, the body is the record’s canonical bytes, and the chain fields below ride as attributes. A collector holds the same bytes the local copy does, and either can be verified.

Tamper evidence

Section titled “Tamper evidence”
Three nodes, each with its own hash chain of audit records, and a raft box holding only the head of each chain; a collector holds a copy of every record

Every record is a link in a chain kept per node. Each entry carries a sequence number, the hash of the entry before it, its own hash, over the previous hash, the node, the key, the sequence number and the record, and an ed25519 signature by the node’s audit key. The key is generated on first start into cluster.data_dir/audit.key (PKCS#8 PEM; audit.dir for a node without a data directory), and the head of the chain is kept beside it so a restart continues the chain rather than starting one.

A chain begins with audit.chain.start, carrying the public key and whether the chain was resumed from disk, is a genesis that will be kept, or is ephemeral. A node with nowhere to keep a key signs with one that dies with the process, which is honest but leaves its restarts unlinkable. It ends, when the node stops on purpose, with audit.chain.close. A chain without one was cut short.

pistra audit verify gateway.log
kubectl logs pistra-0 | pistra audit verify -

Verify the audit trail is the same command with each finding shown and explained.

verify reads a trail in any order with anything else interleaved and reports, per chain: whether every record hashes to its entry, links to the one before it, is signed by the key the chain started with, and whether any sequence number between the first and last seen is missing. A record altered, removed from the middle, moved, or forged is a finding with its sequence number and kind.

A chain by itself cannot show a trail cut at the end. Witnessed heads cover that case. Every node offers its signed head to the cluster about once a minute and once more after its close record. The cluster checks the signature and keeps the last day of them per node in its replicated state, where compaction cannot reach them. GET /admin/v1/audit/heads lists them, and verify -heads reports a trail that stops short of one. A cluster of one witnesses itself, which proves nothing on its own. There the collector is the witness.

Nothing here can show a key used by somebody other than the node. The trail proves what the key signed. Keeping the key where only the node can reach it is the deployment’s part.

What a record never carries

Section titled “What a record never carries”

A record never carries content. Guardrail records name entity types and count them. The matched text is not in the record and cannot be recovered from it.

The argument for in-process guardrails is that content does not leave the process. An audit record that carried the PII a rule had just refused to forward would violate that argument. It would also put the payload in the system most likely to be replicated to a third party.

The honesty fields

Section titled “The honesty fields”

Two fields exist because a reader would otherwise draw a conclusion the gateway cannot support.

deferred names the detectors that contributed only what they already knew about this text from an earlier turn. Their findings are real. Their silence is not evidence, because anything new in this body has not been seen by them yet. A record that named the rule and the entities but omitted this would read as a complete inspection.

It is emitted on every guardrail record, empty list included. Its absence must never be readable as “every detector ran”.

enforced states whether the outcome was acted on. It is false for every async record, because nothing waited for that scan and its verdict is what policy would have done. It is false in shadow mode, and false when an overlap redact rule fired after the bytes were already forwarded.

It is a field rather than something derived from schedule because a reader who has to reconstruct “was this actually blocked?” will eventually get it wrong in the direction that flatters the trail.

What is recorded

Section titled “What is recorded”

Every event, every field of the record and every attribute a collector can filter on are in the audit trail reference, which is generated from the package that emits them. This page used to carry that table by hand, and the table was wrong. It listed fifteen events where the code emitted twenty, and the five it had lost were the secret writes, the break-glass suspensions and the cluster membership changes, the records most likely to be the reason somebody is reading the trail. A list of what a trail contains is the kind of claim that must be generated, because a reader has no way to notice it is short.

The rest of this page covers what the vocabulary does not say: why some of those records are shaped the way they are, and why others do not exist.

A request.auth record carries a bounded reason and, for a denying access rule, the rule’s name. “Denied by policy” without naming which policy sends the reader back to the config file to guess.

The reasons themselves are not listed here. They are a closed vocabulary in the package that emits them, and the audit trail reference is generated from it. Generating the list is the only way it can be trusted, for the argument two paragraphs up.

The trail is allowed to know more than the client is told. Every authentication refusal is one indistinguishable 401 on the wire, so a prober cannot tell unknown from revoked. The record still names the key when the gateway recognized one, because an incident needs revoked key alice was presented from 10.0.0.4. The presented credential itself is never recorded. A mistyped token is very often a real one belonging somewhere else.

A request.budget record names the budget as its target, not just the key. The question it answers is “which limit stopped them”, and a record naming only the caller leaves that to be guessed from timing. budget_exceeded (a limit was reached) and budget_error (the ledger could not answer) stay separate reasons, because conflating them would report an outage as a spending pattern.

Two omissions are deliberate.

Reads and admitted requests are not audited. A trail of every GET, or of every request that went through, is the traffic log, and burying the refusals in it is how an audit trail stops being read. Successful authentication is not its own record either. It is implied by the record of what the request then did.

The one read with a record of its own is admin.probe, the MCP server/discover check. Its effect lands on somebody else. The bytes reach the peer’s server carrying the deployment’s credential, in their logs and against their rate limits, so the trail has to be able to say who caused a request the peer saw. The record states that an operator asked this gateway to probe the named server, and its status states what they were told.

Not every refusal is audited. A malformed body, a fidelity mismatch or an unresolvable endpoint subset are the gateway failing to route, not a decision about a caller. Those stay counted and logged.

One case is recorded even though nothing was refused. In it, policy was configured to look and could not, because the body was too large to buffer or could not be parsed. That is the only way a configured guardrail silently does not apply, so it is recorded rather than assumed away.

One stream is one event

Section titled “One stream is one event”

A guard that refused mid-stream is finished, and the pump does not call Close on it. That was true of every pump before it was written down. It is now part of the StreamGuard contract, and the guard is idempotent regardless. Reporting a stream twice would double a counter and, far worse, put two records in the trail for one event.

The actor

Section titled “The actor”

Every record names who caused it.

For the data plane and the inspector, the actor is whoever sent the traffic: the virtual key, or anonymous when none was presented. Traffic on someone else’s API is not pistra’s to authenticate.

For the admin API, the actor is whoever the issuer says it is. There is no shared admin secret and never was one worth keeping. A string in a config file identifies the credential, not its holder, so a trail built on one can only record that the string was used.

issuers:
  - name: corp
    url: https://login.example.com


admin:
  listen: 0.0.0.0:8485
  policy_file: /etc/pistra/admin.cedar
  issuers:
    - issuer: corp
      audience: pistra-admin
      roles: {claim: groups, map: {"CN=pistra-admins,OU=Groups": [keyadmin]}}

The name identifies a caller, not a permission. The Cedar policy in admin.policy_file decides separately what they may do, and it reads their roles rather than their name. See Restrict what an admin caller can do.

The one actor that is not a person or a workload is loopback, recorded on an issuerless listener bound to 127.0.0.1. It is honest about what it knows, which is only that the caller holds this host. That is the break-glass path, and the trail marks it as clearly as it marks anyone else.

A caller who signed in against an identity provider, or a workload that presented the token its platform issued, records the kind oidc and carries two names rather than one: actor.subject is the claim a policy can name, and actor.name is the readable one from the token. Neither alone is enough. A display name can be reassigned to somebody else, and an opaque subject is a trail nobody checks. actor.issuer says which provider vouched for them.

A refusal by that policy is its own event, admin.authz, carrying the @id of the policy that decided in the rule field. It is kept apart from admin.auth because “we do not know who you are” and “we know exactly who you are and you may not” are different facts. Filing them together hides the more urgent one.

A rule can also say which governance controls it stands for. A controls list on an access rule or a guardrail rule holds identifiers a governance platform assigned, eu-ai-act:art-50, nist-ai-rmf:govern-1.2, in whatever taxonomy it uses. The list rides on every request.auth and guardrail.decision record that rule decides, in the rule’s own order. It is duplicated out of the body as the controls attribute, so a collector can ask for every decision under a control without parsing bodies. The gateway never reads the values, it only carries them. The platform mapping a decision to its control then does not have to join the trail against the configuration that was in force at the time. A rule that declares none produces records without the key, and records from before the field existed hash exactly as they did.

Redact rules accumulate, so one request can be rewritten by several of them: a national identifier by one rule and a card number by another, in a single inspection that produces a single record. That record carries the union of their controls, deduplicated, in rule order, and a rules list saying which rule rewrote how many spans under which controls. rule names the deciding rule when exactly one acted and is absent when several did, because naming one of several reads as a complete attribution and is not. The failure this prevents is quiet: under an earlier version the last rule to act owned the record, so adding an unrelated card number to a request dropped the article the identifier rule cited out of the evidence for a redaction that still happened.

A rule is only as stable as the document it came from. Two deployments can disagree about what keyadmin-owns-keys permits while agreeing on its name. So the policy document itself is recorded when it takes effect, as admin.policy.load, carrying its path in target and the hex SHA-256 of its bytes in policy.hash. The policy is read once at startup, so that record is the only point at which what the gateway enforces changes. It lets a reader take any admin.authz record and say which text produced it. The same hash goes to the ordinary log at startup, next to the policy count. A document that suddenly declares three policies where it declared twelve is a truncated file, and the count says so before a denial does.

A tokenless loopback listener records the principal loopback, which is a real actor kind rather than an absent one. “Anyone with a local socket” is the honest name for that principal.