Configuration reference
Every key of the two configuration documents is here, in the order they are read: each block is followed by the blocks nested inside it, however deep. Configuration is YAML, and an unknown key is an error rather than a silent no-op.
A node runs from two documents. The node file, named by -config, holds
the keys that configure this process: listeners, cluster, admin, tls,
the directories, and deployment, the path of the second document. The
deployment document holds what the deployment is: providers, credentials,
budgets, profiles, guardrails and the rest, which every node must agree on
and which the cluster stores. A deployment key in the node file is refused at
load with the key named, and a node file without deployment is a node
whose deployment arrives through the API. The two are two files because they
have two lifecycles: the node file is read by one process and nobody else,
and the deployment document seeds a stored source that the node then
maintains as its one writer.
${VAR} references in string-valued fields are resolved from the environment
of the node reading them: the node file when it is parsed, the deployment
document on the way to a running configuration, so a document that did not
come from this machine still resolves against this machine. ${file:NAME}
reads the file NAME under this node’s secrets_dir instead. It is the
spelling for a secret that arrives as a mounted file rather than a variable,
and is for the deployment document only. Write $${ for a literal ${. A
reference stands for a string, so a numeric or boolean key cannot be one.
Template those where you render the file.
Starting a cluster with nothing stored seeds it from the deployment document,
as the source named default. After that the stored configuration is what is
served, the document is that source’s one writer, and version decides
whether a later edit lands at start: a document whose version is higher than
the stored one is applied, and one that does not say it is newer waits for
POST /admin/v1/reload, which applies it unless its version says it is
behind. Either way it lands everywhere, not only on the node you asked.
The cluster stores not one document but a set of named sources, each a
document of its own under its own revision. A deployment that has only
ever been configured from a file has exactly one, named default, and behaves
as though there were no such thing. More than one exists so that more than one
thing can maintain configuration (an operator reconciling custom resources, a
Terraform provider, a person with curl) without contending for a single
object. Each writes its own source and none can overwrite another’s. The
deployment is all of them composed, and they must not overlap. Two sources
naming the same provider is refused rather than merged, because a merge needs a
winner and a silent winner is how one change disappears into another. GET /admin/v1/config reports source, the sources it composed and
file_status, so a document that has parted company with what is running
says so, and says what to do about it.
This page is generated from the configuration structs. It is complete by construction and says nothing the code does not. For how to accomplish something see the how-to guides; for why a setting exists at all see the explanations.
Contents
Section titled “Contents”- Top level
clusterissuers[]adminextproctlsauditauthcredentials[]catalog_overlays[]providers[]budgets[]profiles[]mcp[]a2a[]a2a_cardsreconcileaccess_rules[]guardrailsguardrails.detectors[]guardrails.rules[]guardrails.streamguardrails.responseguardrails.overlapguardrails.async
models
Top level
Section titled “Top level”listen · string
listen is the data-plane address, default “:8484”.
metrics_listen · string
metrics_listen is where /metrics is served, default “:9464”. Its
own listener, plain HTTP, and never the data plane: the data
plane is the front door, and a scrape endpoint on it is a list
of every route, provider and model, readable by whoever can
reach the front door. The series are also pushed as OTLP when
the OTEL_* environment asks. See
docs/how-to/watch-the-gateway.md. “none” serves no metrics.
deployment · string
deployment names the deployment document: the providers,
credentials, budgets, profiles, guardrails and the rest, which
every node must agree on and which the cluster stores. A path,
relative to this file’s directory. The two halves are two files
because they have two lifecycles: this file configures one
process and is read by it alone, and the deployment document
seeds a stored source that the node then maintains, so an edit
to it is an edit to the deployment and nothing else. Absent for
a node whose deployment arrives entirely through the API.
cluster · object
cluster holds runtime-mutable state (minted keys, budget usage,
the reconciliation ledgers) in a raft log, and makes replication
itself the invalidation signal. Without it the gateway keeps no
such state: keys can only be declared in this file, and budgets
count in memory until restart.
A single-node deployment is bootstrap: true, a cluster of one.
There is no separate standalone store. One implementation that
scales from one node to several beats two that must agree.
issuers · list of object
admin configures the loopback control listener.
Issuers are the token issuers this node trusts, by name: who
signs, where the keys are, and how this node reaches them. Both
planes bind to entries here. admin.issuers says which may call
the admin API and as whom, and auth.issuers says which may call
the data plane. An issuer is written once, under its own name,
however many audiences it is trusted for.
It is node-half, and file-owned, on purpose. Two of its fields are paths on this machine, which the deployment document may not carry. The deeper reason is who may change it. The deployment document travels through the admin API, so anything in it is something a caller with config write can change. An issuer whose keys that caller controls is a caller who can mint any data-plane identity under any profile. Trust in a signer is operator intent, like the policy file. The document only binds to signers the operator already wrote down here.
admin · object
admin configures the loopback control listener.
Issuers are the token issuers this node trusts, by name: who
signs, where the keys are, and how this node reaches them. Both
planes bind to entries here. admin.issuers says which may call
the admin API and as whom, and auth.issuers says which may call
the data plane. An issuer is written once, under its own name,
however many audiences it is trusted for.
It is node-half, and file-owned, on purpose. Two of its fields are paths on this machine, which the deployment document may not carry. The deeper reason is who may change it. The deployment document travels through the admin API, so anything in it is something a caller with config write can change. An issuer whose keys that caller controls is a caller who can mint any data-plane identity under any profile. Trust in a signer is operator intent, like the policy file. The document only binds to signers the operator already wrote down here.
extproc · object
extproc, when set, additionally serves the inspector role: an
Envoy ext_proc gRPC listener that applies guardrails to traffic a
gateway pistra does not own is routing. The HTTP data plane
keeps serving.
tls · object
tls configures the front door’s certificate: ACME (public or
private directory) or static files.
catalog_overlay_paths · list of string
catalog_overlay_paths are paths (or glob patterns, e.g. an
“overlays/*.json” drop directory) of catalog overlay files, read
from this node’s filesystem. It is the bulk-import spelling, for
a directory a deployment tool syncs overlays into. It is
node-scoped for the same reason data_dir is: a path is a fact
about one machine, and nothing outside that machine can tell
whether it holds what the config claims.
Applied before catalog_overlays, so the overlays written down in this config win over the ones a directory happened to contain.
cache_dir · string
cache_dir is where this node materializes what the config declares
but does not carry: verified model artifacts and the manifests
resolved for them. Default:
It is the counterpart to cluster.data_dir and the line between them is durability: everything here can be fetched again, so losing it costs a download, while losing data_dir costs state nothing else holds. Deploy it as emptyDir if that suits. Models are large enough that a volume usually suits better.
secrets_dir · string
secrets_dir is the directory a ${secret:NAME} reference looks in
first (and ${file:NAME} reads from): the value is the content of
the file NAME inside it, with trailing newlines removed. Unset,
the scheme is refused.
It is node-scoped for the same reason cache_dir is: a path is a fact about one machine. It exists for the machine whose secrets arrive as files rather than variables. A Kubernetes Secret mounted as a volume is a directory of one file per key, rewritten in place when the Secret changes. An environment variable is fixed for the life of the process. The gateway re-reads the files its configuration references every ten seconds and rebuilds when one has changed, so a rotated secret takes effect with no restart and no reload. NAME is one file name and never a path, so a document (which may have arrived through the API) can reach this directory and nothing else on the node.
audit · object
audit configures where this node keeps the audit trail’s signing
key and chain head. Every record in the trail is hash-linked to
the one before it and signed by a per-node key, and the key has
to live somewhere a restart finds it. By default that is
cluster.data_dir, and this block is for the node that has none.
version · integer
version is the edition of this document, a positive integer that
its author raises with every edit. It is a statement about the
document, not about the deployment: [Compose] drops it, no node
serves it, and it can be addressed by no per-object path.
It exists for one reason: it is how a config file says whether it
is newer than the stored source it seeded. A file with no version
seeds the default source once and is otherwise applied only by
a reload. A file with one is applied, at start and on a reload,
exactly when its version is higher than the stored document’s,
and refused with a reason otherwise. The same version over
different content is an edit that forgot the bump. A lower one is
a file behind what somebody else stored. Rendering the file from
something that already counts (a Helm release revision, a CI
build number) makes the bump free.
auth · object
auth is how the data plane authenticates callers. Absent, every
request is anonymous and whatever credential the client sent is
forwarded. That is the shape for a gateway in front of an API
that does its own authentication, and for the inspector role.
Present, every request must carry a credential one of its sources
recognizes, and the client’s credential never reaches an
upstream.
max_body_bytes · integer
max_body_bytes bounds request bodies. Zero means the proxy default.
credentials · list of object
credentials are named provider credentials, separated from
providers so they can be bound and rotated independently: the
filterapi BackendSecurityPolicy split.
catalog_overlays · list of object
catalog_overlays are catalog overlay documents applied in order
onto the compiled provider catalog before presets resolve:
add/replace/patch/remove providers and their channels without a
binary upgrade. Overlays carry catalog facts only: endpoints,
model gates, caps claims within the compiled vocabulary. Policy
(require_caps, fidelity mode) stays here in config. Rebuilt on
every reload with the same all-or-nothing, keep-last-good
semantics as the rest of the file.
They are written inline rather than named by path because an overlay is small, declarative and cluster-wide: an operator who adds a provider means every node to have it, and a value says that where a path only says where one machine should look.
providers · list of object
providers are the upstream endpoints. A request goes to the first
provider whose models patterns match the model it asks for. A
model matching none goes to the default provider.
budgets · list of object
budgets define windowed cost limits. Cost speaks Envoy AI
Gateway’s vocabulary: a filterapi.LLMRequestCostType, or CEL over
model/backend and the token counts. Virtual keys opt in by naming
a budget.
profiles · list of object
profiles are the named policies a virtual key points at. A key
carries who it is (name, metadata, expiry, revocation) and names
a profile for what it may do. Everything else about its traffic
is here.
The indirection is the point. It answers “what can this key do?” by reading one object instead of simulating a rule list. It makes a team’s model allowlist one write instead of N, and it gives a Terraform resource or a CRD something to declare that is not a copy on every key.
mcp · list of object
mcp lists upstream MCP (Model Context Protocol) servers the
gateway fronts at /mcp/
a2a · list of object
a2a lists peer agents the gateway fronts at /a2a/
a2a_cards · object
a2a_cards is how the cards the gateway projects for its peer agents
are signed. Absent, cards are served unsigned.
reconcile · object
reconcile audits the gateway’s wire-metered token usage against
the numbers providers report through their admin usage APIs, and
alerts on drift. It is the check that turns “we hope the scanner
is complete” into a verified invariant. Requires cluster, which
is where its ledgers live. Every node meters, and the raft leader
polls.
default_provider · string
default_provider serves requests whose model matches no provider’s
patterns (and modelless requests like GET /v1/models). With exactly
one provider it is implied.
on_fidelity_loss · string
on_fidelity_loss is the default policy when every channel that
could serve a request loses a capability the request uses:
“reject” (the default, which refuses with a structured error
naming the capability) or “allow” (forward on the best candidate
and count the downgrade). Providers may override per-entry.
public_url · string
public_url is the absolute URL clients reach this deployment’s
data plane at: the front door, e.g. https://ai.example.com. It
is deployment-wide because it names the deployment, not a node.
Nothing routes on it. It exists for the surfaces that hand a client an address to connect to and must hand out the governed one: the MCP registry projection writes it into every entry’s remote URL. Those surfaces refuse to answer until it is set, because the alternative, guessing from a listener or a Host header, hands out whichever address the asker happened to use.
access_rules · list of object
access_rules are the deployment-wide access rules: CEL conditions
evaluated per request, in order, first match wins within this
list, no match means allow.
One list for every hop the gateway admits a request at. A rule’s
at names the hops it runs at: llm (a model request: completion,
embedding, any route a provider serves), mcp (a call at a fronted
MCP server), a2a (a message to a fronted peer agent). A rule
with no at runs at all of them. Each hop has a vocabulary of
its own, listed on AccessRuleConfig. A rule at several hops may
read only what they all declare, and a rule at every hop reads
only who is calling. The list is one because the decision is
one: a key that is denied is denied wherever it turns up, and a
rule that says so should be written once.
They are the mandatory layer. A profile’s own access_rules run after them and can only narrow: a deny here is final, and an allow here ends this list but does not skip the profile’s. That is what makes an org-wide rule an org-wide rule. Otherwise a permissive profile would be a way to opt out of one.
guardrails · object
guardrails configures the inline guardrail ensemble: which
detectors inspect request content, and what CEL policy does with
the annotations they produce. Detection runs in-process, with no
service hop and no sidecar, and its cost lands in
pistra_overhead_seconds like everything else the gateway does.
models · object
models declares the model artifacts a guardrails detector may name
by reference: where each file lives, what it must hash to, and
where fetched copies are cached. Declaring a model here rather
than pointing a detector at a path gets the bytes verified.
cluster
Section titled “cluster”The static multi-node topology. Every node lists the identical peers, and raft elects among them. Three or more nodes tolerate a failure. Two replicate but cannot lose either node.
Required whenever a config file is used: keys, budget spend and
certificates live in the raft log, so a gateway without one has
nowhere to put them. A single node is bootstrap: true, which is
spelled out rather than inferred from the block’s absence. See
Bootstrap for what an implied bootstrap costs.
node_id · string
node_id names this node. It must match one peers entry when this
node bootstraps. A joining node names itself here.
data_dir · string
data_dir holds the raft log and snapshots.
secret · string
secret authenticates follower-to-leader write forwarding and
membership changes, normally an env reference like
${PISTRA_CLUSTER_SECRET}. Every node derives from it the
certificate authority under which the raft transport and the
forward listener speak mutual TLS, so no certificate is
configured for those links and a node without the secret is
refused at the handshake. It is also, by default, the key that
wraps the data key sealed values depend on. See kek_file.
kek_file · string
kek_file names a file of at least 32 bytes of key material this
node holds as a second key-encryption key, beside the one derived
from secret. It is how the wrapping of the data key moves off the
cluster secret: add a wrapping under the file’s key, then drop
the cluster secret’s, and the join credential and the storage
key are two things that rotate apart. Node-scoped. The file is
delivered like any other secret and never enters the log.
advertise · string
advertise overrides the raft address announced to peers. Empty
means this node’s raft_addr. It may be a DNS name, and on
Kubernetes it should be: a pod’s IP changes on every restart while
its StatefulSet name does not, and the membership records what
peers dial. Bind raft_addr to 0.0.0.0 and advertise the name.
forward_advertise · string
forward_advertise overrides the forward address announced to
peers. Empty means this node’s forward_addr. Same rule as
advertise.
peers · list of object
peers is the bootstrap member list, identical on every node that
carries it and including this one. It seeds the first
configuration and nothing after: once a cluster exists its
membership lives in the raft log, and editing this list cannot
change it. A single-entry list is a valid cluster of one, and the
way to start a cluster that grows later.
Mutually exclusive with join and bootstrap.
bootstrap · bool
bootstrap starts a cluster of one from this node: the standalone
deployment, and the way to start a cluster meant to grow. It is
shorthand for a peers list holding only this node, and it is an
explicit flag because the alternative mistake is expensive: a node
that meant to join and instead bootstraps becomes a second cluster
sharing a secret with the first.
Set together with join, it is a fallback rather than an order:
the node tries the join list first and starts a cluster only
after every address has stayed silent for bootstrap_wait. That is
the setting for a node that is usually first but must not assume
it, such as ordinal 0 of a StatefulSet, which may have lost its
volume while the other ordinals kept theirs.
Mutually exclusive with peers.
join · list of string
join lists the forward addresses of existing members. Set it
instead of peers on a node entering a cluster that is already
running. It is tried in order, and again after a pause, until one
member admits the node or sixty seconds pass. It is consulted
only when this node has no state of its own, so a restart replays
the log rather than re-joining.
bootstrap_wait · string
bootstrap_wait is how long every join address must stay silent
before a node with both bootstrap and join starts a cluster.
Silent means a name that does not resolve, a port nobody listens
on, or a peer that holds no cluster itself. A peer that answers in
any other way (no leader yet, a refusal, a timeout) resets the
clock: something is there, and the node must not start a second
cluster beside it. A duration such as “15s”. The default is 5s.
Ignored without join.
rejoin_after · string
rejoin_after is how long this node may go without knowing a leader
before it asks the join list to admit it again. A member that has
merely lost quorum gets no answer and nothing changes. One the
membership no longer holds (removed while it was down, or reaped
under dead_after) is re-admitted with the log it has, catches up,
and votes again, instead of campaigning forever against peers that
ignore it. A member removed while it was running was removed on
purpose, is told so by the leader, and does not ask back in until
its process restarts. A duration. The default is 30s. Ignored
without join.
dead_after · string
dead_after turns on reaping: the leader probes every other member
and removes one that has answered nothing for this long, so a
member that is gone for good stops counting toward quorum. Off
when empty. Set it longer than the longest restart a member may
take. A slow member that is reaped comes back as a stranger and
rejoins, but the cluster ran without its vote in between. A
duration such as “10m”.
raft_addr · string
raft_addr is this node’s raft transport address (host:port).
Required when joining. A bootstrap node may instead leave it empty
and let its peers entry supply it.
forward_addr · string
forward_addr is where this node accepts forwarded writes when it
leads. Required when joining. A bootstrap node may instead leave
it empty and let its peers entry supply it.
cluster.peers[]
Section titled “cluster.peers[]”One cluster member.
id · string
id is the peer’s node_id.
raft_addr · string
raft_addr is the raft transport address (host:port).
forward_addr · string
forward_addr is where the node accepts forwarded writes when it
leads.
issuers[]
Section titled “issuers[]”One trusted issuer of bearer tokens, as this node knows it. Verification only: pistra is a resource server, and getting a token is the caller’s business. For a workload that means a file its platform already put on disk, and no long-lived secret to hand out at all.
What a token must say, and what its claims mean, is not here. That is decided by whichever plane binds to this issuer (admin.issuers for the admin API, auth.issuers for the data plane) because the same directory signs for both and they need different audiences. This entry is the part they share: who signs, and how to fetch the keys.
name · string
name identifies the issuer in the audit trail, in the principal a
policy sees, and in the bindings that reference it. Required,
because a subject is only unique within an issuer: two issuers
both calling something “admin” must not collapse into one
principal.
url · string
url is the iss claim, matched exactly, and where discovery
looks when JWKSURL is empty.
jwks_url · string
jwks_url is where the signing keys live. Empty means discover it,
on the first token rather than at startup. An identity provider
that is unreachable when the config loads must not stop the
gateway serving.
ca_file · string
ca_file is a PEM bundle for an issuer whose TLS certificate the
system does not trust. A Kubernetes API server is the usual one.
token_file · string
token_file holds a bearer token presented when fetching this
issuer’s discovery document and keys, for an issuer that does not
publish them to anyone. A Kubernetes API server binds that right
to system:serviceaccounts, so reading the keys takes a service
account token of the gateway’s own. That cannot be the token
being verified, since the API server refuses one minted for
another audience. Re-read on each fetch, so a projected token
that rotates keeps working.
subject_claim · string
subject_claim identifies the caller. Empty means sub. What it
names is the principal every access rule and audit record is
written over, so a claim callers usually set for themselves, such
as email, is refused unless trust_subject_claim says they cannot
here.
trust_subject_claim · bool
trust_subject_claim allows subject_claim to be one a caller can
usually set for themselves. It asserts what roles.trust_claim
asserts, about the subject instead of the roles: callers at this
issuer cannot set that claim. True of a directory that owns its
mailboxes and forbids profile edits, false of a provider that
federates a social login upstream, and only the operator can tell
which theirs is. A token carrying email_verified: false is
refused whatever this says.
name_claim · string
name_claim carries the readable name for the audit trail. Empty
tries preferred_username, then email, then the subject.
The control plane. It has two doors, and they are
deliberately separate. Listen is how the network reaches this API,
gated by an issuer. local_socket is how somebody holding the host
reaches it, gated by the filesystem. Either may be set alone, and in
production both usually are, because the reason to have the second
one is that the first one depends on an identity provider that can
itself be the thing that broke.
listen · string
listen is the control listener address. Empty serves no admin API
over the network at all, which is the default: minting keys is not
a capability a gateway should offer before it is asked for.
With no issuer configured this must bind loopback, where the
principal is whoever holds the host. Prefer local_socket for that:
loopback TCP is reachable by every process and every user on the
box, and by anything that lands a request forgery, while a socket
is reachable by whoever the file mode says.
local_socket · string
local_socket is a Unix domain socket path serving the same API to
whoever can open it. No issuer, no bearer token, no secret
anywhere: the credential is filesystem access, which is a stronger
claim than any shared secret because a secret travels and host
access does not.
It is the way in before there is anything to authenticate with, and the way back in when the identity provider is what broke. Those are the same door on purpose. A break-glass path that is only exercised during a break-glass is one nobody knows works.
The caller is identified by its peer credentials, so it is a named
principal (unix:501) in Role::“local” rather than an anonymous
one, and policy_file decides what it may do like any other caller.
Created 0600 and owned by this process. A stale socket left by an
unclean exit is replaced, a live one is refused.
policy_file · string
policy_file is a Cedar policy document deciding what each admin
caller may do. It is required once OIDC is set:
an admin API where every credential can mint keys and drop raft
members is not a default worth offering. The default within it is
deny, as Cedar’s own is. A request no policy permits is refused.
It is optional only while nothing can authenticate over the
network: a purely local admin plane, before there is a policy to
write or an issuer to write it about, lets a local caller through.
Once an issuer is configured this is required, and it then decides
for local callers too. That is the point of local_socket being a
second door rather than a second mode. Grant it explicitly:
permit(principal in Role::"local", action, resource);It is file-owned rather than API-mutable for the same reason provider topology is: it is operator intent, it belongs in the review that changes it, and a policy the API can rewrite has to answer who is allowed to rewrite it.
Read once at startup, like the credentials it decides over: a reload that swapped the policy but not the issuers carrying the roles would be half a reload. A policy that fails to compile stops the gateway starting rather than serving without it.
issuers · list of object
issuers are the token issuers whose callers may reach this API
over the network, and the only way to. Each names an entry under
the top-level issuers list and adds what this plane decides for
itself: the audience a token must carry, and how its claims become
roles. Each is a separate entry because the two populations
calling this API do not share one. People arrive from a directory
with their group membership in a claim. Workloads arrive from
their own platform (a Kubernetes API server, GitHub Actions, a
SPIFFE authority) with no groups at all and a structured subject
instead.
admin.issuers[]
Section titled “admin.issuers[]”Binds one issuer to the admin API: which tokens of its count here, and what they say about what a caller may do.
issuer · string
issuer names an entry under the top-level issuers list. Required.
audience · string
audience is what a token must name in aud. Required: accepting
a token minted for a different application is the confused deputy
in its original form.
roles · object
roles is where this issuer’s roles come from.
login · object
login is how pistra login gets a token from this issuer.
Optional, and pistra is a resource server either way: it
verifies tokens and never mints one. This block only tells the
CLI where to go and as whom, so it holds a client id and no
secret. A program on somebody’s laptop cannot keep one, which is
the reason PKCE exists.
admin.issuers[].roles
Section titled “admin.issuers[].roles”Says how one issuer’s claims become roles.
claim · string
claim is read for the caller’s roles: “groups” for a directory,
“sub” for a platform that issues workload identities and has no
groups to give. Empty means “groups”. A string and a list of
strings are both accepted, because providers disagree about which
they send for a single value.
map · map of list of string
map translates what the claim says into the roles the policy is
written against, so a directory’s naming never reaches the policy
file and renaming a group is not a policy change. When it is set,
a value it does not mention grants nothing.
trust_claim · bool
trust_claim allows claim to be one a caller can usually set for
themselves, such as email, preferred_username or upn.
Without it the gateway refuses to start, because reading roles
from a self-asserted claim lets the caller choose their own
roles, and with no map the value becomes the role name directly.
Setting it asserts something specific: that callers at this issuer cannot set this claim. That is a true statement about plenty of corporate directories, which own their mailboxes and forbid profile edits, and a false one about any provider that federates a social login upstream. Nobody outside your directory can tell which yours is, so pistra asks rather than guesses.
It does not override the issuer itself. A token carrying
email_verified: false is refused whatever this says: the
assertion here is that callers cannot set the claim, and such a
token is the issuer reporting that one did.
admin.issuers[].login
Section titled “admin.issuers[].login”What a command-line client needs to obtain a token from this issuer on a person’s behalf.
The gateway serves it unauthenticated from /admin/v1/login, because a client that has to already be signed in to find out how to sign in is no use. Nothing in it is a secret: a public client’s id is public by design, and the issuer URL is on the front of every token it mints. It appears at all only where an operator wrote this block.
client_id · string
client_id is the OAuth client registered for the CLI. Register it
as a public client with no secret, PKCE required, and the
loopback redirect RFC 8252 describes: any port on 127.0.0.1,
which the authorization server must not pin.
The token the CLI ends up presenting must name Audience in aud.
For an ID token (the default, and what works everywhere) the
audience is the client id, so the two are usually the same
string. Setting them differently without a reason is the mistake
that produces a token the gateway then refuses.
scopes · list of string
scopes requested. Empty means openid alone. Add profile or email
where the audit trail should carry a name, and whatever scope
carries group membership at your provider.
extra · map of string
extra are additional authorization-request parameters, verbatim.
The escape hatch for providers that need one to mint a token for
the right audience: Auth0 wants audience, an RFC 8707 server
wants resource. Empty for a provider that needs neither.
token · string
token is which of the two the CLI presents: “id” (default) or
“access”. ID tokens are what an OIDC provider always issues and
their audience is the client id. Choose access where the provider
mints JWT access tokens for a named API and Audience is that
API’s identifier rather than this client’s.
extproc
Section titled “extproc”The inspector role’s ext_proc listener.
inspect_listen · string
inspect_listen is the gRPC address the inspector serves on, e.g.
“:19002”: guardrails over ordinary HTTP traffic, with no routing
decision attached. A filter pointed at it has no way to reach
one. The listener is built without that code path rather than
configured not to take it.
Requires a guardrails section. An inspector with no policy would buffer every request body to do nothing with it.
request_body_mode · string
request_body_mode names the request_body_mode of the ext_proc
filter pointing here: “streamed” (the default when empty) or
“duplex” for FULL_DUPLEX_STREAMED. It has to be told, for the
reason inspect_responses gives below: the wire looks the same in
both modes and the answers the server must give do not.
In streamed mode the server asks Envoy for the body itself, at the headers phase, and answers every chunk exactly once: clear_body while it is holding, the whole rebuilt body at the end. In duplex mode the filter ignores that request and sends every body. The server answers with streamed_response chunks, as many or as few as it likes, and a body nothing reads is echoed chunk by chunk. Duplex is what a gateway with no STREAMED mode (agentgateway) needs, the only mode in which a body that ends on trailers can be inspected (in streamed mode the held chunks are already cleared when the trailers arrive, so such a request is refused with 501 and such a response is counted as uninspected), and it is the mode with no message_timeout on a held chunk.
inspect_responses · string
inspect_responses turns on response inspection for the inspector
listener: “” (default) leaves responses alone, “streamed” or
“duplex” names the ext_proc body mode the responses arrive in,
STREAMED or FULL_DUPLEX_STREAMED.
It is a key rather than something inferred from apply_to: [output] for two reasons. The guardrails config is shared by the
inspector and the data plane, so inferring it would change what a
third-party gateway’s traffic costs the moment an inspect listener
is added. Buffering response bodies on traffic pistra does not
route, from APIs whose shape it does not own, is the most
expensive thing in this design. And it cannot work at all unless
the operator also edits an ext_proc filter pistra does not own, so
this is where pistra can say so.
The value must match the response_body_mode of that filter,
because ext_proc gives the server no way to learn it and the
modes demand incompatible wire behaviour. STREAMED requires
exactly one response per request and carries mutations as
body/clear_body. FULL_DUPLEX_STREAMED forbids those and requires
streamed_response, any number of them. Guessing is wrong half the
time in a way the data plane rejects, or worse, in a way it does
not: a duplex filter answered in the streamed shape hangs the
response rather than failing it. So the operator states which
filter they wrote.
“duplex” is the mode worth preferring where the gateway offers it.
It has no message_timeout at all, so a remote: detector can take
its time on a held chunk, and a windowed stream is released as the
horizon allows rather than once per chunk received. “streamed” is
the mode every Envoy has had for longer, and its one cost has to be
planned for. ext_proc’s message_timeout (200ms by default) applies
to every held chunk, and an ensemble with a remote: detector will
exceed it and fail the stream. Raise message_timeout to seconds, as
the tutorial’s filter does.
Requires inspect_listen and a detector with apply_to: [output].
The converse is not required: output detectors without this key
serve the data plane exactly as before.
Selects the certificate source. Exactly one mode: domains
(ACME via certmagic) or cert_file+key_file (static, operator-rotated).
domains · list of string
domains to obtain and renew certificates for via ACME.
email · string
email is the ACME account contact.
ca · string
ca is the ACME directory URL. Empty means Let’s Encrypt
production. Point it at an internal directory (step-ca, Vault)
for private PKI.
trusted_roots · string
trusted_roots is a PEM bundle for verifying a private ACME CA,
inline rather than a path. Which roots to trust is a property of
the deployment, not of one node’s filesystem, so it has to be a
value the config can carry. ${PISTRA_ACME_ROOTS} works for a
bundle an operator would rather not paste.
dns · object
dns solves the ACME DNS-01 challenge through a DNS provider
instead of HTTP-01 on the data listener. DNS-01 needs no inbound
reachability from the CA and is the only challenge type for
wildcard domains.
eab · object
eab carries External Account Binding credentials (RFC 8555
7.3.4) for a CA that will not create an ACME account without
them: ZeroSSL, Google Trust Services, Sectigo, DigiCert, SSL.com,
and step-ca or Vault when their operator requires it. Without it
ca can only name a directory that accepts anonymous
registration.
cert_file · string
cert_file and key_file serve a static certificate instead of ACME.
key_file · string
cert_file and key_file serve a static certificate instead of ACME.
tls.dns
Section titled “tls.dns”Selects and authenticates a DNS provider for the ACME DNS-01 challenge. Secrets are normally env references like ${CLOUDFLARE_API_TOKEN}.
provider · string
provider is “cloudflare” or “rfc2136”.
api_token · string
Cloudflare: an API token with Zone.DNS:Write. zone_token is an
optional separate Zone:Read token when api_token is zone-scoped.
zone_token · string
Cloudflare: an API token with Zone.DNS:Write. zone_token is an
optional separate Zone:Read token when api_token is zone-scoped.
server · string
RFC 2136 (TSIG-signed dynamic updates: BIND, Knot, PowerDNS).
Server is the authoritative server as host:port. key_name, key_alg
and key identify the TSIG key (e.g. hmac-sha256, base64 secret).
key_name · string
RFC 2136 (TSIG-signed dynamic updates: BIND, Knot, PowerDNS).
Server is the authoritative server as host:port. key_name, key_alg
and key identify the TSIG key (e.g. hmac-sha256, base64 secret).
key_alg · string
RFC 2136 (TSIG-signed dynamic updates: BIND, Knot, PowerDNS).
Server is the authoritative server as host:port. key_name, key_alg
and key identify the TSIG key (e.g. hmac-sha256, base64 secret).
key · string
RFC 2136 (TSIG-signed dynamic updates: BIND, Knot, PowerDNS).
Server is the authoritative server as host:port. key_name, key_alg
and key identify the TSIG key (e.g. hmac-sha256, base64 secret).
tls.eab
Section titled “tls.eab”Binds the ACME account this gateway creates to an account
the CA already holds for the customer. Both values are issued out of
band, from a CA portal or an enterprise PKI ticket, and are useless
apart: key_id names the account, mac_key proves possession of it.
mac_key is a credential and belongs in the environment, not the file:
${ZEROSSL_EAB_MAC_KEY}. The chart and the Terraform module both
refuse a literal here for the same reason they refuse a literal
api_key. A rendered config is a ConfigMap, readable by anything
that can get one.
The binding is used once, when the ACME account is created. ACME
has no operation that rebinds an existing account, so neither does
pistra. Changing these values alone does nothing: the stored
account is reused as it stands, and a CA that has revoked the old
binding starts refusing orders with nothing in the config to explain
it. New credentials need a new account, and the account is keyed by
ca and email, so changing email registers one. That takes a
restart. A reload does not rebuild the certificate manager.
key_id · string
key_id is the key identifier the CA issued. “The key identifier
MUST be an ASCII string.” (RFC 8555 7.3.4)
mac_key · string
mac_key is the CA’s MAC key, base64. RFC 8555 asks for
base64url, but CA portals hand out the standard alphabet and
padding often enough that both are accepted here and normalized
on the way to the ACME client.
The node half of the audit trail. The trail itself follows the OTEL_* environment like every other signal, and a node decides only where its key lives.
dir · string
dir holds the node’s audit signing key (audit.key, PKCS#8 PEM) and
the head of its chain (audit-head). Default: cluster.data_dir.
An inspector-only node has no data_dir, and without this set it
signs with a key that dies with the process, recorded as
“ephemeral” in its start record. That is honest, but it leaves a
restart unlinkable to what came before.
Names the credentials the data plane accepts. A request must present one of them, and policy, budgets and the audit trail then see the caller as whichever one it presented.
Two sources, and they are two populations. Virtual keys are minted
by the admin API and carried by whatever has no identity provider
behind it: a service that was handed a key, a developer on a laptop.
Issuer tokens are minted by an identity provider the node trusts
and carried by whatever already has one. That is an agent running
as a workload with its platform’s token, or a person’s application
with the token their directory issued. The second population is how a
request comes to have an agent and a user rather than only a
key.
virtual_keys · bool
virtual_keys accepts keys minted by the admin API. Off, a key is
refused like any other unrecognized credential.
issuers · list of object
issuers accepts JWTs from the named issuers, each bound to the
audience a token must carry and to what its subject is.
auth.issuers[]
Section titled “auth.issuers[]”Binds one issuer to the data plane.
issuer · string
issuer names an entry under the top-level issuers list. Required.
audience · string
audience is what a token must name in aud. Required, and not
the admin API’s: a token minted to administer the gateway must
not also be a token to send traffic through it, or the reverse.
A token naming the resource it is presented to is accepted as
well, once public_url is set: the front door itself, or one MCP
server at <public_url>/mcp/
subject · string
subject says what a token’s subject is: “agent” where the issuer
mints identities for workloads (a Kubernetes service account, a
SPIFFE ID, a CI job) and “user” where it mints them for people.
Required, because policy reads the two differently and a token
does not say which it is.
Under “user” the token also names the application acting for the
person, in client_id or azp, and that is what agent reads.
A token carrying neither has a user and no agent. Under “agent”
there is no user: the workload acts for itself.
profile · string
profile is the policy governing traffic from this issuer’s
callers (allowed models, budget, rules, guardrail selection), the
way a virtual key names one. Empty means no profile, which is
unrestricted but for the deployment-wide rules. Where profiles
selects one from the token, this is the fallback for a token that
selects nothing.
profiles · object
profiles chooses the profile from a claim in the token, so a
person’s groups pick their policy the way an admin caller’s groups
pick their roles. Optional. Without it every caller from this
issuer is under profile.
auth.issuers[].profiles
Section titled “auth.issuers[].profiles”Says how a token’s claims choose a profile.
claim · string
claim is read for the values select matches: “groups” for a
directory, “sub” for a platform whose subjects are the identity.
A string and a list of strings are both accepted. Required.
It must name something the issuer assigns rather than something the caller can set. That is the same rule as roles.claim on the admin plane, and for the same reason: a caller who chooses the claim chooses their own budget and models.
trust_claim · bool
trust_claim allows claim to be one a caller can usually set for
themselves. See roles.trust_claim for what it asserts.
select · list of object
select is the ordered list of matches: the first entry whose
value the claim carries names the profile. Ordered rather than a
map because a caller in two groups can be under only one policy,
and the configuration, not the token’s group order, decides
which. A token matching nothing falls back to Profile.
auth.issuers[].profiles.select[]
Section titled “auth.issuers[].profiles.select[]”One match: a claim value and the profile it selects.
value · string
value is the claim value that selects, compared exactly: a group
name as the directory spells it, or a subject.
profile · string
profile is the profile a token carrying value is put under. It
must exist. A select naming a profile the configuration lacks is
refused when the configuration loads.
credentials[]
Section titled “credentials[]”One named provider credential.
name · string
name is what a provider or MCP server refers to in its credential
field.
api_key · string
api_key is the credential value, normally an env reference like
${OPENAI_API_KEY}. More credential kinds (Azure, GCP) arrive
with the declarative driver, mirroring filterapi.BackendAuth.
aws · object
aws signs upstream requests with SigV4 instead of sending a
static value. Mutually exclusive with api_key and oauth. Only LLM
providers accept it (the bedrock preset without a Bedrock API
key). Standalone mode only: the Envoy-mode extproc has no AWS
auth handler.
oauth · object
oauth obtains bearer tokens via the oauth 2.0 client_credentials
grant instead of a static value. That is the machine-to-machine
shape MCP servers behind an authorization server expect. Tokens
are fetched lazily, cached, and refreshed before expiry. Mutually
exclusive with api_key. Currently only MCP servers accept oauth
credentials (LLM providers speak API keys).
token_exchange · object
token_exchange makes the gateway act as the caller rather than as
itself. The token the caller presented is exchanged at its issuer
for one minted for the upstream server, so the server sees the
person or workload behind the request, with the gateway named as
the client acting for them. The exchange is the identity
provider’s own (RFC 8693 token exchange, or Microsoft Entra’s
on-behalf-of grant), and the gateway mints nothing.
A server under such a credential can only be called with a token from that issuer: a virtual key has no identity at an identity provider, and is refused there. Mutually exclusive with the other kinds. Only MCP servers accept it.
credentials[].aws
Section titled “credentials[].aws”Selects AWS credentials for SigV4 request signing. With explicit keys set, those are used as-is. With all key fields empty, the SDK default chain resolves them (env vars, shared profile, IMDS instance role, IRSA/pod identity, SSO) and refreshes short-lived credentials automatically. That is the enterprise-preferred mode, since it needs no long-lived secret in config at all.
access_key_id · string
access_key_id / secret_access_key are static keys, normally env
references like ${AWS_ACCESS_KEY_ID}. Both or neither.
secret_access_key · string
access_key_id / secret_access_key are static keys, normally env
references like ${AWS_ACCESS_KEY_ID}. Both or neither.
session_token · string
session_token accompanies temporary STS credentials.
service · string
service is the SigV4 signing service name. Default “bedrock”.
credentials[].oauth
Section titled “credentials[].oauth”One client_credentials grant.
token_url · string
token_url is the authorization server’s token endpoint.
client_id · string
client_id identifies the gateway to the authorization server.
client_secret · string
client_secret is normally an env reference like ${MCP_CLIENT_SECRET}.
scopes · list of string
scopes are requested with the grant. Empty asks for whatever the
authorization server issues by default.
credentials[].token_exchange
Section titled “credentials[].token_exchange”Exchanges the caller’s token for one the upstream accepts, at the issuer that minted it.
issuer · string
issuer names an entry in the node’s issuers list. The exchange is
sent to that issuer’s token endpoint, found by discovery, and
only tokens it minted are exchanged. Naming a trusted issuer
rather than a URL keeps a config writer from pointing every
caller’s token at a server of their choosing.
client_id · string
client_id identifies the gateway to the issuer. It is the client
the issuer will name as acting for the caller.
client_secret · string
client_secret is normally a reference like ${secret:JIRA_EXCHANGE}.
audience · string
audience is the logical name of the upstream the issued token is
for, as the issuer knows it. Sent as the RFC 8693 audience
parameter. The on_behalf_of grant ignores it and names the
target through scopes.
resource · string
resource is the upstream’s URL, sent as the RFC 8693 resource
parameter for issuers that mint by resource rather than by
audience. One of audience and resource is required for the
token_exchange grant.
scopes · list of string
scopes are requested with the exchange. Entra’s on-behalf-of
grant requires one, of the form api://
grant · string
grant selects the issuer’s exchange protocol: “token_exchange”
(RFC 8693; Okta, Keycloak, Auth0 and most others; the default)
or “on_behalf_of” (Microsoft Entra’s jwt-bearer grant with
requested_token_use=on_behalf_of).
subject_token_type · string
subject_token_type is what the gateway says it is exchanging, for
the token_exchange grant. Default
urn:ietf:params:oauth:token-type:access_token, which is what an
issuer minted for the gateway’s audience. An issuer that wants
the jwt or id_token type accepts it here.
catalog_overlays[]
Section titled “catalog_overlays[]”One inline catalog overlay document. It is the same schema an overlay file holds, plus the name that file’s base name would have supplied.
name · string
name labels this overlay in catalog provenance and in the errors
its operations raise, the way an overlay file’s base name does.
providers · list of object
providers are the per-provider operations, in order: the same
documents an overlay file’s “providers” array holds. See the
add-a-provider-with-an-overlay how-to for the op vocabulary.
catalog_overlays[].providers[]
Section titled “catalog_overlays[].providers[]”One per-provider operation.
Ops: “add” introduces a provider absent from the view; “replace” swaps an existing provider for a full new definition; “patch” changes base_url, key_env, docs and channels of an existing provider (any other field requires replace: partial edits to mutation lists or the schema would be deep merges, and deep merges manufacture claims nobody made); “remove” deletes the provider, so config referencing it fails loudly at the next build.
name · string
name identifies the provider and must equal the file’s base name,
so a catalog entry is found by the name that refers to it.
schema · object
schema is the request dialect the provider speaks natively.
base_url · string
base_url is the root every request to this provider is sent to.
key_env · string
key_env names the environment variable holding the provider’s API
key. It documents the provider for an operator reading the
catalog; the gateway takes its credentials from configuration.
docs · string
docs is the provider’s own API documentation, carried so a claim
made here can be checked against its source.
remove_request_fields · list of string
remove_request_fields lists top-level JSON fields the provider
rejects; they are stripped before forwarding.
set_request_fields · map of raw JSON
set_request_fields forces top-level JSON fields to fixed values,
e.g. a required service tier. Values are raw JSON.
set_request_headers · map of string
set_request_headers adds or overwrites request headers.
remove_request_headers · list of string
remove_request_headers strips request headers before forwarding.
strip_path_prefix · string
strip_path_prefix is removed from the incoming request path before
it is appended to base_url (providers that serve the OpenAI
surface without the /v1 prefix).
op · string
op is which of those four operations this document is.
channels · list of object
channels shadows the embedded plain channel list: overlay
channels always parse with an op. For add/replace the op must be
empty (the list is the definition); for patch each entry is an
add/replace/remove keyed by channel id.
catalog_overlays[].providers[].schema
Section titled “catalog_overlays[].providers[].schema”The on-disk JSON schema reference (filterapi schema name + optional version).
name · string
name is the dialect, and must be one the data plane can serve.
version · string
version pins the dialect’s revision where the provider needs one.
Empty takes the schema’s default.
catalog_overlays[].providers[].channels[]
Section titled “catalog_overlays[].providers[].channels[]”One channel definition, with an op when the enclosing provider op is patch. Replacement is whole-channel: caps and model gates are explicit-set claims, never merged.
id · string
id names the channel within its provider, and is what a route
selects it by.
dialect · string
dialect is the request shape this channel accepts.
tier · string
tier says how the channel is served: the provider’s own surface,
or a compatibility surface it offers for another dialect.
base_url · string
base_url and path_prefix override the provider’s root for this
channel. A compatibility surface usually needs them, because it
is served from somewhere else on the same host.
path_prefix · string
base_url and path_prefix override the provider’s root for this
channel. A compatibility surface usually needs them, because it
is served from somewhere else on the same host.
models · list of string
models restricts the channel to the model ids it serves. Empty
means every model the provider offers.
caps · list of string
caps are the capabilities the channel is claimed to support.
docs · string
docs is the documentation for this channel where it differs from
the provider’s.
verified_at · string
verified_at (YYYY-MM-DD) and verified_by record when a caps claim
was last established and how. Required on provider_compat
channels: theirs is the claim most likely to rot, because it
describes someone else’s server, which can change behavior
without telling anyone.
verified_by · string
verified_at (YYYY-MM-DD) and verified_by record when a caps claim
was last established and how. Required on provider_compat
channels: theirs is the claim most likely to rot, because it
describes someone else’s server, which can change behavior
without telling anyone.
op · string
op is add, replace or remove, and is read only when the enclosing
provider op is patch.
providers[]
Section titled “providers[]”One upstream provider.
name · string
name identifies this provider in routing decisions, CEL rules,
budgets, logs and metrics. Defaults to the preset name.
preset · string
preset names an entry in the compiled provider catalog
(internal/catalog), which supplies dialect, base URL and provider
quirks. Name defaults to the preset. base_url may override the
catalog endpoint (e.g. a proxy or region). dialect must be left
empty, because the catalog owns it.
dialect · string
dialect is the API the upstream speaks, “openai” or “anthropic”.
It decides whether a request is forwarded byte-for-byte or
translated on the way. A preset owns its dialect. Set this only
for a provider declared without one.
base_url · string
base_url is the upstream root that request paths are appended to.
With a preset it overrides the catalog endpoint (a regional host,
an internal proxy, a VPC endpoint), leaving everything else the
preset knows about the provider in place.
pool · object
pool declares this provider to be a set of self-hosted model
servers rather than one endpoint. Mutually exclusive with
base_url, which names a single upstream.
region · string
region fills the {region} placeholder in region-parameterized
catalog endpoints (e.g. the bedrock preset’s
bedrock-mantle.{region}.api.aws). Required by presets whose base
URL carries the placeholder unless base_url overrides it.
on_fidelity_loss · string
on_fidelity_loss overrides the file-level policy for this provider:
“reject” or “allow”. Empty inherits the file default.
metadata · map of string
metadata is what a rule may know about this provider beyond its
name, as provider.metadata.<key> in a CEL condition:
residency: sa, tier: sovereign, contract: dpa-2026. The
gateway reads none of it. It exists because the facts a
residency or a classification rule turns on are facts about the
destination, and a rule that had to spell them as a list of
provider names went stale the day a provider was added. Region
is not this: it fills a URL placeholder for a preset, and a
rule about where data may go should not depend on how an
endpoint is spelled.
api_key · string
api_key is the real provider credential inline, normally an env
reference like ${OPENAI_API_KEY}. Mutually exclusive with
Credential. One of the two is required when auth is set: the
client’s credential is the gateway’s own then, and the alternative
would be forwarding it to the provider.
credential · string
credential names an entry in the credentials list.
models · list of string
models routes requests here: exact names or trailing-* prefixes,
e.g. “gpt-4o” or “claude-*”.
model_aliases · map of string
model_aliases rewrite the model field on the way out: the key is
what clients ask for, the value is what the provider is sent. The
alias is advertised in GET /v1/models too, so a client can use
“fast” without knowing what it currently resolves to.
stream_usage · bool
stream_usage controls whether the gateway asks a streamed
OpenAI-dialect response to report token usage
(stream_options.include_usage). Unset means yes, whenever there is
a budget or a meter that will read the numbers.
It has to be asked for. An OpenAI-dialect stream reports no usage unless the request requested it, so without this a streamed request settles against the reservation estimate, a guess from body length and max_tokens. A self-hosted pool has no usage API to reconcile that guess against later. The gateway adds the option and removes the extra chunk it produces, so the client’s stream is the one it would have received either way.
Set it to false for an OpenAI-compatible server that rejects unknown request fields rather than ignoring them. Ignored for Anthropic-dialect providers, which report usage unasked.
first_byte_timeout · string
first_byte_timeout bounds how long one attempt at this provider may
take to return response headers, e.g. “10s”. An attempt past it is
abandoned: retried or failed over when a policy names “timeout”,
failed as unreachable otherwise. It is required by, and refused
without, a “timeout” trigger that could fire it: this provider’s
retry.when or failover.when, or the failover.when of a provider
that lists this one. Opt-in because a provider that was merely
slow may still complete, and bill, the request it was abandoned
on.
retry · object
retry tries this provider again, after a wait, when an attempt
fails before any byte has reached the client. Off unless set. A
provider’s retries run out before failover begins, and a failover
candidate’s own retry policy governs its attempts.
failover · object
failover names the providers to try, in order, when this one fails
before any byte has reached the client. A candidate is any other
provider in this file, with its own credential, dialect, aliases
and pool. It need not claim any model of its own. Each candidate
is admitted the way a primary is (access rules with provider
set to it, the fidelity guard against this request’s body), and
one the key may not use is skipped, not tried.
What cannot move: a stream once its first frame has reached the client, a WebSocket session, an MCP call. A request that fails everywhere gets the last provider’s own response, not a synthetic one.
cost_per_session · integer
cost_per_session is the flat budget cost reserved for one WebSocket
session tunneled to this provider (OpenAI Realtime, Gemini Live),
in the bound budget’s units. Default 1. On top of it, token usage
the upstream reports inside the session (response.done /
usageMetadata events) spends incrementally as it streams past.
When the budget breaks mid-session the gateway closes both sides
with WebSocket close code 1008.
providers[].pool
Section titled “providers[].pool”A set of self-hosted model servers behind one name, for the pool that nothing else schedules: a rack of vLLM machines with no inference gateway in front of them.
Nothing here reads a cluster API. Service is a name that resolves to
every server (a headless service in Kubernetes, a round-robin record
anywhere else), and that answer is the pool membership.
A pool that needs real scheduling, such as prefix-cache affinity or prefill/decode disaggregation, is fronted by an inference gateway built for it (llm-d, or any Gateway API Inference Extension implementation). It is then not a pool to pistra at all: it is a provider with a base_url, the gateway’s, and the gateway schedules inside it.
service · string
service is the root the pods serve, normally a headless service:
http://vllm-llama.default.svc.cluster.local:8000. Required.
picker · object
picker is who decides which of this pool’s model servers serves
a request: llm-d’s Router, upstream’s LWEPP, anything speaking
the Gateway API Inference Extension’s picker protocol. The
gateway asks it per request and dials what it names, after
checking the answer against the pool’s own membership.
Empty is a pool nothing schedules. Requests still go round its members in turn rather than to the pool’s own address, because leaving them on the address pins one model server per connection for that connection’s life. Spreading in turn is not scheduling and does not claim to be: it reads nothing about the servers and cannot rank them.
A pool that wants ranking, whether on queue depth, prefix-cache affinity or prefill/decode disaggregation, gets a picker rather than a setting here, because that ranking is an inference gateway’s job and not this one’s.
subsets · list of object
subsets are named parts of this pool that policy can confine a
request to: a tenant’s reserved pods, a premium tier, a group of
nodes with particular hardware.
A virtual key naming a subset may only be served from it. That is enforced, not requested: the gateway routes to the subset’s own endpoints, and a pick outside them is refused.
providers[].pool.picker
Section titled “providers[].pool.picker”The remote scheduler for a pool.
The gateway consults it over the same ext_proc service it serves to Envoy, with the direction reversed: here the gateway is the client, and the picker answers with an endpoint to dial.
The answer is not trusted. It is checked against the addresses the pool’s own name resolves to before anything is dialed, because a destination chosen by another process and dialed without a check is arbitrary-destination forwarding. That check has a cost worth knowing: the picker watches Kubernetes and this gateway watches DNS, so a pod the picker knows about can be one the pool has not resolved yet, and a pick inside that window is refused and falls back.
service · string
service is the picker’s address, and the scheme chooses the
transport: http:// dials it in the clear, https:// with TLS. The
port is required, because an endpoint picker serves gRPC on a
port of its own (9002 by convention) rather than at a web root:
http://llama-epp.default.svc.cluster.local:9002. Required.
on_error · string
on_error is what happens when the picker cannot be reached or does
not answer in time. Two values, and the default is the first:
“fallback” serves the request without it, the way a pool with no picker is served. A scheduler that stops answering is a degradation rather than a policy bypass. The fence is on this side, so nothing reaches an endpoint outside the pool either way.
“refuse” fails the request instead. For the deployment where being scheduled is the point, such as disaggregated serving or a pool whose pods are not interchangeable, being spread over it arbitrarily is worse than a refusal.
A picker answering with a refusal of its own is not this case. 503 (nothing eligible) and 429 (shedding load) are that request’s answer, and reach the caller whichever value this holds.
timeout · string
timeout bounds one decision. The default is 100ms. It is spent
before any byte goes upstream, so it is a floor under every
pooled request: a picker slower than this costs more than the
imbalance it corrects. A duration such as “250ms”.
insecure_skip_verify · bool
insecure_skip_verify encrypts the connection to the picker without
authenticating it. Only for an https:// service.
This exists because the reference implementations generate a self-signed certificate in memory at startup and serve TLS with it by default. Nothing signs it, it is written nowhere, and it changes every restart, so there is no bundle any amount of trust configuration could name. A gateway that wants to speak to one as deployed has the choice of not verifying, or of running the picker with TLS off and naming it http://. Both are offered because both are real.
A picker with a certificate somebody signed needs no setting here. The connection is verified against this node’s trust store, so a mesh CA or a cert-manager issuer is trusted by putting its bundle where the machine keeps them. There is deliberately no field for that path. A bundle is a path on one machine, and this document is the half that travels through the admin API, a CRD, a Terraform resource and the raft log, so it may not carry one. It is the same reason issuers are node-half and file-owned.
What it gives up, precisely: anything that can occupy the picker’s address can answer as the picker. What that buys an attacker is bounded by the fence, which is on this side and is not part of the trust decision: the endpoints an answer may contain are the ones the pool’s own name resolves to, so a picker nobody authenticated can influence which model server serves a request and cannot send it anywhere else. Bounded is not nothing, and it is not the same as safe. Prefer a signed certificate where the deployment can issue one.
providers[].pool.subsets[]
Section titled “providers[].pool.subsets[]”Names part of a pool.
name · string
name is what a virtual key’s pool_subset refers to. Unique within
the provider. The same name may be declared by several pools, so
one key can mean “my tenant’s pods” across all of them.
service · string
service is the subset’s address, a headless service selecting the
pods in it, on the pool’s port. Required.
providers[].retry
Section titled “providers[].retry”A provider’s retry policy: the same provider again, after a wait.
attempts · integer
attempts is how many further tries this provider gets after the
first, 1 to 5. Required. The cap is deliberate: a gateway
retrying an overloaded provider is how outages spread, so the
policy that wants more is refused rather than honoured.
when · list of string
when lists the failures retried: “connect”, “5xx”, “429”, and
“timeout” (which needs first_byte_timeout). Empty means connect,
5xx and 429. Spelled “when” for the same reason failover’s is.
backoff · string
backoff is the wait before the first retry, “200ms” by default.
Each further retry doubles it, jittered, up to max_backoff. A wait
the request’s own deadline cannot hold is not taken. The plan
moves on to failover instead.
max_backoff · string
max_backoff caps the wait between retries, “2s” by default.
retry_after_cap · string
retry_after_cap bounds how long a 429’s Retry-After is honoured,
“5s” by default. A 429 that names a wait is retried after exactly
that wait rather than the backoff. One asking for more than this
is not waited for, and the plan moves on to failover.
providers[].failover
Section titled “providers[].failover”A provider’s failover policy.
to · list of string
to lists the candidate providers in the order they are tried.
Required. Every name must be another provider in this file. Only
this list is walked. A candidate’s own failover.to applies when
that candidate is the primary, not here.
when · list of string
when lists the failures that move a request to the next candidate:
“connect” (the upstream could not be reached), “5xx”, “429”, and
“timeout” (no response headers within the provider’s
first_byte_timeout, which this provider and every candidate must
then set). Empty means connect, 5xx and 429. A 4xx other than 429
never fails over: the same request would fail everywhere.
The key is “when” and not “on” because “on” is a YAML 1.1 boolean: the YAML pass turns the key into true before the strict decoder sees it, and an unquoted on: list is refused as an unknown field.
budgets[]
Section titled “budgets[]”One windowed cost limit.
name · string
name is what a profile names in its budget field. Nothing
references a budget directly from a key: a key names a profile
and the profile names the budget, so an allowance is edited once
for everyone drawing on it.
cost · object
cost is what one request draws from the allowance.
limit · integer
limit is the number of cost units per window.
window · string
window is minute/hour/day/week or a Go duration (“12h”). The
default is day. Windows are fixed, epoch-aligned, in UTC.
shared · bool
shared pools all referencing keys into one bucket. The default
gives each key its own allowance of limit.
budgets[].cost
Section titled “budgets[].cost”Selects how a request’s cost is computed.
type · string
type is a filterapi.LLMRequestCostType: InputToken, OutputToken,
TotalToken (default), CachedInputToken, CacheCreationInputToken,
ReasoningToken, or CEL.
cel · string
cel is the expression when type is CEL, e.g.
“input_tokens + output_tokens * uint(4)”.
profiles[]
Section titled “profiles[]”One named policy. Keys reference it by name, and nothing here is per-key, so it can be edited once for everyone holding it.
A key that needs its own limit gets its own profile. That reads as more configuration than a per-key override would, and it is: an override is a second place the answer can live, which is the thing profiles exist to remove.
name · string
name is what a virtual key names in its profile field.
allowed_models · list of string
allowed_models restricts traffic to these model names or
trailing-* prefixes. Empty means every model the routing table
serves.
require_caps · list of string
require_caps excludes channels missing any listed capability.
Policy, not preference: on_fidelity_loss: allow never overrides
it.
pool_subset · string
pool_subset confines traffic to one named subset of whichever pool
serves it. Empty means the whole pool.
budget · string
budget names the cost limit this profile’s traffic draws from.
Empty is unmetered. Per-key allowances come from the budget’s own
shared: false default, so one profile naming one budget still
gives every key holding it its own bucket.
access_rules · list of object
access_rules are this profile’s own rules, evaluated after the
deployment-wide access_rules at whichever hop a rule’s at
names, reading the same variables. They narrow: a deny is final,
an allow ends this list only.
guardrails · object
guardrails narrows which of the deployment’s detectors inspect
this profile’s traffic. Omitted means all of them.
profiles[].guardrails
Section titled “profiles[].guardrails”The part of the guardrail ensemble a profile’s traffic is inspected by.
A profile selects. It does not define. There is one ensemble in a
deployment and one set of detector instances behind it, so two
profiles that both select pii are inspected by the same pii.
That is what lets them share the delta-scan cache, and what stops
the expensive tier from being loaded once per profile.
The rules are deliberately not here. A guardrail rule reads key and
so already reads key.profile, which makes a rule about one
population expressible today. A rules block on the profile would give
it a second place to live, which is the thing profiles exist to
remove.
detectors · list of string
detectors names the members that inspect this profile’s traffic,
from guardrails.detectors[].name. Omitting the field is every
detector. An explicitly empty list is none of them, which is how a
population is exempted from inspection.
A detector may only be left out when no rule reads it, because
leaving one out can quietly stop a deployment-wide rule from
matching. Rules say what they read in guardrails.rules[]. requires, and one that says nothing depends on all of them, so a
deployment that has annotated no rules permits no selection.
One upstream MCP server, exposed at /mcp/
name · string
name is the path segment the server is reachable at, /mcp/server.
url · string
url is the server’s Streamable HTTP endpoint, used verbatim,
e.g. https://mcp.example.com/mcp.
api_key · string
api_key, when set, replaces client credentials with
“Authorization: Bearer <api_key>”, the shape MCP servers with
static tokens expect. Normally an env reference like
${GITHUB_MCP_TOKEN}. Unlike providers, a credential is optional
even when auth is set: unauthenticated internal servers are
legitimate, and the gateway strips the client’s credential before
forwarding either way.
credential · string
credential names an entry in the credentials list. Mutually
exclusive with api_key.
cost_per_call · integer
cost_per_call is the budget cost of one request to this server, in
the bound budget’s units. Default 1: MCP calls have no token
usage, so budgets meter requests. A key whose budget also meters
LLM tokens mixes units. Give MCP-heavy keys their own budget.
restore · list of string
restore names the entity types whose format-preserved values are
turned back into the originals in this server’s tool arguments,
after guardrails have run. Empty restores nothing. That is the
default, and the answer for every server somebody has not
deliberately decided about.
It is the declaration that this server is inside the boundary: a system of record that both may see the real values and needs them, because a lookup against a stand-in finds the wrong customer or none. That is a fact about the counterparty, not about the hop, so it is named here beside the URL rather than as a rule somewhere.
Rehydration happens only at a hop the gateway itself dials. That asymmetry is the security argument and not a preference: here the destination is this configured upstream and nowhere else, while a value handed back in a response goes to whoever called and is then out of sight. Declassifying to a caller is a different act and has no spelling.
Each type listed must be one the fpe operator has an alphabet for, and guardrails must be configured with fpe_key. A list that could never restore anything is refused at load rather than sitting inert. Reversal is bounded to tool arguments. See guardrails.Engine.Restore for why position is the only discriminator available, and what it costs.
registry · object
registry gives the server an identity in the deployment’s MCP
registry projection (/registry/v0.1). A server without one is
still served at /mcp/
mcp[].registry
Section titled “mcp[].registry”One MCP server’s registry identity and display metadata: the fields of the MCP Registry’s server.json that describe a server rather than route to it. Routing and governance (URL, credential, cost, restore) stay on the parent MCPServerConfig, so this block carries no secrets and never needs one.
name · string
name is the registry name: reverse-DNS namespace, one slash,
one short name, as in com.example/payments. The namespace is the
Cedar publish boundary (Resource::“registry/com.example”), and
the whole name is a second path the data plane serves the server
at, /mcp/
version · string
version is the publisher’s version string, echoed as metadata.
The gateway keeps one live entry per name, latest wins, so
this orders nothing. Empty is projected as “0.0.0”.
title · string
title is the human display name.
description · string
description is what a host shows next to the name. Required, and
capped at 100 characters, because the projection publishes it
into a schema that requires exactly that.
website_url · string
website_url links the server’s homepage.
repository · object
repository points at the server’s source, when it is somewhere.
icons · list of object
icons are display icons, passed through as published.
mcp[].registry.repository
Section titled “mcp[].registry.repository”Locates a server’s source repository.
url · string
url is the repository’s browse URL.
source · string
source names the host: “github”, “gitlab”, …
subfolder · string
subfolder is the path within the repository, for monorepos.
mcp[].registry.icons[]
Section titled “mcp[].registry.icons[]”One display icon.
src · string
src is where the icon is fetched from, an https URL.
mime_type · string
mime_type is the icon’s media type, e.g. image/png.
sizes · list of string
sizes lists the raster sizes, e.g. “48x48”, or “any” for SVG.
theme · string
theme is “light” or “dark”, where the icon is theme-specific.
One upstream peer agent, exposed at /a2a/
name · string
name is the path segment the agent is reachable at, /a2a/peer.
url · string
url is the agent’s JSON-RPC interface, used verbatim: the url
its card lists for the JSONRPC protocol binding.
api_key · string
api_key, when set, replaces client credentials with
“Authorization: Bearer <api_key>”. Normally a reference like
${secret:PLANNER_TOKEN}. Optional even when auth is set: the
gateway strips the client’s credential before forwarding either
way.
credential · string
credential names an entry in the credentials list: a static key,
a client_credentials grant, or a token_exchange that lets the
agent see the caller. Mutually exclusive with api_key.
cost_per_call · integer
cost_per_call is the budget cost of one request to this agent, in
the bound budget’s units. Default 1: A2A calls have no token
usage, so budgets meter requests.
restore · list of string
restore names the entity types whose format-preserved values are
turned back into the originals in the message parts this agent
receives, after guardrails have run. The same declaration, with
the same argument, as an MCP server’s: this agent is inside the
boundary and needs the real values, and this hop is one the
gateway itself dials.
card_url · string
card_url is where the agent’s own card is, when it is not at the
well-known path of the URL’s origin. The gateway fetches it,
rewrites the address and the security schemes to its own, signs
the result and serves it at /a2a/
a2a_cards
Section titled “a2a_cards”Signs the cards the gateway projects for its agents.
signing_keys · list of string
signing_keys are EC P-256 private keys in PEM, normally references
like ${secret:a2a-card-key}. The first signs. Every one is
published at /.well-known/a2a-jwks.json under its RFC 7638
thumbprint as kid, so rotation is add the new key first, drop the
old one once nothing cached carries its signature.
reconcile
Section titled “reconcile”Tunes the usage reconciliation loop. Applied on every rebuild, like the rest of the deployment: a reload, or a write to whichever configuration source owns this block, reconfigures the poller in place.
It needs somewhere to keep its ledgers, so cluster must be set. In
a cluster every node meters and only the raft leader polls. What
polling produces is replicated, so the reconciliation view at
GET /admin/v1/reconcile is the same on every node and reading it never
means finding the leader first.
Metering, once started, outlives the block. Turning reconciliation off stops the polling, including the admin-API credential below. The node goes on recording what it served, because the audit compares a provider’s report against what this cluster metered and can only recognise a day it never metered at the start of the ledger, not in the middle of it. A block that came and went would otherwise leave a hole that reads as drift.
interval · string
interval between fetch/judge rounds, a Go duration. Default “1h”.
maturity_lag · string
maturity_lag is how long after a UTC day closes before it is
judged. The usage APIs trail and restate recent buckets. Default
“48h”.
lookback · string
lookback is how far back each round refetches, and so how long an
outage can last before its days go unjudged. Default “168h” (7
days). Must exceed maturity_lag by at least a day or no day is ever
both fetched and mature. A smaller value is widened with a
warning. At most 248 days, what one fetch can walk; a larger
value is narrowed with a warning.
retention · string
retention is how much ledger history to keep, a Go duration.
Default “2160h” (90 days). Days older than it are dropped at the
end of each round, so the metered and provider ledgers stay
bounded. The judged-through watermark and the metering-since
floor are scalars and are never pruned. Must exceed lookback or
pruning would outrun judging. A smaller value is widened with a
warning. Set a negative duration to keep history forever.
drift_ratio · number
drift_ratio is the relative disagreement per token class that
breaches, as a fraction of the larger side. Default 0.02.
drift_floor · integer
drift_floor is the absolute token delta below which a class never
breaches. Default 10000.
providers · list of object
providers selects which providers are audited. One absent here is
not checked at all: reconciliation reads an admin API with a
credential the data plane never holds.
reconcile.providers[]
Section titled “reconcile.providers[]”Audits one provider.
provider · string
provider names the providers entry whose metered usage this
audits.
type · string
type selects the usage API dialect: “openai” (organization usage
API) or “anthropic” (Admin API usage report). Explicit because a
provider’s data-plane dialect does not imply an admin API. Most
OpenAI-compatible vendors have none.
admin_key · string
admin_key is the org-scoped admin credential, normally an env
reference like ${OPENAI_ADMIN_KEY}. Not the data-plane key.
base_url · string
base_url overrides the admin API host (tests, gateways). Defaults
to the provider’s canonical API host.
api_key_ids · list of string
api_key_ids scopes the query to these provider API key IDs. Scoped,
drift in either direction is a hard signal. Unscoped,
provider-over-metered drift may just be traffic on the same
account that never traversed the gateway.
access_rules[]
Section titled “access_rules[]”Also configured at profiles[].access_rules[].
One CEL rule over one request, at one or more of
the hops the gateway admits requests at. What condition may read
depends on at:
- at every hop: key (name, metadata), agent (name, issuer, subject) and user (name, issuer, subject, email), naming who is calling, as the credential established it;
- llm: model, provider and route (chat_completions, embeddings and the other routes a provider serves);
- mcp: server, method (the JSON-RPC method, “tools/call”), name (the tool, prompt or resource named; “” on other methods) and args (the tools/call arguments as read from the body, so they are the values the upstream will execute and argument policy needs no schema and no header trust);
- a2a: peer (the agent’s configured name), method (“SendMessage”), role (ROLE_USER or ROLE_AGENT, on the two message methods), task_id and context_id where the method names them, and parts, the message’s parts as a list of maps with kind (text, file, data), media_type, url and filename. Those say what a part is, never what it says, which is guardrails’ to read.
name · string
name appears in the refusal and in metrics, so it is what an
operator reads when a request is denied.
at · list of string
at names the hops this rule runs at: llm, mcp, a2a. Empty means
every hop, and such a rule may read only what every hop declares,
which is key, agent and user. A rule at two hops may read what
both declare. Its condition is compiled against each and refused
where either does not know a name it uses.
condition · string
condition is the CEL expression. The rule matches when it
evaluates true. The strings, lists and sets extension libraries
are available, as they are in Kubernetes admission policy.
action · string
“allow” or “deny”
message · string
message is the refusal text of a deny rule: what the client reads
instead of request denied by access rule "<name>". It is static
on purpose. An expression here would evaluate over the request
and write the result into the response, which is a channel out
for the data policy_error keeps in.
on_error · string
on_error is what a condition that cannot be evaluated means:
“fail_closed” (default) refuses the request with 403
policy_error, “fail_open” skips the rule and logs. A condition
fails to evaluate in two cases. It reads a key that is not there
(key.metadata.team on a key with no team, args.path on a
tool call without one), which has() guards against. Or it
exceeds its cost budget over arguments the client sized. The
default is closed because the second case is the client’s to
cause, and a deny rule that a large enough argument list could
switch off is not a deny rule.
controls · list of string
controls are the governance controls this rule enforces, as
identifiers an external system assigned (eu-ai-act:art-50,
nist-ai-rmf:govern-1.2), carried onto every audit record the
rule decides and never interpreted. Opaque strings by design: the
framework and its taxonomy are the governance platform’s to
choose. Checked for shape only: non-empty, no whitespace, no
duplicates.
guardrails
Section titled “guardrails”The guardrails: section of the gateway config. Guardrails are
configuration, not delivery packages: the same file that carries
access_rules carries which detectors run and what policy does with their
annotations.
detectors · list of object
detectors are the ensemble members, in the order they are planned.
rules · list of object
rules are CEL conditions over the annotations of one request,
evaluated in order. A matching deny rule is terminal. Redact and
annotate rules accumulate. No matching rule means allow.
score_threshold · number
score_threshold drops annotations scoring below it before policy
sees them. Presidio’s default_score_threshold, same position in the
pipeline: after context enhancement, before deduplication.
context_enhancement · bool
context_enhancement enables the context-word score boost. Defaults
to true. Set false to score patterns in isolation.
agreement_boost · number
agreement_boost is added to an annotation’s score when a second
detector independently annotates an overlapping span with the same
entity type. Zero disables it. It does nothing with one detector.
allow_list · list of string
allow_list removes annotations whose matched text is listed. It is
Presidio’s allow_list, for the company address and the demo credit
card number that must not keep tripping the gateway.
allow_list_match · string
allow_list_match is how [Config.allow_list] entries are read:
“exact” (the default, and Presidio’s) or “regex”.
exact is string equality against the whole matched span, case and
all. regex joins every entry into one alternation and asks whether
it is found anywhere inside the span, under Presidio’s own
default flags (case-insensitive, dot matches newline, ^ and $ at
line boundaries). Both of those are wider than exact, so regex
with a bare term like acme allows every span containing it.
Anchor it (^acme$) to say what exact would have said.
The distinction matters because this list suppresses detections. Widening it is not a convenience, it is a hole. So the default stays exact and case-sensitive even though the opposite reads friendlier.
conflicts · string
conflicts selects cross-detector conflict resolution:
“merge_similar_or_contained” (default) drops duplicates and
contained spans of the same entity; “remove_intersections” also
resolves partial overlaps, keeping the higher-scoring span; “none”
keeps everything.
cache_entries · integer
cache_entries bounds the delta-scan cache (segments already scanned
by an identical engine). Default 4096. Zero disables caching,
which is only sensible in tests.
explain · bool
explain keeps the per-annotation decision trace
(analysis_explanation) in results. On by default. The trace holds
the matched pattern, not the matched text.
stream · object
stream configures inspection of streamed responses. It only does
anything when some detector lists output in apply_to, because a
response guard delays the stream and that is never implicit.
response · object
response configures inspection of unary (non-streamed) responses.
Same gate as stream: nothing happens unless a detector asked for
output.
overlap · object
overlap configures detectors scheduled alongside the upstream call.
It does nothing unless some detector sets schedule: overlap.
async · object
async configures detectors nothing waits for. It does nothing
unless some detector sets schedule: async.
pseudonym_key · string
pseudonym_key keys the “pseudonym” operator, as hex. Any length is
accepted and 32 bytes is the sensible one.
It makes a pseudonym stable: the same value reaches the same stand-in in every request, in every replica, with nothing stored between them, because the key is the only state there is. That is also the cost. Rotating it renames everybody, and two gateways in front of one workload want the same key or they will disagree about who is who.
Deployments that redact but never pseudonymise leave it empty. A rule naming the pseudonym operator without it is refused at Compile rather than quietly rewriting everyone to the same name.
fpe_key · string
fpe_key keys the “fpe” operator, as hex. It is an AES key, so it is
16, 24 or 32 bytes and nothing else (unlike pseudonym_key, which
feeds an HMAC and takes any length).
It is a separate key from pseudonym_key because it is a separate
promise. A pseudonym is one-way and its key can be rotated
whenever somebody is willing to have everyone renamed. An FPE key
is the only thing that can ever turn a value back, so rotating it
is a migration and not a preference. Sharing one key would make
the cheaper rotation impossible without noticing.
fpe_tweak · string
fpe_tweak separates one deployment’s permutation from another’s
under the same key. It is not a secret (NIST’s tweak never is),
and it is the knob for correlation: two tenants given different
tweaks cannot tell that they hold the same account number, and
two gateways given the same tweak agree about every value.
fpe_alphabets · map of string
fpe_alphabets declares, per entity type, which character set that
type’s values are written in: one of digits, upper, letters,
alnum_upper or alnum. It overrides the built-in table, which
covers the types whose values are decimal digits everywhere they
are issued and nothing else.
The alphabet fixes the radix, which fixes the ciphertext. Two gateways that disagree here disagree about every value of that type, so it is named per type rather than per rule.
guardrails.detectors[]
Section titled “guardrails.detectors[]”Configures one ensemble member.
type · string
type selects the detector implementation (“pii”).
name · string
name distinguishes two instances of the same type. It defaults to
Type.
entities · list of string
entities restricts which entity types this detector reports. Empty
means everything it supports. This is the primary cost lever: the
pattern engine’s work is linear in the recognizer count.
recognizers · list of string
recognizers restricts the underlying recognizer classes by name,
for detectors that have them (pii). Empty means all active ones.
A custom recognizer is selected by its name, like a class.
custom_recognizers · list of object
custom_recognizers adds pattern recognizers the deployment writes
itself beside the Presidio-derived ones (an employee-id format, a
list of project codenames, a customer-number scheme), for the pii
detector only. Each is Presidio’s PatternRecognizer: patterns
with scores, a deny list of whole words, context words. Once
compiled it is an ordinary recognizer with the same scoring,
context boost, deduplication and streaming horizon as a built-in.
Two things differ from a built-in. The regex is RE2 as written, with none of the lookaround translation the generated tables went through, and a pattern RE2 refuses is refused here. The one exception is a leading or trailing \b, which is held to unicode word boundaries at the match edges as the built-ins are. And no prefilter knows the pattern, so it scans every segment whole. That is linear in the text, about a millisecond per pattern per 64 KB. Two or three are lost in the noise of a scan. Ten cut clean-prose throughput four times, so a country’s worth of identifiers is built in rather than shipped as a pack of these.
phone_regions · list of string
phone_regions lists the regions (ISO 3166 alpha-2) whose national
number formats the pii detector’s PhoneRecognizer reads, for
example [SA, AE, GB]. A number written internationally (+966 …)
is found whatever the list says. The list makes 0501234567 a
Saudi mobile rather than ten digits. Empty is Presidio’s default
(US GB DE FR IL IN CA BR) and the six Gulf states (SA AE QA KW BH
OM). A region libphonenumber does not know is refused at load.
Each region is a library call per run of digits long enough to be one of its numbers, so the list is also a cost. On text dense with long numbers the fourteen defaults spend about 40% more in this recognizer than Presidio’s eight, and a deployment that names its own two or three spends a fraction of either. Ordinary prose is unaffected. A year or a time is shorter than any region’s shortest number and is never shown to the library.
remote · object
remote configures the “remote” detector type: where the analyzer
lives and how to talk to it.
nlp · object
nlp configures the “nlp” detector type: a token-classification
model run in this process through ONNX Runtime.
llm · object
llm configures the “llm” detector type: a judge, which is a chat
model asked whether a policy written in prose applies to a segment.
classify · object
classify configures the “classify” detector type: a
sequence-classification model (a prompt-injection or
content-safety checkpoint) whose one row of scores per segment
becomes a verdict over it.
embed · object
embed configures the “embed” detector type: an embedding model and
the operator’s own examples of each topic, so a segment is scored
by how near it sits to what the examples describe (scope, known
jailbreak phrasings, subjects the assistant must not discuss),
with no training and in any language the model embeds.
prefilter · string
prefilter selects how the “pii” detector narrows the text its
pattern pass scans. Empty or “native” uses Vectorscan through
libhs, which the gateway image ships and the packages depend on.
On a host without it startup fails naming this setting. “pure”
uses the built-in re2c DFA, which needs nothing installed.
The built-in pass is anchored (it restarts the DFA at every byte offset), so its cost is quadratic in the length of an unbroken run of word characters, which no ordinary prose contains and a caller can send anyway. Vectorscan is linear. The gap therefore has no single value: it grows with the run, and quoting one number for it is how this comment used to be wrong.
On ordinary traffic Vectorscan is the faster of the two as well, for a reason the built-in cannot copy: it reports which pattern each match came from, so the pattern pass behind it runs each pattern over its own regions alone. The built-in is one union automaton that can only say “something matched here”, and every pattern has to be run over every region. The difference is largest on the text a gateway mostly carries (prose with an ordinary number in it, a percentage, a time, a count), where the built-in runs some hundred and fifty thousand pattern-region pairs per 64KB and Vectorscan a few thousand. Before attribution it was the slower of the two on that text, by up to 16x. The earlier version of this comment said so, and was right at the time.
Detect over 256KB, on an Apple M4 (BenchmarkPrefilterTraffic and BenchmarkPrefilterPathological in pii, which say how to reproduce them):
pure nativeprose, no digits 13.1ms 8.0ms native 1.6x fasterprose with digits 46.1ms 12.2ms native 3.8x fasterPII-dense corpus 327ms 277ms native 1.2x faster64KB repeated letter 1.38s 3.27ms native 422x faster256KB repeated letter 23.7s 8.36ms native 2830x fasterSo “native” is the faster pass everywhere and the only bound on
the worst case, and that is why it is the default. It costs a
native library on the host, which the image carries and a bare
binary needs installed (libvectorscan5 on Debian and Ubuntu,
vectorscan on Fedora, brew install vectorscan on a Mac). A
request deadline bounds the worst case either way (see the
detector timeout). Only “native” keeps it off the clock. Both find
the same PII, and the same Presidio vectors are replayed through
each to keep it that way.
There is deliberately no “use it if you find it” setting. It would make one configuration mean two different things on two machines, in both directions and by orders of magnitude, with nothing in the logs to say which you got. A missing library is a startup error you can read, not a silent downgrade, and “pure” is how a host without the library says so on purpose.
apply_to · list of string
apply_to restricts the segment kinds this detector reads: system,
user, assistant, tool_args, tool_result on the request side, output
for model responses, and header and query for the request envelope.
Empty means every writable request kind, and deliberately not
output, header or query. Each of the three is opt-in for a cost of
its own. Reading responses delays a stream. Reading headers and
the query runs the whole ensemble (remote detectors included) over
twenty to forty values per request, one of which is usually
authorization: Bearer sk-…. A config that starts shipping
credentials to an analyzer, or writing findings about them into
audit records, because a list was left blank is the worst
available default. So the list has to say so.
header and query are read-only ([Kind.Writable]): a detector may find things there and policy may deny or annotate on them, but no rule can rewrite them. See [KindQuery] for why that is structural rather than a limitation waiting to be lifted.
A detector type may narrow that default to the kinds it is for.
The llm judge reads user and tool_result unless told otherwise:
those are the two kinds an outsider writes, and a judge run over
the operator’s own system prompt answers a question nobody asked.
Naming the kinds here always wins.
The lever is all-or-nothing across every header name, and there is
no per-name restriction yet. The rule can be narrower than the
detector, though. An envelope annotation’s ref addresses the
value it came from, so a.ref != ".headers.authorization" scans
the query string and the other headers while ignoring the
credential the request presented at the door. That matters most
with the credential recognizers on, since otherwise every request
annotates its own key. The delta-scan cache absorbs most of the
repeat cost (header values are the most repetitive text in the
system). Even so, a deployment that enables header scanning
should raise cache_entries, because those values now compete with
conversation segments for the same 4096 slots.
mode · string
mode is “inline” (default; annotations reach policy and can deny or
redact) or “shadow” (annotations are recorded for audit and
deliberately ignored by policy: how a new detector earns trust).
schedule · string
schedule is when this detector runs relative to the request:
sync (default) the request waits for it before anything is forwarded, so it can still deny or rewrite the body. overlap it runs alongside the upstream call and the response is held until it finishes, so it can still deny. async nothing waits for it at all. Its findings land in the delta-scan cache, where the next turn’s sync pass reads them and enforces them at cache speed. A conversation therefore converges on full coverage one turn after any given text first appears. By construction an async detector is best-effort, so a detector that must not be skipped must not be async.
Schedule is orthogonal to mode: mode says whether policy reads the
findings, schedule says whether the request waits for them.
It governs request inspection only. A detector pointed at output
is inspecting a response that has by definition already arrived, so
there is nothing left to overlap with and it runs inline there
whatever this says.
role · string
role is “discover” (default; scans whole segments) or “confirm”
(runs only over windows another detector already annotated, so an
expensive detector costs a function of the findings rather than of
the traffic).
confirm_window · integer
confirm_window is how many bytes around a discovered span a confirm
detector sees. Default 96.
narrow_spans · bool
narrow_spans lets a confirm detector narrow the spans it confirms:
where a discovered span strictly contains what this detector found
for the same entity, the discovery shrinks to the hull of those
findings. It pairs an imprecise locator with an exact one and lets
the exact one say where the bytes are.
It exists because a detector can be right about what it found and wrong about where it begins. An analyzer running a transformers NlpEngine reports whole spaCy tokens under the default alignment_mode (see [RemoteConfig]). A pattern engine confirming that finding knows the exact bytes, and this is how it gets to say so.
Off by default, and it should stay off unless the pairing is deliberate. Narrowing asserts that this detector’s span is the whole entity, so a confirm pass matching only part of a longer one would leave the rest in the clear. Resolution otherwise prefers the longer span, following Presidio’s remove_duplicates. This inverts that where an operator asks for it and nowhere else.
Only meaningful with role “confirm”.
threshold · number
threshold drops this detector’s annotations below the given score.
It is read after context enhancement, like the global ScoreThreshold,
so the score compared is the one the audit trail shows: a match
lifted by the words around it clears a threshold its bare pattern
would not.
timeout · string
timeout bounds one detector’s work on one request, a Go duration.
Empty means the request’s own deadline governs.
on_error · string
on_error is “fail_open” (default; a failing detector is logged and
skipped) or “fail_closed” (its failure refuses the request). A
detector that must not be bypassed is a policy decision, so it is
spelled out rather than inferred.
guardrails.detectors[].custom_recognizers[]
Section titled “guardrails.detectors[].custom_recognizers[]”One entry of [DetectorConfig.CustomRecognizers]. The field names are Presidio’s recognizer configuration, so an entry from a presidio-analyzer recognizers file pastes across.
name · string
name identifies the recognizer in annotations and in
recognizers: selection. It must not be the name of a built-in
class.
entity · string
entity is the entity type every match carries, in the vocabulary
rules are written against, such as pii/EMPLOYEE_ID for a pii
detector.
patterns · list of object
patterns are the regexes, each with a score. A recognizer with no
patterns is a deny-list recognizer.
deny_list · list of string
deny_list finds whole-word occurrences of each literal, with
Presidio’s semantics: the first listed word wins at a position and
matches do not overlap.
deny_list_score · number
deny_list_score is the score of a deny-list match, in [0,1]. Zero
means Presidio’s default, 1.0.
context · list of string
context words found near a match raise its score, by the same
rule and the same amount as a built-in recognizer’s.
validator · object
validator is the checksum the identifier carries, if it has one:
a matched value the checksum accepts scores 1.0 and one it
rejects is dropped, as with a built-in recognizer. The one part
of a built-in a custom recognizer could not have, and the reason
a pattern for a national ID is otherwise a pattern for any
ten-digit number.
examples · list of object
examples hold the recognizer to texts it must scan a known way. A
recognizer whose examples fail is refused at load, so the
evidence that it works travels with its definition. With a
validator, every found value is also altered one digit at a time
and must then not be found: the checksum is proven to decide, not
just declared.
case_sensitive · bool
case_sensitive turns off IGNORECASE. Presidio compiles every
pattern recognizer with DOTALL, MULTILINE and IGNORECASE unless
told otherwise, and so does this one. An entry pasted from a
Presidio configuration behaves here as it did there.
category · string
category is the annotation namespace matches carry, empty for
pii. An in-house API key format says credential here, and
every rule already written against that namespace covers it
without naming the new entity type. It must be one of the
namespaces the engine knows. An unknown one is refused at load,
because a rule can only select what the vocabulary contains.
guardrails.detectors[].custom_recognizers[].patterns[]
Section titled “guardrails.detectors[].custom_recognizers[].patterns[]”One regex of a [CustomRecognizer].
name · string
name is what the annotation’s explanation reports as the pattern
that matched. Defaults to “pattern N”.
regex · string
regex is the pattern, in RE2 syntax
(https://github.com/google/re2/wiki/Syntax). It is not PCRE and
not Python re: no lookaround, no backreferences, no atomic or
possessive constructs, and \w \s \b are ASCII. Anything outside
it is refused at load. Digits are read before the pattern runs:
every script’s decimal digits are folded onto ASCII, so \d, [0-9]
and a literal 7 all match ٧.
score · number
score is the match’s score before context enhancement, in (0,1].
guardrails.detectors[].custom_recognizers[].validator
Section titled “guardrails.detectors[].custom_recognizers[].validator”The checksum of a [CustomRecognizer]. The kinds are the few that national identifiers actually use, so a deployment that can name one does not have to ship code to get it.
kind · string
kind is luhn, verhoeff, iso7064 (MOD 97-10, letters read
A=10..Z=35), iban (MOD 97-10 as ISO 13616 applies it, with the
country code and check digits moved behind the account number
first), weighted or crc32_base62.
strip · string
strip lists characters removed before the check, such as “- “ for
a value written with dashes and spaces. It may not contain digits.
weights · list of integer
weights and modulus define a weighted check: the digits, left to
right, times the weights in order must sum to a multiple of
Modulus. The check digit’s own weight is in the list, so a value
with a different number of digits is rejected. Most national
schemes reduce to this. The UK NHS number is weights 10 down to 2
then 1, modulus 11.
modulus · integer
weights and modulus define a weighted check: the digits, left to
right, times the weights in order must sum to a multiple of
Modulus. The check digit’s own weight is in the list, so a value
with a different number of digits is rejected. Most national
schemes reduce to this. The UK NHS number is weights 10 down to 2
then 1, modulus 11.
prefix · string
prefix and sum_len define a crc32_base62 check, the shape an
API-key format uses: a literal prefix, a body, and sum_len trailing
characters carrying the CRC-32 of the body text in base 62 (0-9,
a-z, A-Z). A deployment that mints its own keys this way can hold
a recognizer for them to the checksum instead of the shape. The
checksum tells a real key from a documentation placeholder.
sum_len · integer
prefix and sum_len define a crc32_base62 check, the shape an
API-key format uses: a literal prefix, a body, and sum_len trailing
characters carrying the CRC-32 of the body text in base 62 (0-9,
a-z, A-Z). A deployment that mints its own keys this way can hold
a recognizer for them to the checksum instead of the shape. The
checksum tells a real key from a documentation placeholder.
guardrails.detectors[].custom_recognizers[].examples[]
Section titled “guardrails.detectors[].custom_recognizers[].examples[]”One text a [CustomRecognizer] is held to at load.
text · string
text is scanned with this recognizer alone.
finds · list of string
finds lists the values the scan must produce, in order of
position. Empty means the text must yield nothing. That is how a
value with a bad checksum, or a near miss, is pinned as such.
guardrails.detectors[].remote
Section titled “guardrails.detectors[].remote”Points a detector at an analyzer running somewhere else.
The wire protocol is presidio-analyzer’s /analyze, chosen rather
than invented: a Presidio deployment is a valid pistra detector with
no adapter at all, and anything else needs only to answer the same
shape. That is also why [Annotation] was built as a RecognizerResult
superset, and this edge is the payoff.
One server-side setting is worth naming here because it changes what our spans mean and cannot be set from this side. A Presidio analyzer running a transformers NlpEngine re-projects its model’s span onto spaCy tokens with Doc.char_span, and the default alignment_mode “expand” takes every token the span merely touches, whole. Presidio pays nothing for that, since it replaces the span wholesale. We pay in bytes, because ours drive a byte-exact rewrite of the segment. Configure the analyzer with alignment_mode “strict”, where a span that does not align is dropped rather than widened: losing a finding is the better failure for a gateway that forwards everything it does not act on. Where that is not possible, the transport trims whitespace off every span it is handed, and a confirm detector with [DetectorConfig.NarrowSpans] set can narrow the rest.
endpoint · string
endpoint is the analyzer’s base URL. Required.
path · string
path is the analyze route on that base. Default “/analyze”.
language · string
language is Presidio’s language field. Default “en”.
headers · map of string
headers are sent with every call. This is the place for an API
key.
concurrency · integer
concurrency bounds in-flight calls per request. Presidio analyzes
one text per call and a request has many segments, so this is the
difference between one round trip and a serial chain of them.
Default 4.
category · string
category namespaces the annotations this analyzer produces, since
Presidio’s entity vocabulary carries no namespace of its own.
Default “pii”.
offsets · string
offsets declares what the server’s start/end index into when it
does not say so itself: “chars” (the default: Presidio’s offsets
are Python str indices, i.e. code points), “bytes”, or “utf16” for
a server written against a UTF-16 runtime such as JavaScript, Java
or .NET. A server that stamps
recognition_metadata[“pistra.offsets”] overrides this per
annotation.
Getting it wrong is not a near miss. The units agree across the whole Basic Multilingual Plane and diverge by one per character outside it, so the wrong answer here works until the first emoji and then rewrites at the wrong byte. See [OffsetsUTF16].
guardrails.detectors[].nlp
Section titled “guardrails.detectors[].nlp”Configures the “nlp” detector: a transformer token classifier run in this process, through the tokenizer shim and ONNX Runtime.
The remote detector’s doc says a transformer NER model wants a GPU and gigabytes of weights and a gateway wants neither, and that is still true of the models it was written about. This exists for the other shape: a distilled, quantized token classifier of tens of megabytes, where the network hop and the second deployment cost more than the inference does. Which of the two a deployment has is a question about the model, not about the gateway, so both transports stay.
Both native libraries are dlopened. Absent either, building this detector fails at startup with an error naming the missing one. There is never a silent downgrade to a different tier, for the reason Prefilter states at length.
ref · string
ref names a model the host resolves into the paths below, instead
of naming those paths directly. It is deliberately opaque here:
this module has no view of where artifacts come from, and the
gateway’s config package knows how to fetch and verify them. A
Ref that survives to Compile is an error rather than a
default, because the alternative is a detector quietly running
against whichever files happened to be on disk.
remote · object
remote runs the graph on another machine over the KServe v2
inference protocol, instead of loading it here. Set it and model
must be empty. tokenizer is still required, and that asymmetry
is the shape of this mode.
A GPU is worth having on one box and not on every gateway replica,
while the offsets deciding which bytes get redacted are worth
keeping where the bytes are. So the split is: token ids out,
logits back, and every piece of span arithmetic stays here. It is
the opposite trade from the remote detector type, which sends
text to someone else’s analyzer and inherits its idea of where an
entity starts.
model · string
model is the path to the exported .onnx file. Set directly, or
filled in by the host from ref. Must be empty when remote is set.
tokenizer · string
tokenizer is the path to the matching tokenizer.json. It must be
the one the model was exported with: a tokenizer that segments
differently produces ids the model was never trained on, and
nothing downstream can tell.
labels · list of string
labels is the model’s id2label in index order. Empty reads
id2label from labels_path, or from a config.json beside the model,
as an optimum-cli export onnx directory contains.
labels_path · string
labels_path is an explicit HuggingFace config.json to read id2label
from. It exists because “beside the model” stops being true the
moment artifacts are content-addressed: each file then lives under
its own digest, in one flat directory, with no adjacency left to
read anything from.
entity_map · map of string
entity_map renames the model’s labels to the entity vocabulary
policy is written against, such as a model saying PER where
Presidio says PERSON. A label with no entry keeps its own name.
Empty uses [DefaultNLPEntityMap].
aggregation · string
aggregation is how a word’s subword scores combine: none, simple,
first, average or max. Empty means max, which Presidio also
configures its own transformers recognizer with.
It is worth choosing rather than inheriting, because what counts as one “word” comes from the tokenizer and the strategies disagree about words whose subwords disagree. One disagreement the decoder settles itself: a SentencePiece tokenizer does not split trailing punctuation off, so “Montréal.” or “الشمري?” is one word of two tokens, the name and the full stop labelled nothing, confidently. Under “max” the full stop would decide the word and the name be lost. Under “first” it would survive with the full stop inside its span. So punctuation at either edge of a word is made a word of its own before any strategy runs, as a WordPiece tokenizer would have done. The choice here is about the subwords of the name itself. Both behaviours otherwise match transformers’ aggregate_word.
strip_marks · string
strip_marks removes a script’s combining marks from the text before
the tokenizer sees it, and lands what the model finds back on the
text as sent. “arabic” (the default) drops the harakat, the
combining hamza and maddah, the Quranic annotation marks and the
tatweel; “none” hands the model the bytes as written.
It is on by default because nearly every model was trained on text without them, and vocalized Arabic is rare outside scripture and teaching material. A name written مُحَمَّد is a different token sequence from the محمد the model learned, seen through subwords it has no reason to label. The fold is a fold and not a normalization: letters are never changed (alef forms, taa marbuta), since the model learned those distinctions and erasing them would move its input off what it was trained on. A span keeps the marks of its own last letter and excludes the marks of the letter before it, because a mark belongs to the letter it sits on. See [fold.ArabicMarks].
max_tokens · integer
max_tokens is the model’s maximum sequence length, special tokens
included. Default 512. Text longer than it is split into
overlapping windows rather than truncated.
stride · integer
stride is how many tokens consecutive windows share. Default 14,
Presidio’s value. An entity at a window edge is seen by the model
without context on one side, so the overlap gives every position
at least one window where it is not at an edge.
threads · integer
threads is ONNX Runtime’s intra-op thread count. Default 1, and
deliberately not “however many cores there are”: this runs inside
a process already serving many requests concurrently, where a
detector fanning out to every core turns one request into a stall
for all the others.
library · string
library and tokenizer_library override where the native libraries
are loaded from. Empty tries each package’s default paths.
tokenizer_library · string
library and tokenizer_library override where the native libraries
are loaded from. Empty tries each package’s default paths.
category · string
category is the annotation namespace for what this detector finds.
Empty means pii.
guardrails.detectors[].llm
Section titled “guardrails.detectors[].llm”Points the llm detector at a chat model and tells it what to ask. The detector is a judge: it hands the model a policy written in prose, a fixed list of labels, and one segment of text, and reads back which labels the model says apply. The finding is a verdict over the whole segment ([SpanSegment]), because the model was asked about the segment, not where in it.
It is the door every judge-shaped guardrail walks through: a prompt-injection check, an off-topic check for a bot that may only discuss one product, a content-safety model such as Llama Guard or a NemoGuard NIM (both are chat models behind an OpenAI-shaped endpoint, so both are providers here), a “does this answer contradict the context” check on responses. The prompt and the labels vary between them. Where the model is, what it costs, when it runs and what policy does with the answer do not vary, and belong to the engine as for every other detector type.
Every call is a model call, and a model call is hundreds of
milliseconds and a bill. Three things keep that in proportion. The
delta-scan cache means a segment is judged once and remembered by its
text, so a conversation’s history costs nothing on the turns after the
one that introduced it. schedule: overlap hides the latency behind
the provider’s own. schedule: async hides it entirely at the price
of enforcing a turn late. And mode: shadow is how a prompt earns the
right to deny anything: the verdicts are recorded and ignored until
the false-positive rate has been read off the audit trail.
A judge cannot locate. It says the segment is an injection attempt.
It does not say which bytes, so redact on its verdict replaces the
whole segment, which is rarely what anyone wants. deny and
annotate are the actions written for it.
And a judge can be argued with. The text it reads is exactly the text an attacker wrote, and a model that follows instructions in its context can be told to answer “no verdicts”. The contract this detector appends tells the model the text is data, which helps and does not settle it. A small classifier with no instruction-following to exploit is the stronger injection detector, and this one is the broader, cheaper-to-write, easier-to-argue-with one.
provider · string
provider names a configured provider the judge’s calls go through:
its endpoint, its credential, its catalog quirks. Required. The
host resolves it. A library consumer sets Client directly.
model · string
model is the name the provider runs the judge under. Required.
labels · list of string
labels is the verdict vocabulary: the entity types this detector
can emit, and the only answers the model is allowed to give. At
least one, conventionally upper case (INJECTION, OFF_TOPIC). An
answer naming a label outside this list is a detector error, not
a finding: a judge that invents labels is not following the
contract, and on_error is where that gets decided.
prompt · string
prompt is the policy, in the operator’s words: what each label
means and when it applies. Required. The detector appends the
output contract (the label list, the JSON shape of the answer,
that the text is data), so the prompt has only to say what the
labels mean.
category · string
category is the annotation namespace the verdicts land in:
“injection” (default), “topic”, or any other. Policy reads a
verdict as <category>/<LABEL>.
format · string
format is the shape the model answers in, and with it how the
policy reaches the model. “json” (the default) is this detector’s
own contract, appended to the prompt: a closed label list and one
JSON object back, which any chat model that follows instructions
honours. The other two are for guard models fine-tuned to a fixed
prompt and a fixed answer, which ignore a contract and, some of
them, the system message:
- “nemotron”: NVIDIA’s Nemotron Content Safety family. The prompt goes to the server as the template’s custom_policy, with the labels appended as its safety categories. The answer read is “User Safety: unsafe” and “Safety Categories: …”, the categories mapped onto the labels.
- “llama_guard”: Meta’s Llama Guard 3 and 4. The labels go as
the template’s categories S1…Sn, each described by the prompt.
The answer read is “safe”, or “unsafe” and the codes, mapped
back onto the labels. The template has no room for context,
so
contextis refused with it.
Both need a server that renders chat_template_kwargs, such as
vLLM, llama.cpp or a NIM. A fixed-format answer carries no score,
so a verdict is 1.0 and threshold has nothing to cut.
max_bytes · integer
max_bytes is the most text one call carries. A longer segment is
judged in overlapping windows of this size and the verdicts
unioned, so nothing is silently left unread past a cut-off.
Default 16384, and never below 256.
max_tokens · integer
max_tokens bounds the model’s answer. A verdict is a line of JSON.
The default of 256 leaves room for the reasons. An answer the
model could not finish within it is an error, never a verdict.
concurrency · integer
concurrency bounds in-flight calls per request. Default 4.
context · list of string
context lists segment kinds of the same request shown to the judge
beside the text it judges, as a CONTEXT block: [system] gives it
the assistant’s own instructions, so “off topic” means off the
topic the application set and “overrides the instructions” names
instructions it has read. Empty means the judge sees the text
alone.
Opt-in for a reason that is not cost. The system prompt is often the most sensitive text in a request (product logic, what is not yet announced), and this sends it to whichever provider the judge lives on, once per new segment. Name the kinds knowing that. A judge on the same provider the traffic already goes to adds no new reader.
A verdict is remembered under the context it was reached in: the
same user turn under a different system prompt is judged again.
The kinds come from the same pass, since on a response inspection
the request’s segments are not there to be read. The header and
query kinds are refused. The point of those kinds being opt-in is
that authorization: Bearer … never leaves the process by
accident.
guardrails.detectors[].classify
Section titled “guardrails.detectors[].classify”Configures the classify detector: a
sequence-classification model, which reads a whole input and answers
with one row of scores. That is the shape of Prompt Guard,
ProtectAI’s injection DeBERTa, the moderation checkpoints, any
AutoModelForSequenceClassification export. Where the nlp
detector reads a label per token and lands spans, this one reads a
label per segment and lands a verdict over it, [SpanSegment], in the
category named (injection by default).
Against the llm judge it is the narrower and the stronger tool for
the question it was trained on: no instruction-following to exploit,
a few milliseconds on a CPU, calibrated scores, and a cost that does
not scale with the answer. It cannot take a policy in prose. Its
labels are the checkpoint’s, and a question the checkpoint was not
trained on is the judge’s.
Loading is the nlp detector’s: the same tokenizer binding, the same
graph in this process or on a KServe server, the same ref to a
declared model, the same windowing over long text (scored window by
window, the best score any window gave a label kept, as the
published guidance for these checkpoints says). The fields that
repeat NLPConfig’s mean what they mean there.
ref · string
ref names a declared model. The host fills in model, tokenizer
and labels_path from it. See [NLPConfig.Ref].
remote · object
remote runs the graph on a model server over the KServe v2
protocol. See [NLPRemoteConfig].
model · string
model is the ONNX graph. Tokenizer is its tokenizer.json.
tokenizer · string
model is the ONNX graph. Tokenizer is its tokenizer.json.
labels · list of string
labels is id2label in index order. Empty reads it from labels_path
or the config.json beside the model, which also says whether the
labels compete (see activation).
labels_path · string
labels is id2label in index order. Empty reads it from labels_path
or the config.json beside the model, which also says whether the
labels compete (see activation).
negative · list of string
negative lists the labels that mean “nothing found”: the class a
classifier answers with when the text is fine. They are never
findings and never in the vocabulary, and a name here that is not
one of the model’s labels is refused. Unset means the usual names
under softmax (SAFE, BENIGN, OK, NONE, NEUTRAL, CLEAN, NORMAL,
LABEL_0, matched case-insensitively), and none under sigmoid,
where every label is its own yes/no and a clean class does not
exist. A checkpoint that names its clean class something else, or
a multi-label one that does have a “nothing applies” label, names
it here. A list given replaces the defaults entirely.
label_map · map of string
label_map renames the model’s labels to the vocabulary policy is
written against, such as LABEL_1 to INJECTION. A label with no
entry keeps its own name.
activation · string
activation is how a row of logits becomes scores: “softmax” (the
classes compete, and the verdict is the best one) or “sigmoid”
(each class is its own yes/no, and several may apply). Empty reads
problem_type from the model’s config.json and falls back to
softmax, which nearly every classifier is.
strip_marks · string
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of the window here rather than 14 tokens: a verdict over a window
needs the window to carry context, not just the token at its edge.
max_tokens · integer
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of the window here rather than 14 tokens: a verdict over a window
needs the window to carry context, not just the token at its edge.
stride · integer
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of the window here rather than 14 tokens: a verdict over a window
needs the window to carry context, not just the token at its edge.
threads · integer
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of the window here rather than 14 tokens: a verdict over a window
needs the window to carry context, not just the token at its edge.
library · string
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of the window here rather than 14 tokens: a verdict over a window
needs the window to carry context, not just the token at its edge.
tokenizer_library · string
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of the window here rather than 14 tokens: a verdict over a window
needs the window to carry context, not just the token at its edge.
category · string
category is the annotation namespace the verdicts land in:
“injection” (default), “topic”, or any other.
guardrails.detectors[].classify.remote
Section titled “guardrails.detectors[].classify.remote”Also configured at guardrails.detectors[].embed.remote, guardrails.detectors[].nlp.remote.
Points the nlp detector at a model served elsewhere
over the KServe v2 inference protocol, which Triton, KServe, Seldon
and OpenVINO model Server all speak.
endpoint · string
endpoint is the server’s base URL, e.g. https://triton.internal:8000.
model · string
model is the name the server serves the model under.
version · string
version pins a model version. Empty lets the server apply its own
default-version policy.
headers · map of string
headers are sent on every request, for authentication.
timeout · string
timeout bounds one inference call, as a duration string. Default
30s.
guardrails.detectors[].embed
Section titled “guardrails.detectors[].embed”Configures the embed detector: a sentence-embedding model
and, for each topic the operator names, a handful of example texts.
A segment is embedded the same way and its cosine to every example
taken. The topic whose nearest example it sits closest to is
reported as a verdict over the segment, [SpanSegment], in the
category named (topic by default). It is reported when it wins
clearly enough against the other topics and against background,
the operator’s examples of ordinary traffic.
It is the third shape of classification beside classify and llm,
and the one with no training and no bill. The labels are whatever
the operator writes examples for, in whatever language the model
embeds, and the cost is one forward pass per window however many
topics there are. It trades away calibration. A classifier’s score
is a probability with a boundary it was trained to. A cosine is not,
and the cosines of the embedding families are compressed: unrelated
text sits well above zero, and a topic wins by hundredths. So the
score is the nearest topic’s lead over the background on the
temperature’s scale, a logistic of the difference. One half is a
topic exactly as near as ordinary traffic, the boundary between “on
some topic” and “none of these”, and the default threshold. How fast
the score rises past it is the temperature’s doing, not the world’s.
The threshold that fits a deployment is read off labelled traffic
with pistra guardrails eval -sweep.
Loading is the nlp detector’s: the same tokenizer binding, graph in
this process or on a KServe server, ref to a declared model,
windowing over long text with the best window kept. Added to that
are the two things an embedding model brings: how its hidden states
are pooled, and the instruction prefix some families were trained
with.
ref · string
ref, remote, model and tokenizer are [NLPConfig]’s.
remote · object
ref, remote, model and tokenizer are [NLPConfig]’s.
model · string
ref, remote, model and tokenizer are [NLPConfig]’s.
tokenizer · string
ref, remote, model and tokenizer are [NLPConfig]’s.
topics · list of object
topics are the labels, each with the examples that define it. A
topic’s cosine to a text is its nearest example’s, so a topic
with several distinct phrasings is a topic with several
examples, not a topic with an averaged one. Five to twenty
examples per topic, in the languages and register the traffic
uses, is the usual amount. The names are the vocabulary a rule
tests, topic/billing.
background · list of string
background is what none of the topics looks like: examples of the
application’s ordinary traffic, and the alternative every topic
is scored against. Required, because without it “none of these”
is not an answer the detector can give. A greeting is nearest to
some topic, and with nothing else to be nearer to it would be
reported as that topic. A dozen examples of the traffic a topic
must not be confused with is the usual amount.
temperature · number
temperature is the scale a topic’s lead over the background is
read at. Empty means 0.05, where a lead of 0.04 in cosine scores
0.69 and one of 0.08 scores 0.83. Lower is sharper: the same
lead scores nearer 1, and a threshold above one half selects
less. Higher is flatter. It never moves the one-half boundary.
Change it only with the sweep open.
pooling · string
pooling is how the model’s hidden states become one vector: mean
over the attended tokens (the default; e5, bge-small, MiniLM) or
cls, the first token’s state (bge-m3, the original bge). An
export that pools inside the graph and emits one vector is taken
as it is under either. The wrong choice does not fail. It
silently produces a worse space, so it is worth reading off the
model card.
prefix · string
prefix is put before every text embedded, examples and segments
alike. It is tokenized once and laid out as ids at the start of
every window, so a long text carries it in each window and the
offsets stay the segment’s own. The e5 family was trained with
“query: “ in front of short texts and degrades measurably without
it. Most other families want nothing.
strip_marks · string
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of max_tokens.
max_tokens · integer
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of max_tokens.
stride · integer
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of max_tokens.
threads · integer
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of max_tokens.
library · string
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of max_tokens.
tokenizer_library · string
strip_marks, max_tokens, stride, threads, library and
tokenizer_library are [NLPConfig]’s. Stride defaults to an eighth
of max_tokens.
category · string
category is the verdict’s category. Empty means topic.
guardrails.detectors[].embed.topics[]
Section titled “guardrails.detectors[].embed.topics[]”One label and the texts that define it.
name · string
name is the label, one word with no slash: what types carries
under the category, and what a rule tests.
examples · list of string
examples are texts that belong to the topic.
guardrails.rules[]
Section titled “guardrails.rules[]”One guardrail policy rule.
name · string
name appears in the refusal, the decision log and the
pistra_guardrail_decisions_total metric, so it is how an
operator finds which rule acted.
when · string
when is a CEL condition over annotations and request context.
Empty means the rule fires whenever a detector found anything,
so a rule that acts on everything its detectors report needs no
condition. A rule about the route or the key rather than about
what was found spells that as its condition, and one that must
fire on every request belongs in access_rules.
action · string
action is “deny”, “redact” or “annotate”.
requires · list of string
requires names the detectors this rule’s decision depends on.
It makes profiles[].guardrails.detectors safe. A rule is
written against the findings of an ensemble, so a profile that
selects a subset can silently stop this rule from ever matching:
fewer annotations, and with agreement_boost lower scores on the
ones that remain. Nothing static can work that out: a condition
naming no entity type at all (annotations.size() > 0) depends on
every detector, and a score threshold depends on every detector
that can corroborate the type it reads, not just the one that
finds it.
So the rule says, and it says it here rather than the detector saying “nobody needs me”, because this is where the dependency changes. A claim about the rules kept in the detector block would go stale the first time a rule was added and nothing made anyone revisit it.
Omitting the field means the whole ensemble. A deployment that annotates nothing permits no selection at all. An explicitly empty list means no detector: a rule about the route or the key and not about what was found.
select · string
select is a CEL predicate over a single annotation a, choosing
which annotations a redact rule rewrites. Empty means every
annotation that survived resolution.
controls · list of string
controls are the governance controls this rule enforces, as
identifiers an external system assigned: eu-ai-act:art-50,
nist-ai-rmf:govern-1.2, gopal:international.eu_ai_act.v1.transparency.
The gateway does not interpret them. It carries them onto every
audit record this rule decides, so a governance platform reading
the trail can map a decision to its control without joining
against the configuration that was in force at the time.
Opaque strings on purpose: which framework, and which taxonomy within it, is the platform’s to choose, and a vocabulary fixed here would be the one thing in this list that went stale on someone else’s schedule. The only checks are shape: non-empty, no whitespace, no duplicates.
kinds · list of string
kinds restricts a redact rule to annotations on segments of the
given kinds. Empty means every writable kind, as every rule
written before this field existed already did.
It narrows. It cannot widen. Naming a read-only kind (header,
query) here is refused at Compile with the reason, and that
refusal is the point of the field. select is a CEL predicate
whose intent cannot be read statically, so “redact this header”
has exactly one spelling that a config can be told off for, and
every other spelling is simply inert (see [Engine.selected]).
operator · string
operator is how a redact rule rewrites a span: “replace”
(default), “redact”, “mask”, “hash”, “placeholder”, “pseudonym” or
“fpe”. The first four are presidio-anonymizer’s, names and
semantics alike. The last three keep distinct values distinct,
which none of the first four do. See the operator constants.
replacement · string
replacement is the replace operator’s text. Empty means
“<ENTITY_TYPE>”.
mask_char · string
mask_char is the mask operator’s fill character (default “*”).
mask_chars · integer
mask_chars is how many characters the mask operator covers.
Zero means the whole span.
from_end · bool
from_end masks from the end of the span instead of the start.
message · string
message is the refusal text of a deny rule.
on_error · string
on_error is what a condition that cannot be evaluated means:
“fail_closed” (default) refuses the request the way a fail_closed
detector’s failure does, “fail_open” skips the rule and records
the error in Result.Errors. A condition fails to evaluate in two
ways: when it reads a key that is not there, such as
key.metadata.team on a key with no team, or when it exceeds its
cost budget over a body the client sized. Guard the first with
has(). The default is closed because the
second case is the client’s to cause, and a deny rule that a
large enough argument list could switch off is not a deny rule.
guardrails.stream
Section titled “guardrails.stream”Governs how a streamed response is inspected. The awkward truth this section exists to make explicit: bytes already sent to a client cannot be recalled, so anything the guard might want to rewrite has to be held back first, and holding back is latency.
policy · string
policy is “window” (default), which holds back only as much text
as a match could span and redacts inside that window, or
“buffer”, which holds the whole response, decides once, and then
emits. Buffer is the honest choice for policy that must see whole
text. It costs the entire streaming experience.
max_hold_bytes · integer
max_hold_bytes caps the window when a detector’s match length is
unbounded (an email local part, a URL). Default 512. Reaching it
means text is released without ever being proven safe, so it is
counted, not silent.
on_hold_overflow · string
on_hold_overflow is what happens when the enabled detectors have no
finite bound, so the cap is a guess rather than a proof: “emit”
(default) streams anyway, counts every unproven release, and
rewrites whatever tail of an over-long match is still held;
“buffer” refuses to guess and holds the whole response instead.
The choice is a miss or the latency. Policy picks it, and the
gateway does not decide quietly.
live_horizon · bool
live_horizon narrows the commit horizon to what the text in hand
could still be growing into, rather than holding the full span a
match might ever reach. Default on.
It only ever shortens, and only on evidence: a tail of prose cannot be the start of a card number, so none of it is held, while a tail of digits is held to the full horizon. On running prose the held text falls from the horizon to a byte or two.
The reason it can be turned off: what is held becomes a function of what the text looks like. An observer who can see frame timing but not frame contents learns a little about whether the response resembles an identifier. That is a weak signal next to what frame sizes already leak, and the client holds the plaintext regardless. A deployment that would rather release on a fixed cadence can still set this false and pay the latency.
scan_chunk_bytes · integer
scan_chunk_bytes is how much new text has to arrive before the
detectors run again.
A scan cannot start where the last one stopped: a match may begin inside the held tail and only complete now, so every pass re-reads the horizon behind it. Scanning once per frame therefore re-reads that tail once per token. For a 512-byte horizon and a six-byte token, every byte is scanned around ninety times, which is most of what guarding a stream costs.
Waiting for a chunk amortizes it, and buys nothing back except delay: text is already held for the horizon, and a scan that runs sooner cannot release anything sooner. The cost is that a release waits for up to one chunk of further text, so the default is the horizon itself or 64 bytes, whichever is smaller. At worst it doubles a delay the horizon already imposes. Under the “buffer” policy nothing is released early anyway, so the full default applies. Set 1 to scan on every frame.
guardrails.response
Section titled “guardrails.response”Governs inspection of a unary response body. A unary response is the easy case, since nothing has been sent and the gateway can still change its mind about the whole thing. The question left is how much of a body it is willing to hold in memory to decide.
max_body_bytes · integer
max_body_bytes bounds what the gateway buffers in order to inspect a
response. Default 1 MiB, which is far past any chat completion and
well short of anything that threatens the process.
on_oversize · string
on_oversize is what happens to a body past that bound, which was
never examined: “allow” (default) forwards it and counts the miss,
“deny” refuses it. Unexamined content either reaches the client or
it does not. Policy picks, and either way it is counted rather
than quietly assumed safe.
guardrails.overlap
Section titled “guardrails.overlap”Governs the one thing an overlap schedule cannot do.
An overlap detector runs while the upstream call is in flight, so its verdict arrives after the request was forwarded and before the response is released. That is enough to refuse the response and enough to record what was seen. It is not enough to rewrite the request: those bytes are with the provider, and masking a copy of them afterwards would be theatre.
So a redact rule that fires on an overlap pass has no honest way to do what it says, and the gateway does not get to pick quietly which half of the operator’s intent to keep: forward content policy wanted changed, or refuse a request policy wanted allowed.
on_redact · string
on_redact is what a redact rule means once the bytes are gone:
“deny” (default) refuses the response, treating the rule as a
statement that this content must not pass unaltered; “allow”
forwards the response and records the finding, treating it as a
statement about presentation. Either way it is counted, and the
decision belongs to policy rather than to timing.
guardrails.async
Section titled “guardrails.async”Bounds work no request is waiting for.
An async detector is decoupled from the request in both directions: the request does not wait for it, and it does not stop when the request ends. That is what makes it free, and it is also why an async detector is the one place the gateway could accumulate unbounded work. So the bound is explicit and what happens past it is counted.
max_in_flight · integer
max_in_flight caps concurrent async scans. Past it a scan is
dropped and counted rather than queued: queueing work whose whole
value is being current just makes it stale. Default 4.
timeout · string
timeout bounds one async scan, a Go duration. It exists because
the request context cannot be used here. That context is
cancelled the moment the response is written, which for an async
detector is immediately. Default 30s.
models
Section titled “models”Declares the model artifacts detectors may reference.
Naming a model here rather than pointing a detector at a path gets
the artifacts verified: every file carries a digest, and the
digest is checked on the way in. A manifest either lists its files
with their digests, or names a HuggingFace repository at a commit and
takes the Hub’s digests. See modelstore.HFSource. A detector
configured with model: and tokenizer: paths directly still works
and is still the right answer for a file an operator placed
themselves. It just makes the operator, rather than this config, the
thing that vouches for it.
models · list of object
models are the declared manifests.
models.models[]
Section titled “models.models[]”One named set of files: either listed here with their digests, or named as a HuggingFace repository at a commit for the store to resolve into that list.
name · string
name is how a detector configuration refers to this model.
revision · string
revision distinguishes two publications of the same name. It is
documentation, not identity: the digests are identity. A resolved
HF model carries its commit here.
source · string
source, quantization and license record where these bytes came
from. An exported and quantized model is a DERIVED file, and a
digest alone does not let anyone reproduce it or know what they
are permitted to do with it.
quantization · string
source, quantization and license record where these bytes came
from. An exported and quantized model is a DERIVED file, and a
digest alone does not let anyone reproduce it or know what they
are permitted to do with it.
license · string
source, quantization and license record where these bytes came
from. An exported and quantized model is a DERIVED file, and a
digest alone does not let anyone reproduce it or know what they
are permitted to do with it.
hf · object
hf names the model on the HuggingFace Hub instead of listing its
files. Set it and files must be empty; [Store.Resolve] fills them.
files · list of object
hf names the model on the HuggingFace Hub instead of listing its
files. Set it and files must be empty; [Store.Resolve] fills them.
models.models[].hf
Section titled “models.models[].hf”Names a model on the HuggingFace Hub at one commit.
The commit is what makes this a manifest rather than a pointer. A
branch would let the artifact change under a configuration that did
not: under a security control, which is the one place a mutable
reference is least welcome, so revision is the commit and nothing
else, and pistra models pin is what turns a branch into one.
The Hub’s tree listing gives each file’s identity: a sha256 for every LFS file, which the graph and any large tokenizer are, and a git blob id for the rest. The store verifies every byte against those, so the commit pins the files and the Hub’s own digests check them, and no trust is placed in the host serving the bytes.
repo · string
repo is the repository, “owner/name”.
revision · string
revision is the commit, as 40 hex characters. Required.
model · string
model is the path of the ONNX graph within the repository.
Default [DefaultHFModel]. A detector whose graph runs on an
inference server never asks for it, so a repository with no
export at all (the original PyTorch checkpoint, say) serves
that detector’s tokenizer and labels fine.
endpoint · string
endpoint is the Hub or a mirror of it. Default [DefaultHFEndpoint].
token · string
token authenticates against a gated repository, sent as a bearer.
Normally ${secret:HF_TOKEN}; a literal is refused where the
document travels, like every credential.
models.models[].files[]
Section titled “models.models[].files[]”One artifact: where to get it, how big it should be, and what it must hash to.
role · string
role is what this file is to the detector that loads it.
url · string
url is where the bytes live. Hosting only: see the package doc.
sha256 · string
sha256 is the digest the fetched bytes must have, hex-encoded.
This is the only reason to believe anything the URL returns.
size · integer
size is the file’s exact length in bytes. It bounds the download
before the digest can, so a host cannot fill the disk with bytes
that were only ever going to be rejected.