Keep it. A gateway you already run terminates TLS, holds every route in the company, and has a rota of people who get paged when it is down. Nothing in this series is a reason to replace it, and a request for keys on the model route and a spend cap per team is answered by that gateway’s own AI plugin. Kong has one, Apigee has one, Tyk and Traefik have one, and the Envoy family has a whole project. Turn it on.
Redacting card numbers on the model route, in both directions, and showing that it happened, is the request the plugin cannot take alone.
Routes and headers
A gateway’s vocabulary is the route: a path, a set of headers, a backend, and policies attached to that route. It is a good vocabulary. It has carried every API for fifteen years, and it lets a gateway rate-limit, authenticate and observe traffic it knows nothing about. The same vocabulary leaves it nothing to say about what is inside the body of a model call.
A prompt is not a route. A tool call is a JSON object inside a JSON array inside a body, and a card number is sixteen digits somewhere in that. None of those has a name in the gateway’s language. So when a gateway grows an AI plugin, the plugin does what the vocabulary allows. It takes the body, hands it to something that can read it, and turns the answer into a route decision. Any gateway’s list of AI plugins is the same list twice, a plugin per detector vendor, each one a wrapper that calls out. The box at the end has one such list, dated.
This is the shape of the thing rather than a fault in it, and it puts the AI plugin where post 9 puts every detector: called out to with a string.
On the way in, the plugin can call a detector and get a verdict back before it forwards, which is enough for a refusal. On the way out, a model streams, and a redaction has to land before the frame carrying the digits is forwarded, which means holding the stream as far as a match could reach. A call-out per frame does not do that, and buffering the whole response to make it possible defeats streaming. As for showing that it happened, the gateway has an access log in the gateway’s vocabulary: a route, a status, a latency. It has no signed record of which rule rewrote which span, having never known there was a span.
So keep the gateway and add one thing beside it. The inspector sits on that route, and keys and budgets stay where routing happens.
Two kinds of gateway
The vendors in this position split on one line, and the split decides how that inspector is deployed.
The incumbents, Kong, Apigee, Tyk and Traefik, sell “you already run us, and AI is another protocol”. Their AI layer is a plugin on the route, priced the way the gateway is priced, and the extension point is whatever their plugin system is.
The Envoy family, Envoy AI Gateway, kgateway and agentgateway, sell the Kubernetes-native standard, with support from Tetrate or Solo. Their extension point is Envoy’s own: an external processor the gateway streams each request and response through, which can rewrite the body in flight and return a verdict, on a route the gateway picks. The inspector uses that mechanism, so the inspector shape needs an Envoy-family gateway. A gateway that is not one is chained instead, by URL, in either order.
Azure’s API Management is the same product seen from the incumbent side, and it lives in the next post with the rest of the cloud’s own services. The inference routers, llm-d and the Gateway API’s InferencePool, are not on this map. They sit behind the gateway, choosing which replica of a self-hosted model gets the call, and pistra treats them as a provider.
Who should use the plugin anyway
Anyone who needs keys, quotas and a model allowlist. That is most requests, and the plugin was built for them. The gateway already holds the identity, the routes and the rota, and putting a second thing on the path to do what the first already does costs more than it returns.
The line falls at the body. The first request that names something inside it, a card number, a tool call, a prompt, is the one the route vocabulary cannot carry, and a plugin does not change the vocabulary.
What pistra does at this box
pistra is not a gateway class and does not program an Envoy. Beside a gateway you keep, it runs in its inspector role. The gateway streams the model route’s requests and responses through it, and it does three of the four things there. It redacts, in flight, with the stream held as far as the enabled recognizers could reach. It decides on content, which the gateway then enforces as a route decision. And it records, signed, which rule did what. The inspector refuses to be given providers or credentials, and it cannot route. When it is down, the route fails closed by default, because a guardrail that fails open leaves only a log line saying it was there.
Keys, budgets and metering stay where routing happens, in the gateway, or in pistra as the front door where there is none.
As of 28 August 2026
Kong's AI Gateway plugin list, as an illustration of the shape: AI Proxy, AI Prompt Guard (allow and deny lists), AI Semantic Prompt Guard, AI PII Sanitizer, and a wrapper each for Azure Content Safety, AWS Guardrails, GCP Model Armor and Lakera Guard, plus AI Custom Guardrail for the next vendor. developer.konghq.com/plugins. Envoy AI Gateway's capabilities page lists a unified API with vendor-specific fields, failover, token-aware rate limiting, upstream auth and an MCP gateway. aigateway.envoyproxy.io.
pistra's inspector wirings, per the docs of the same date: Envoy Gateway, kgateway and agentgateway are proven by the chart's end-to-end test on a kind cluster; Istio and GKE are documented wirings not covered by that test. Chaining a non-Envoy gateway by URL in either order is a consequence of pistra speaking the OpenAI and Anthropic dialects on both sides, not a tested integration with any named product.