Performance

Every figure here is measured and reproducible on your own hardware. The commands are at the bottom of the page.

Apple M4 · Go 1.27.0 · darwin/arm64 · medians of -count 6 at -benchtime 2s · load 2.4 before, 2.8 after · measured 2026-08-29

These numbers were measured on a busy machine and are pessimistic. The load average above is the evidence. Every figure below is inflated by whatever else was running, roughly uniformly, so the table can look internally consistent while being wrong as a description of the gateway. Re- run on an idle machine before quoting any of it.

Every number below is a median with its run-to-run spread, because a single benchmark run is not a measurement. A spread worth distrusting is marked. Treat two rows whose spreads overlap as equal.

End-to-end request

A full client-to-upstream request over loopback HTTP, including the network hop. The baselines are what the same client costs with no gateway at all, and with the standard library’s reverse proxy in the middle. The difference between those two rows is what an extra HTTP hop costs before any logic runs.

	per request	spread
direct to upstream, no proxy	30.6 µs	±2%
stdlib httputil.ReverseProxy	68.0 µs	±1%
pistra passthrough	76.1 µs	±2%
pistra with a model alias	76.6 µs	±1%
pistra translating Anthropic⇄OpenAI	74.9 µs	±1%

Aliasing patches one field with sjson and unary translation runs a full parse and re-serialize, and both land within run-to-run noise of passthrough. Translation costs on streams, not on unary bodies. Fidelity is the argument for passing bytes through, not latency.

direct to upstreamno proxy at all

30.6 µs

stdlib reverse proxyone more HTTP hop, no logic

68.0 µs

pistra passthroughthe same hop, and all of pistra's own work

76.1 µs

Three rows of the table above, to scale. Most of what a gateway costs is the extra hop; pistra's own work is the step past the stdlib proxy.

Internal overhead

Request time minus time blocked on the provider and on the client. This is the quantity usually published as “gateway overhead”, since it excludes the hop. It is measured here by the same accounting that ships in the binary as pistra_overhead_seconds, so it is observable in a real deployment rather than only in a benchmark.

	per request	spread
serial, passthrough	10.2 µs	±3%
at saturation, passthrough	61.5 µs	±10%
sustained rate at saturation	46,642 req/s	±3%

The in-process parallel harness inflates overhead at saturation through its own scheduler queuing. tools/loadtest is the honest number under load: three separate processes, a generator that paces a fixed rate and never adapts to response times.

Streaming

One SSE response of 53 frames, measured end to end. Frames are flushed individually. Buffering would improve these numbers and delay every token.

	per response	spread
direct to upstream, no proxy	31.9 µs	±1%
pistra passthrough	168 µs	±1%
pistra translating Anthropic⇄OpenAI	239 µs	±2%

The governance path

Routing a request without any I/O: what authenticating a virtual key, resolving a provider, evaluating policy and reserving budget cost on their own. Each row adds to the one above it.

	per request	spread
virtual-key auth + provider resolution	652 ns	±7%
plus two CEL access rules	1.61 µs	±2%
plus budget reservation and settle	1.34 µs	±2%
the same, from every core at once	851 ns	±5%
ledger reserve+settle alone	130 ns	±1%
ledger reserve+settle, contended	252 ns	±3%
CEL cost expression	227 ns	±2%

Guardrails

Guardrails are opt-in and priced separately, because a request that inspects content is doing real work rather than moving bytes. The request rows are measured on a 7 KB, 20-turn conversation with every recognizer enabled. The response rows say which recognizers they run, and the streamed-frame row runs the card recognizer alone.

	per request	spread
every segment rescanned (cold cache)	341 µs	±1%
conversation already seen, one new turn	16.8 µs	±1%
conversation seen entirely before	16.2 µs	±2%
the same, rewriting what it found	8.14 µs	±2%
body segmentation alone	14.1 µs	±2%
segmenting one MCP tools/call (400 B)	1.51 µs	±1%
segmenting a unary response	468 ns	±1%
guarding a unary response (1 KB, cold)	2.48 µs	±1%
the same, one recognizer enabled	2.51 µs	±1%
guarding one streamed frame	1.87 µs	±2%

A chat request replays its entire history on every turn, so inspection cost should track new content rather than context length. That is the gap between the first two rows. A response is the opposite case. A fresh completion has never been seen, so the cache cannot help and the number is the cold cost.

Pattern detection

The tier-0 detector on 64 KB of text, with and without the generated DFA prefilter that decides which regions any recognizer could match in. “Clean” is prose with no PII in it, which describes nearly all traffic. “With numbers” is the same prose carrying what business text carries, meaning a year, a sum, two times, a ticket number and a percentage, every one of them a region for the prefilter and none of them PII. “Dense” is the adversarial case where the prefilter has to hand almost everything to the exact matchers.

	throughput	spread
clean prose, prefiltered	19.7 MB/s	±1%
clean prose, every recognizer run in full	0.35 MB/s	±3%
prose with numbers, prefiltered	1.17 MB/s	±2%
PII-dense, prefiltered	0.10 MB/s	±0%
PII-dense, every recognizer run in full	0.09 MB/s	±0%
the prefilter DFA on its own	135 MB/s	±1%

The prefilter is why inline detection is affordable at all, and the clean- prose rows are the ones that describe production traffic. The numbers row is where a prefilter earns or loses its keep. Each number is a candidate region, and the pass costs the regions’ size times the patterns asked about each. Regions run to word boundaries and no further (a padding that once chained a number every twenty bytes into a sentence-long region had this row at the full-scan figure), and a pattern is not asked about a region shorter than its shortest possible match. The dense rows carry the phone recognizer, which is not a pattern but libphonenumber read over every run of digits for each configured region. That is about three quarters of the dense cost with the fourteen default regions, and none of it when entities: leaves PHONE_NUMBER out or phone_regions: names the two or three a deployment actually has. Prose with a year or a time in it does not pay that cost, because a run of digits shorter than a region’s shortest number is never shown to the library.

Byte-level components

The parsers everything above is built on.

	throughput	spread
SSE frame reader	2589 MB/s	±2%
WebSocket decoder, unmasked frames	12582 MB/s	±1%
WebSocket decoder, masked frames	258 MB/s	±1%

Masked frames are byte-wise XOR and far slower, and it does not matter: masking is a client-to-server requirement, and the direction the gateway meters is server-to-client, which is never masked.

Reproducing these numbers

Run this command:

$ go run ./tools/benchdoc -count 6 -benchtime 2s

That rewrites this page in place. It runs only the benchmarks the page quotes. go test ./... -run XXX -bench . runs everything, including the golden benchmarks vendored with the translators.

Load behaviour is a separate harness. tools/loadtest runs the upstream, the gateway and an open-loop generator as three processes, paces a fixed rate regardless of how the gateway responds, and reads the overhead metric out of /metrics. That states offered load, client-observed latency and internal overhead together without an in-process harness distorting all three.

$ go run ./tools/loadtest upstream -listen 127.0.0.1:9101
$ pistra -listen 127.0.0.1:8484 -upstream http://127.0.0.1:9101
$ go run ./tools/loadtest load -url http://127.0.0.1:8484 -rps 5000 -duration 15s \
    -metrics http://127.0.0.1:8484/metrics