Metrics

This is every series the gateway exports: 34 in all, 20 counters, 8 gauges and 6 histograms. The page is generated from the registry a scrape serves, so it lists what this build exports rather than what was true when someone last wrote it down. Each description is the metric’s own help text, verbatim, and it is the same sentence curling /metrics returns.

There is one registry and two ways out of it. /metrics is served on metrics_listen (:9464 by default) and never on the data plane; with the OTEL_* environment set, the same registry is bridged and pushed as OTLP. A collector receives the same series a scrape would, under the names below. See Watch the gateway for the switch, the dashboard and what to alert on.

These names are the gateway’s own. The gen_ai.* OpenTelemetry semantic conventions appear on spans, not here. A dashboard built for the GenAI metric conventions will find none of them.

The label vocabulary is action, budget, cap, channel, code, detector, disposition, ended, entity, from, mode, model, outcome, provider, reason, result, route, rule, side, state, stream, suffrage, to and token_type. Every one of them is a bounded set (an HTTP status, a provider or route named in the configuration, a detector’s entity type) and none is a caller or a payload. There is no key label and no identity label anywhere on this page. A per-key cut is the ledger’s job, a per-caller one is the audit trail’s, and a label a client can mint is unbounded cardinality served to whoever can scrape. The two labels that grow with a deployment rather than with traffic are provider and model.

Contents

• The request path
• Admission
• Routing and delivery
• Pools
• Guardrails
• Metering
• The cluster
• The audit trail

The request path

Every request contributes to these series, whoever sent it. pistra_overhead_seconds is the flagship: total time minus the time blocked on sockets the gateway does not control, which is the only latency that is the gateway’s fault. It is a subtraction of the two wait histograms from the duration, so all four are exported and the subtraction can be checked rather than trusted.

pistra_downstream_wait_seconds · histogram · route, stream

Time blocked writing and flushing to the client.

Buckets: 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60, 120.

pistra_overhead_seconds · histogram · route, stream

Gateway-attributable overhead: total minus upstream and downstream socket waits.

Buckets: 0.00001, 0.000025, 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25.

pistra_request_duration_seconds · histogram · route, stream, code

End-to-end request wall time as seen by the gateway.

Buckets: 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60, 120.

pistra_streams_total · counter · ended

SSE streams by how they ended: done saw the [DONE] marker, truncated did not.

pistra_upstream_wait_seconds · histogram · route, stream

Time blocked on the provider: response headers plus every streamed body read.

Buckets: 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60, 120.

Admission

These count requests stopped before a provider was reached. A refusal here is policy working, so the alert is on the approach rather than the event. pistra_budget_used_ratio above 0.8 is the difference between finding out at 80% and finding out from a 429.

pistra_auth_failures_total · counter · reason

Requests refused before reaching a provider, by bounded reason.

pistra_budget_buckets · gauge · budget, state

Open budget buckets by whether they are at their limit. Read alongside pistra_budget_used_ratio, which cannot tell one exhausted key from every exhausted key: both report 1.

pistra_budget_used_ratio · gauge · budget

Fraction of a budget’s allowance drawn by whichever of its buckets is closest to the limit, in the open window. This is the one to alert on below 1: refusals only start once a client tries, so a threshold here is the difference between finding out at 80% and finding out from a 429.

Routing and delivery

These say what the gateway did when the first choice did not answer. Retries and failovers are successes from the client’s point of view, and they are counted for that reason. A provider that is steadily failing is invisible in the error rate and obvious here.

pistra_channel_downgrade_total · counter · provider, channel, cap, action

Fidelity-guard decisions: requests using a capability a resolution candidate loses. skipped passed over a lossy candidate, allowed forwarded lossy by policy, rejected refused the request.

pistra_failover_skipped_total · counter · provider, reason

Failover candidates a request could not be admitted to and so passed over without trying, by provider and the refusal it would have got: model_denied, fidelity_loss, pool_subset_missing, or a rule’s own code. A candidate that is always skipped is a failover policy that does not do what it reads.

pistra_failover_total · counter · from, to, reason

Attempts abandoned for another provider, by the provider that failed, the one tried next, and why: connect, 5xx, 429 or timeout. A provider that appears here steadily is failing steadily; failover is hiding it from clients, not fixing it.

pistra_retry_total · counter · provider, reason

Attempts abandoned and re-sent to the same provider after a wait, by provider and why: connect, 5xx, 429 or timeout. Retries hide transient failures from clients; a steady rate here is a provider that is steadily flaky, and a rising one under load is the early sign of a retry storm.

Pools

These cover self-hosted model servers behind one name. The gateway does not rank them; it asks the pool’s endpoint picker and dials what it names, after checking the answer against the pool’s own membership. A pool with no picker is spread over its members in turn. The outcome to watch is fallback: the pool declares a scheduler and is not being scheduled, which nothing else reports.

pistra_pool_picks_total · counter · provider, outcome

How pooled requests were sent, by provider and outcome. epp means the pool’s endpoint picker chose an endpoint and the gateway dialed it; refused means the picker decided against the request, which reaches the caller as its own 503 or 429; fallback means the picker did not answer and the request was spread over the pool’s members in turn instead; failed means it did not answer and the pool refuses rather than serve unscheduled; round_robin is a pool with no picker, spread in turn; unresolved is a pool whose name answered with nothing, left on the pool address where one connection can hold one model server for its lifetime. A rising fallback is the one to alert on: the pool declares a scheduler and is not being scheduled, and nothing else reports that.

Guardrails

These cover what inspection found, what policy did about it, and what was never inspected. That last group exists because a guardrail can fail by doing nothing. pistra_guardrail_uninspected_total, pistra_guardrail_async_total{disposition="dropped"} and pistra_guardrail_detector_errors_total are each a way of scanning less than the configuration claims, and none of them is an error the client sees. Every label here comes from configuration vocabulary or a detector’s entity list; none comes from content.

pistra_guardrail_async_total · counter · route, disposition

Async scans by disposition: started, or dropped because async.max_in_flight was reached. Dropped means that request’s text was never seen by the async detectors and will not be in the cache for the next turn, so it is counted rather than assumed to have run.

pistra_guardrail_decisions_total · counter · action, rule, side

Guardrail policy decisions by action, the rule that decided, and the side of the exchange; action allow with an empty rule means no rule matched. A redaction several rules acted on increments once per acting rule, so the sum over rules exceeds the inspection count.

pistra_guardrail_detector_errors_total · counter · detector

Detector failures that did not refuse the request (fail_open).

pistra_guardrail_findings_total · counter · detector, entity, mode, side

Annotations produced, by detector, entity type and side of the exchange. mode inline means policy acted on them, shadow means they were recorded only.

pistra_guardrail_model_tokens_total · counter · detector, provider, model, token_type

Tokens a detector spent on a model of its own, such as an llm judge, by detector, provider, model and token type (input, output). These are the gateway’s calls, not a client’s, so pistra_tokens_total does not carry them. This is what judging costs.

pistra_guardrail_overlap_redact_total · counter · route, outcome

Redact rules that fired on an overlap pass, after the request was already forwarded and could no longer be rewritten, by what policy did instead: denied the response, or allowed it and recorded the finding. Non-zero means a rule is not doing what its author wrote.

pistra_guardrail_scan_seconds · histogram · route, side

Time one guardrail inspection took, by side of the exchange. Part of pistra_overhead_seconds, broken out so guardrail cost is attributable.

Buckets: 0.00001, 0.000025, 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25.

pistra_guardrail_segments_total · counter · result, side

Text segments by whether they were scanned or served from the delta-scan cache. A healthy conversation workload is mostly cached.

pistra_guardrail_stream_actions_total · counter · action

Streaming guardrail actions: redacted spans, partial redactions whose start had already been sent, and releases made under a guessed rather than proven commit horizon.

pistra_guardrail_stream_frames_total · counter · result

Response frames by fate: released to the client, or dropped because a guardrail stopped the stream.

pistra_guardrail_stream_held_bytes · histogram · route

High-water mark of response text held back on one stream. This is the latency a response guardrail costs; it is deliberately not folded into pistra_overhead_seconds, which measures gateway work rather than deliberate delay.

Buckets: 16, 32, 64, 128, 256, 512, 1024, 4096, 16384, 65536.

pistra_guardrail_uninspected_total · counter · route, reason

Response bodies a configured guardrail did not examine, by reason: oversize past response.max_body_bytes, or unparseable. The one way a response guardrail does not apply, so it is counted rather than assumed away.

Metering

These are the token ledger and the second opinion on it. pistra_tokens_total is what the provider reported, so it matches the bill rather than the wire; the reconcile series are the gap between the two, and auditing your metering is about that gap.

pistra_reconcile_drift_ratio · gauge · provider

Worst per-class disagreement between wire-metered and provider-reported tokens on the most recently judged day; 0 means the ledgers agree.

pistra_reconcile_fetch_errors_total · counter · provider

Failed provider usage-API reconciliation rounds.

pistra_reconcile_judged_day_seconds · gauge · provider

Unix time of the start of the most recently judged UTC day; staleness here means the poller is stuck.

pistra_tokens_total · counter · route, provider, model, token_type

Tokens the provider reported, by route, provider, upstream model and token type (input, output, cached_input, cache_creation_input, reasoning). Provider-reported, so it matches the bill, not the wire; see pistra_reconcile_drift_ratio for the gap.

The cluster

These are raft as this node sees it, read fresh on every scrape. The chart scrapes every pod rather than a service for that reason. Which node leads and what each one holds are per-pod facts, and a scrape that load-balances across them reports a different node each time.

pistra_cluster_is_leader · gauge

1 on the node that leads; sums to 1 across a healthy cluster.

pistra_cluster_leader_known · gauge

1 when this node knows a raft leader, 0 between elections or with quorum lost.

pistra_cluster_members · gauge · suffrage

Nodes in the committed raft configuration, by suffrage. A voter that is gone is still a voter until removed.

pistra_config_revision · gauge

Revision of the config snapshot currently serving traffic.

The audit trail

There is one series here, and it is the only one on this page to alert on at any value above zero. See the audit trail for why a gap in this signal is different in kind from a gap in a log.

pistra_audit_export_failures_total · counter

Audit records that failed to export to the collector. Any value above zero is a hole in the trail whose only remaining copy is this node’s text log.

Regenerate with make gen-docs.