Schedules

Set mode and schedule independently. mode controls enforcement. schedule controls when the request waits for a detector.

• mode, does policy read this detector’s findings? inline or shadow.
• schedule, does the request wait for it? sync, overlap or async.

Three schedules on one time axis: sync runs the detector before the upstream call, overlap runs both at once, async never waits; the marked moment when the provider first holds the bytes moves earlier on each row

shadow disables enforcement, but it does not remove detector latency. Use a schedule to decide when that work runs.

sync

The request waits before anything is forwarded. This is the only schedule where the provider never sees the content, and it is the default because it is the only one that can make that promise.

overlap

A request-side scan can continue while the gateway calls the upstream provider. It must finish before the gateway returns a response.

Added latency is max(0, detector − provider latency). For a 590 ms scan against an 800 ms time-to-first-token, that is zero.

The gateway joins after it reads a unary response or the first stream frame. It joins before it writes the status, so a refusal is a 403.

An overlap pass runs the whole ensemble. The delta cache answers for members that already scanned the same segments. This lets one rule require a fast and a slow detector to agree.

Caution

The provider receives the bytes before an overlap detector finishes. A redact finding cannot change content that has already been forwarded. Set overlap.on_redact to choose whether the gateway denies or allows that response. Both outcomes are counted.

async

Nothing waits, not the request and not the response.

Async scans write their findings to the delta cache. When a later turn repeats the same text, a synchronous pass reads the cached finding.

Async enforcement begins on a later turn. Async members are consulted on the request path but never scanned there.

Account for these limits:

• The turn where text first appears is governed only by detectors that were waited for.
• Single-shot traffic, an embedding call, a one-off completion, has no next turn, so async there is purely audit.
• Silence from an async detector is not evidence, so Result.Deferred names them and an audit record can never imply one ran when it did not.

Set async.timeout because the request context ends when the response is written. Set async.max_in_flight to bound concurrent scans. The gateway drops and counts scans above that limit. It never queues them.

Late enforcement, deliberately not built

An async verdict reaches the next turn through the cache, but never reaches back into the request it came from. Killing the in-flight stream (terminate) and auto-disabling the key (quarantine) are not implemented.

Quarantine would disable a key as a durable control-plane action. It is not a scheduling decision, so pistra does not implement it.

The trust ramp this creates

Use the following example:

async  + shadow   free observation
overlap + shadow  measure the true added latency, at zero risk
overlap + inline  enforce
sync   + inline   only if the provider must never see it

See run a detector in shadow mode for the configuration at each step.