Skip to content
pistra

Redact PII from model output

Inspect what the model sends back, rather than what the client sends.

Turn on response scanning

Section titled “Turn on response scanning”

Response scanning is opt-in. A detector reads responses only if its apply_to names output. Leaving apply_to empty means every request kind and not output.

guardrails:
  detectors:
    - type: pii
      entities: [CREDIT_CARD, US_SSN]
      apply_to: [user, tool_args]     # requests
    - type: pii
      entities: [CREDIT_CARD]
      apply_to: [output]              # responses
  rules:
    - name: never-echo-a-card
      when: 'side == "response" && "CREDIT_CARD" in entities'
      action: redact
      operator: mask
      mask_chars: 12

Use two detector entries rather than one with both kinds listed. The request and response sides usually want different entity sets, and the response side costs latency that the request side does not.

Narrow rules to one side

Section titled “Narrow rules to one side”

Every rule sees side, which is request or response. Without it a rule written for responses also fires on requests.

    - name: strip-emails-from-answers
      when: 'side == "response"'
      action: redact
      select: 'a.entity_type == "EMAIL_ADDRESS"'
      operator: replace
      replacement: "[email removed]"

when decides whether the rule fires. select decides which annotations it rewrites. A redact rule with no select rewrites everything that survived resolution.

Bound what a non-streamed response can cost

Section titled “Bound what a non-streamed response can cost”

Add this configuration:

guardrails:
  response:
    max_body_bytes: 1048576   # 1 MiB
    on_oversize: allow        # allow | deny

A body over the bound is not inspected. Whichever way on_oversize sends it, the fact is counted:

pistra_guardrail_uninspected_total{route,reason="oversize"}

Alert on that metric.

Choose what streaming costs

Section titled “Choose what streaming costs”

A streamed response cannot be inspected after the fact, because bytes already sent cannot be recalled. The guard holds back exactly as far as a match could reach:

guardrails:
  stream:
    policy: window            # window | buffer
    max_hold_bytes: 512
    on_hold_overflow: emit    # emit | buffer
  • policy: window streams with a computed delay. Use this one.
  • policy: buffer waits for the whole response. It ends streaming. Use it only when a partial answer is worse than a slow one.
  • on_hold_overflow applies when the enabled recognizers have no finite maximum match length. emit releases with the risk counted in pistra_guardrail_stream_actions_total{action="unproven_release"}. buffer refuses to guess and pays the latency.

Restrict the entity set to shorten the hold. With CREDIT_CARD alone the stream runs 20 bytes behind. With everything enabled there is no finite bound, because entities like EMAIL_ADDRESS and URL have unbounded repeats. See the commit horizon.

Watch the delay

Section titled “Watch the delay”

Use the following example:

pistra_guardrail_stream_held_bytes{route}

It is not part of pistra_overhead_seconds, which measures work the gateway does rather than delay it chooses.

Under a gateway

Section titled “Under a gateway”

The inspector role applies the same response guards to traffic a gateway you already run is routing. Set extproc.inspect_responses: streamed and match the filter’s response_body_mode. See deploy guardrails beside your gateway.

Section titled “Related”