Skip to main content

Recipe: Block secrets and PII egress to LLMs

The problem​

Your agent needs to call external LLMs to do its job, so you cannot just block llm:*. What you actually want is: allow the call, but never let an API key, password, token, SSN, or credit card number leave the building. A leaked provider key funds someone else's model bills; a leaked customer SSN is a regulator conversation.

The policy​

version: '1'
# Block secrets egress.
#
# Agent can call external LLMs, but the Gateway DLP layer scans the
# outbound request body (prompt text + tool arguments) and blocks on
# secret-like or PII-like findings. Benign prompts pass through.
#
# DLP rules run AFTER the policy layer's allow decision and can
# downgrade the decision to deny with reason_code DLP_BLOCKED.
settings:
  default_action: deny
  default_on_missing: deny
  default_on_tamper: deny
rules:
  - id: allow-llm-calls
    allow: 'llm:*'
    reason: 'LLM calls are allowed by default; DLP layer blocks secret-like content.'
dlp:
  - id: block-api-keys
    action: block
    pattern_group: secret_like
    reason: 'Request body matched a secret-like pattern (API key, token, or password).'
  - id: block-pii
    action: block
    pattern_group: pii_detected
    reason: 'Request body matched a PII pattern (SSN, credit card, or similar).'

Why it works​

The Gateway scans LLM request bodies before forwarding; the browser extension scans pastes before they reach web-based LLMs. Each layer runs the same DLP pattern library, so a secret caught in an API payload looks the same in an audit record as a secret caught in a browser paste. When DLP matches, the policy-level allow is downgraded to deny with reason_code: DLP_BLOCKED and the Agent never sees the upstream response.

What gets blocked​

Agent callExtracted actionDecisionreason_code
llm call with sk_live_... keyllm:calldenyDLP_BLOCKED
llm call with Bearer tokenllm:calldenyDLP_BLOCKED
llm call with AWS access key IDllm:calldenyDLP_BLOCKED
llm call with SSN 123-45-6789llm:calldenyDLP_BLOCKED
llm call with credit card 4111 1111 1111 1111llm:calldenyDLP_BLOCKED

What gets allowed​

Agent callExtracted actionDecisionreason_code
llm call asking for a customer replyllm:callallowRULE_MATCH
llm call with a public URLllm:callallowRULE_MATCH
llm call with a UUID that is not a keyllm:callallowRULE_MATCH

Test it yourself​

curl -O https://docs.controlzero.ai/recipes/block-secrets-egress/policy.yaml
curl -O https://docs.controlzero.ai/recipes/block-secrets-egress/scenarios.json

# Requires SDK + Gateway with DLP enabled. The test driver mocks the
# DLP hit-list so you do not need a live Gateway to run the scenarios.
controlzero test-policy policy.yaml --scenarios scenarios.json

The full list of built-in DLP patterns (65 at launch, plus custom regex support) lives in DLP coverage. If your org has domain- specific secrets, add them as a custom pattern before you ship this recipe.

Caveats​

  • Obfuscated secrets (base64 with whitespace, character-substituted, hand-split across two prompts) are not caught by regex-only DLP. The Gateway's semantic-similarity DLP tier catches some of these; even so, assume a motivated insider can smuggle data out and pair this recipe with approval-workflow recipes when the data is high-value.
  • Screenshots pasted into web-based LLMs are not OCR-scanned by the browser extension today. Strip screenshots at the client before allowing them into the chat UI.
  • A prompt that asks the model to "repeat the following after rot13-decoding" is still a policy-level llm:call allow. Pair the recipe with a per-provider model allow-list (see LLM model allow-list) to at least pin which models see the payload.