LiteLLM Integration

Control Zero enforces policy at the LiteLLM proxy with a Generic Guardrail API endpoint, out of process. Your LiteLLM proxy stays the router — its models, virtual keys, teams, budgets, and spend logs are untouched. Control Zero adds one config block and records every verdict outside the proxy, on the same signed policy that already governs your coding agents.

The same integration observes in library mode (Python SDK) without a proxy.

Try it now

No account, no key. This curl against our hosted proxy returns BLOCKED in under two minutes:

curl https://try-litellm.controlzero.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-try-it-public" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Here is an AWS access key: AKIAIOSFODNN7EXAMPLE"}]
  }'

The response is HTTP 400 with a blocked_reason in the grammar below.

Enforce at the proxy

Mint a guardrail-scoped key from the dashboard (Integrations → LiteLLM, or controlzero litellm add), then add one block to your proxy's config.yaml:

guardrails:
  - guardrail_name: controlzero
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]
      api_base: https://gateway.controlzero.ai
      default_on: true
      fail_on_error: true
      unreachable_fallback: fail_closed
      timeout: 30
      api_key: os.environ/CONTROLZERO_GUARDRAIL_KEY

litellm_settings:
  callbacks:
    - controlzero

Restart the proxy. Every request now flows app → your proxy → (guardrail hook) → Control Zero → verdict → your proxy honours it.

• mode: [pre_call, post_call] inspects the prompt and tool definitions (pre) and the response and proposed tool calls (post).
• The key is cz_guardrail_*, scoped to litellm:guardrail only — a leaked proxy environment yields nothing else.
• Identity travels on every call (user_api_key_team_alias, user_api_key_alias, user_api_key_org_id), so per-team policy works against plain LiteLLM OSS.

Observe in library mode

In library mode the Python SDK observes and audits but does not block — LiteLLM swallows exceptions from its callback hooks, so a raising hook cannot stop a call. Use the Observer for the audit feed:

from controlzero.integrations.litellm import ControlZeroLiteLLMObserver

litellm.callbacks = [ControlZeroLiteLLMObserver()]

The Observer is batched and best-effort: dropped batches are counted, never surfaced as proxy latency. For enforcement, use the proxy path above.

The four switches

Four LiteLLM settings turn the control off. They are fail-closed by default and each bypass is counted and logged at critical level with the call id:

Switch	Off state	What it does
fail_on_error: false	a guardrail error passes the call	set true
unreachable_fallback: fail_open	network errors and 5xx pass even with fail_on_error	set fail_closed
permissions: {controlzero: false}	a key or team opts out	remove the opt-out
default_on: false	a request that does not name the guardrail passes	set true

Versions tested

The compatibility matrix is generated from the conformance rig, which drives a real LiteLLM proxy against the endpoint. Verified lines and the two known upstream findings are below.

LiteLLM version	Result	Known upstream findings
1.101.0	10/10 — 4 scenarios, 4 mutations, 2 chaos	guardrail timeout ignored; key opt-out dead code
1.100.1	10/10	guardrail timeout ignored; key opt-out dead code
1.99.1	10/10	guardrail timeout ignored; key opt-out dead code

The two findings are present across all three tested lines and are LiteLLM's to fix; the rig records them as expect_pass=False so a future version that fixes one turns that case green rather than failing the gate.

Why was my call blocked

A BLOCKED verdict carries a stable blocked_reason. Each code has a page under docs/errors/<code>:

• dlp_blocked — a DLP rule matched the prompt or response.
• tool_denied — a proposed tool call was refused.
• policy_denied / no_rule_match / rule_match — policy outcomes.
• bundle_missing / bundle_tampered / bundle_expired — the policy bundle could not be trusted; the call fails closed.
• control_plane_unreachable — the control plane was unreachable during a degraded window.
• engine_panic / dlp_scan_error / dlp_mask_write_back_failed / unscanned_payload — the adapter could not scan or rewrite the payload and refused rather than passed it.
• network_error / integrity / machine_quarantined / unknown_effect — the remaining fail-closed reasons.

Next steps

• Policies: one signed policy for the proxy and your coding agents.
• Enforcement behavior: how verdicts are recorded outside the proxy with LiteLLM's call and trace ids.