Skip to main content

Recipe: LLM model allow-list

The problem​

Your org has vetted Claude Sonnet 4, Claude Opus, GPT-4, and Gemini 2.5. You do not want agents to quietly call a new model an engineer discovered on Twitter last night, because (a) the legal team has not seen the privacy terms, (b) billing is not set up, and (c) evals have not been run.

The policy​

version: '1'
# Model allow-list for outbound LLM traffic.
#
# The Gateway (and SDK guard() wrapper) extracts the target model_id
# from the request body before forwarding. Only the models named
# below are permitted; everything else hits default_action deny.
settings:
  default_action: deny
  default_on_missing: deny
  default_on_tamper: deny
rules:
  - id: allow-claude-family
    allow: 'llm:call'
    when:
      model: 'claude-sonnet-4*'
    reason: 'Allowed model: Claude Sonnet 4 family.'
  - id: allow-claude-opus
    allow: 'llm:call'
    when:
      model: 'claude-opus-*'
    reason: 'Allowed model: Claude Opus family.'
  - id: allow-gpt-4
    allow: 'llm:call'
    when:
      model: 'gpt-4*'
    reason: 'Allowed model: OpenAI GPT-4 family.'
  - id: allow-gemini-2-5
    allow: 'llm:call'
    when:
      model: 'gemini-2.5*'
    reason: 'Allowed model: Google Gemini 2.5 family.'

Why it works​

Every LLM call (whether it arrives via the Gateway, the Python SDK, or the Node SDK) carries the target model in its request body. The policy layer reads that field, matches it against the glob patterns in each rule's when block, and allows only when a rule matches. If an engineer fires off sketchy-new-model-v0.1, no rule matches and default_action: deny fires.

What gets blocked​

Agent call (model)Extracted actionDecisionreason_code
sketchy-new-model-v0.1llm:calldenyNO_RULE_MATCH
claude-2.1 (deprecated)llm:calldenyNO_RULE_MATCH
gpt-3.5-turbo (not on list)llm:calldenyNO_RULE_MATCH
empty model fieldllm:calldenyNO_RULE_MATCH

What gets allowed​

Agent call (model)Extracted actionDecisionreason_code
claude-sonnet-4-6llm:callallowRULE_MATCH
claude-opus-4-7llm:callallowRULE_MATCH
gpt-4-turbollm:callallowRULE_MATCH
gemini-2.5-prollm:callallowRULE_MATCH

Test it yourself​

curl -O https://docs.controlzero.ai/recipes/llm-model-allowlist/policy.yaml
curl -O https://docs.controlzero.ai/recipes/llm-model-allowlist/scenarios.json

controlzero test-policy policy.yaml --scenarios scenarios.json

To see your current traffic mix before you flip this recipe to deny, run it with default_action: warn first for 48 hours and watch the audit log for warns on models you did not expect agents to use.

Caveats​

  • model_id is what arrives in the request body. Some providers accept aliases -- for example, gpt-4o-mini-2024-07-18 vs gpt-4o-mini. Write your when: patterns against the IDs that actually flow through your Gateway, not the marketing names. The audit log shows the raw IDs.
  • Routing layers like OpenRouter or LiteLLM may translate one model ID to another before the provider sees it. Install the Gateway upstream of the router so the policy decision runs against the client-supplied ID, not the translated one. Otherwise the rules see openrouter/anthropic/claude-sonnet-4-6 and fail to match claude-sonnet-4*.
  • This recipe does not cover locally-hosted / offline models. Pair with your Ollama-provider allow-list if that is in scope.