Skip to main content

Setup for DevOps engineers

Surface: Gateway Modes supported: Hosted Hybrid Local Tiers: Free Solo Teams

Who this is for​

You run the services. AI calls flow out of apps you may not own the source of, and you need a control and audit point in front of them, without asking every team to ship a code change.

What you typically want governed​

  • One choke point for every LLM call across services, set by configuration.
  • Model and cost guardrails applied uniformly, not per-app.
  • An audit trail that survives even for agents you cannot instrument.

Which surface to install​

The Gateway. It is a transparent proxy: change one base URL and add two headers, and the gateway governs the LLM calls that flow through it. No application code changes.

# Point your app's provider base URL at the gateway.
export OPENAI_BASE_URL=https://gateway.controlzero.ai/v1

Add the Control Zero headers to outbound requests (set them in your service's config or sidecar):

X-ControlZero-API-Key: cz_live_your_key_here
X-ControlZero-Agent-ID: payments-service

The gateway also runs self-hosted if you need it inside your own boundary -- see the Gateway guide.

Starter policy​

Start in observe so no production call breaks while you roll the gateway out, then tighten. This allows traffic and records everything:

version: '1'
settings:
  # Roll out in observe; flip to "deny" per service once audit looks clean.
  default_action: allow
  default_on_missing: allow
  default_on_tamper: warn
rules:
  - id: allow-vetted-models
    allow: 'llm:generate'
    resources:
      - 'model/claude-*'
      - 'model/gpt-*'
    reason: 'Only vetted model families while we establish a baseline.'

Once the audit log shows a stable set of models, move default_action to warn, then deny, and add a deny catch-all so any model not on the list is blocked. The same policy promotes through dev, staging, and prod by changing only that one setting -- see Dev warns, prod denies and LLM model allow-list.

What you'll see​

  • Every request and response decision in the Audit Log, attributed to the X-ControlZero-Agent-ID you set per service, so you can tell which app made which call.
  • Pre-flight checks (model blocking, cost caps, PII detection) recorded on every request before it reaches the provider.

Next steps​