Ollama Integration

Add governance to locally-hosted Ollama models. Control Zero intercepts Ollama API calls through the Governance Gateway and enforces your dashboard policies on every request.

Overview

Ollama runs open-source LLMs locally. When routed through the Control Zero Governance Gateway, every chat completion and model interaction is governed by your policies and logged for audit.

Ollama uses an OpenAI-compatible wire format, so the integration reuses the same interceptors and policy enforcement as the OpenAI integration.

Setup

1. Configure the Gateway

Enable the Ollama provider in your Governance Gateway configuration:

# Environment variables for the gateway
CZ_GATEWAY_OLLAMA_ENABLED=true
CZ_GATEWAY_OLLAMA_API_URL=http://localhost:11434/v1

2. Route Requests Through the Gateway

Instead of calling Ollama directly, point your client at the gateway's Ollama endpoint:

import openai

# Route through Control Zero gateway instead of direct Ollama
client = openai.OpenAI(
    base_url="http://localhost:8000/ollama/v1",
    api_key="cz_live_your_key_here",
)

response = client.chat.completions.create(
    model="llama3",
    messages=[{"role": "user", "content": "Summarize this quarter's sales data."}],
)
print(response.choices[0].message.content)

3. Define Policies

Create policies in the dashboard to govern Ollama usage:

{
  "name": "ollama-production-rules",
  "rules": [
    {
      "effect": "allow",
      "action": "llm:chat.completions",
      "resource": "ollama/*",
      "conditions": { "model": ["llama3", "mistral"] }
    },
    {
      "effect": "deny",
      "action": "llm:chat.completions",
      "resource": "ollama/*",
      "conditions": { "model": ["*-uncensored"] }
    }
  ]
}

What Gets Governed

API Endpoint	Policy Action	Description
POST /ollama/v1/chat/completions	llm:chat.completions	Chat completions (streaming and non-streaming)
POST /ollama/v1/completions	llm:completions	Text completions
GET /ollama/v1/models	llm:models.list	Model listing

Streaming Support

The gateway fully supports streaming responses from Ollama. Server-sent events are forwarded transparently while policy enforcement and audit logging occur on the initial request.

Audit Logging

All Ollama requests routed through the gateway are logged in the audit trail with the provider identified as ollama. View them in the dashboard under Audit Log with the provider filter.

Tool Calling

Tool-calling via the OpenAI-compatible endpoint is supported for llama3.1+, mistral-nemo, and other tool-capable Ollama models. The gateway passes tool_calls through transparently, and every tool call is evaluated against your policies just like any other provider.

Limitations

• The gateway proxies to a single Ollama instance per configuration. For multiple Ollama hosts, run multiple gateway instances.

Related

• OpenAI Integration: Same wire format, same governance patterns.
• DeepSeek Integration: Another OpenAI-compatible provider.
• MCP Integration: Govern MCP tool calls.
• API Reference: Full API documentation.