Skip to main content

Cohere Integration

Add governance to Cohere Command R API calls. Control Zero intercepts requests through the Governance Gateway and enforces your dashboard policies on every interaction.

Overview​

Cohere provides language models including Command R and Command R+. When routed through the Control Zero Governance Gateway via the OpenAI-compatible /v1/chat/completions endpoint, every request is governed by your policies and logged for audit.

Cohere supports the OpenAI-compatible wire format, so the integration reuses the same interceptors and policy enforcement as the OpenAI integration.

Setup​

1. Configure the Gateway​

Enable the Cohere provider in your Governance Gateway configuration:

# Environment variables for the gateway
CZ_GATEWAY_COHERE_ENABLED=true
CZ_GATEWAY_COHERE_API_KEY=your_cohere_api_key

2. Route Requests Through the Gateway​

Point your client at the gateway's Cohere endpoint:

import openai

# Route through Control Zero gateway instead of direct Cohere
client = openai.OpenAI(
    base_url="http://localhost:8000/cohere/v1",
    api_key="cz_live_your_key_here",
)

response = client.chat.completions.create(
    model="command-r-plus",
    messages=[{"role": "user", "content": "Summarize the key points of this quarterly report."}],
)
print(response.choices[0].message.content)

3. Define Policies​

Create policies in the dashboard to govern Cohere usage:

{
  "name": "cohere-usage-policy",
  "rules": [
    {
      "effect": "allow",
      "action": "llm:chat.completions",
      "resource": "cohere/*",
      "conditions": { "model": ["command-r-plus", "command-r"] }
    },
    {
      "effect": "deny",
      "action": "llm:chat.completions",
      "resource": "cohere/*",
      "conditions": { "max_tokens_gt": 4096 }
    }
  ]
}

What Gets Governed​

API EndpointPolicy ActionDescription
POST /cohere/v1/chat/completionsllm:chat.completionsChat completions (streaming and non-streaming)
POST /cohere/v1/completionsllm:completionsText completions
GET /cohere/v1/modelsllm:models.listModel listing

SDK Wrapping​

Since Cohere uses the OpenAI wire format, use wrap_openai() in the Python SDK or the standard OpenAI client wrapper in Node.js:

from controlzero import Client
from controlzero.integrations.openai import wrap_openai
import openai

cz = Client(api_key="cz_live_your_key_here")

client = wrap_openai(
    openai.OpenAI(
        base_url="http://localhost:8000/cohere/v1",
        api_key="cz_live_your_key_here",
    ),
    cz,
)

response = client.chat.completions.create(
    model="command-r-plus",
    messages=[{"role": "user", "content": "Hello"}],
)

Streaming Support​

The gateway fully supports streaming responses from Cohere. Server-sent events are forwarded transparently while policy enforcement and audit logging occur on the initial request.

Tool Call Governance​

Cohere Command R models that support tool use are fully governed. The gateway intercepts tool call responses and evaluates each tool invocation against your policies before forwarding to the client.

Audit Logging​

All Cohere requests routed through the gateway are logged in the audit trail with the provider identified as cohere. View them in the dashboard under Audit Log with the provider filter.