Braintrust Integration

Note: Control Zero handles policy enforcement. Braintrust is a complementary evaluation and logging tool. For evaluation and experiment tracking, these tools work alongside Control Zero.

Use Control Zero for policy enforcement alongside Braintrust for evaluation.

Overview

Control Zero and Braintrust are complementary:

• Control Zero: Enforces governance policies at runtime (prevention)
• Braintrust: Provides evaluation, logging, and experiment tracking (measurement)

Use Control Zero to block unauthorized actions. Use Braintrust to evaluate the quality and correctness of the actions you allow.

Installation

pip install controlzero braintrust

Combined Setup

import controlzero
import braintrust

# -- Control Zero: policy enforcement --
cz = controlzero.ControlZero()
cz.initialize()

# -- Braintrust: evaluation and logging --
# Braintrust uses the BRAINTRUST_API_KEY environment variable

Usage Pattern

Enforce Then Log

import controlzero
from braintrust import traced

@traced
def generate_response(prompt: str, model: str = "gpt-4") -> str:
    """Generate a response with enforcement and Braintrust tracing."""

    # Step 1: Control Zero enforces policy
    cz.enforce(
        action="llm.generate",
        resource=f"model/{model}",
        context={"agent_id": "eval-agent"},
    )

    # Step 2: Braintrust traces the call
    import openai
    client = openai.OpenAI()
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )

    return response.choices[0].message.content

Evaluate with Policy Enforcement

Run Braintrust evaluations where each test case goes through policy enforcement:

from braintrust import Eval

async def task(input_data):
    """Task function with Control Zero enforcement."""
    cz.enforce(
        action="llm.generate",
        resource="model/gpt-4",
        context={"agent_id": "eval-agent"},
    )
    return generate_response(input_data["prompt"])

Eval(
    "my-project",
    data=lambda: [
        {"input": {"prompt": "Summarize Q4 earnings"}, "expected": "..."},
        {"input": {"prompt": "Analyze market trends"}, "expected": "..."},
    ],
    task=task,
    scores=[braintrust.Factuality],
)

Next Steps

• See the Python SDK for the full Control Zero API reference.
• Learn about Policies for defining governance rules.