Logging

Control Zero Self-Managed produces two categories of logs: application logs (operational events from each service) and audit logs (governance decisions). This guide covers configuration, file locations, rotation, and debug mode.

Log Levels

Five log levels are available, from least to most verbose:

Level	Description	Use Case
`error`	Errors that require attention. Service failures, unhandled exceptions.	Production (minimal output)
`warn`	Warnings that do not stop operation but indicate potential issues.	Production (default)
`info`	Standard operational events. Service startup, configuration loaded, policy bundle refreshed.	Production (recommended)
`debug`	Detailed operational events. Request/response metadata, policy evaluation details, timing.	Troubleshooting
`trace`	Maximum verbosity. Full request/response bodies, internal state dumps.	Deep debugging only

Default level: info for all services.

Setting Log Levels

All services at once

czctl set-log-level <level>

For example, to set all services to debug:

czctl set-log-level debug

Per-service

czctl set-log-level <level> --service <service-name>

Via runtime configuration

Log levels can also be changed at runtime through the cz_config table. Updates take effect within 60 seconds without a service restart:

curl -s https://localhost:8080/api/v1/config \
  -X PATCH \
  -H "Authorization: Bearer <admin-token>" \
  -H "Content-Type: application/json" \
  -d '{"key": "gateway.log_level", "value": "debug"}'

Available runtime config keys:

Config Key	Equivalent Environment Variable
`gateway.log_level`	`CZ_GATEWAY_LOG_LEVEL`
`api.log_level`	`CZ_API_LOG_LEVEL`
`engine.log_level`	`CZ_ENGINE_LOG_LEVEL`
`proxy.log_level`	`CZ_PROXY_LOG_LEVEL`
`scout.log_level`	`CZ_SCOUT_LOG_LEVEL`

Via environment variables

Set the log level in config/controlzero.env or directly in docker-compose.yml:

Service	Environment Variable
Gateway	`CZ_GATEWAY_LOG_LEVEL`
API	`CZ_API_LOG_LEVEL`
Policy Engine	`CZ_ENGINE_LOG_LEVEL`
SSL Proxy	`CZ_PROXY_LOG_LEVEL`
Scout	`CZ_SCOUT_LOG_LEVEL`

After changing environment variables, restart the affected service:

docker compose restart <service-name>

Log File Locations

Container logs (via Docker)

docker compose logs <service-name>
docker compose logs <service-name> --tail 100
docker compose logs <service-name> --follow

Persistent log files

If file-based logging is enabled (default for enterprise deployments), logs are written to:

Service	File Path
Gateway	`/var/log/controlzero/gateway.log`
API	`/var/log/controlzero/api.log`
Policy Engine	`/var/log/controlzero/engine.log`
SSL Proxy	`/var/log/controlzero/proxy.log`
Scout	`/var/log/controlzero/scout.log`
Audit DB	`/var/log/controlzero/audit-db.log`

These paths are inside the containers and mapped to the host via Docker volumes. The host path defaults to /opt/controlzero/logs/.

Log Rotation

Default rotation policy

Setting	Default Value
Maximum file size	100 MB
Maximum file count	10 (per service)
Compression	gzip after rotation

Configuring rotation

Log rotation is managed through the Docker logging driver. Configure it in docker-compose.yml under each service:

services:
  gateway:
    logging:
      driver: json-file
      options:
        max-size: '100m'
        max-file: '10'

Apply changes:

docker compose restart

External log rotation

If you prefer to manage rotation with an external tool (logrotate or similar), set the Docker logging driver to local or none and configure your external rotation tool to manage files in /opt/controlzero/logs/.

SIEM Compatibility

All Control Zero services write Docker JSON logs to the standard Docker log path:

/var/lib/docker/containers/<container-id>/<container-id>-json.log

These logs are compatible with any SIEM or log collector that supports the Docker JSON log format, including:

Splunk -- Use the Splunk Docker logging driver or forward from /var/lib/docker/containers/
Elasticsearch / OpenSearch -- Use Filebeat with the Docker input
Datadog -- Use the Datadog Agent with Docker log collection enabled
Fluentd / Fluent Bit -- Use the Fluentd Docker logging driver or tail the JSON log files
Syslog -- Use the Docker syslog logging driver for direct forwarding

To forward logs via the Docker daemon, configure the logging driver globally in /etc/docker/daemon.json or per-service in docker-compose.yml.

Audit event export: CEF and Syslog

The methods above forward the raw Docker JSON application logs. For the governance audit trail specifically (every allow/deny/warn decision), Control Zero emits structured ArcSight CEF and RFC 5424 syslog so the records land cleanly in Splunk, IBM QRadar, and Micro Focus ArcSight without a custom parser. There are two delivery modes; both are available on self-managed and air-gapped installs.

1. Pull (export endpoint). Add ?format=cef or ?format=syslog to the audit export endpoint. json (the default) is unchanged and byte-identical to prior releases.

# CEF lines (one event per line)
curl -H "Authorization: Bearer $CZ_TOKEN" \
  "https://controlzero.example/api/projects/$PROJECT_ID/export?format=cef&start=$START_MS&end=$END_MS"

# RFC 5424 syslog frames wrapping CEF
curl -H "Authorization: Bearer $CZ_TOKEN" \
  "https://controlzero.example/api/projects/$PROJECT_ID/export?format=syslog"

The export respects the same RBAC (org membership), time-range, and feature gating (audit_export, Solo+ tier) as the JSON export. An unwired audit store returns 501; a non-member returns 403; an unrecognized format returns 400.

2. Push (outbound forwarder). Set SIEM_SYSLOG_ADDR to stream each durably-stored decision to a collector over UDP, TCP, or TLS. Configure it via environment (operator-only; never tenant-controlled):

SIEM_SYSLOG_ADDR      host:port of the collector (presence enables forwarding)
SIEM_SYSLOG_PROTO     udp | tcp | tcp+tls            (default: tcp)
SIEM_SYSLOG_FORMAT    cef | json                     (default: cef)
SIEM_SYSLOG_APP_NAME  RFC5424 APP-NAME               (default: control-zero)
SIEM_SYSLOG_TLS_CA    PEM CA bundle path             (tcp+tls only)
SIEM_SYSLOG_TLS_CERT  client cert path for mutual TLS (optional)
SIEM_SYSLOG_TLS_KEY   client key path for mutual TLS  (optional)

The forwarder is fire-and-forget: it never blocks or fails audit ingest. A slow or unreachable collector causes rows to be dropped (the durable audit store remains the system of record), counted by the Prometheus metric controlzero_siem_forward_failed_total (labels: buffer_full, dial, write). On an air-gapped install the collector sits on the customer's private network; the forwarder intentionally permits private destinations because the target is operator-configured, not a tenant-supplied URL.

CEF field mapping. Each audit row renders as:

CEF:0|ControlZero|control-zero|<ver>|<decision>|<tool>|<severity>|rt=<epoch_ms> src=<client_ip> suser=<user_email> shost=<hostname> act=<decision> outcome=<status> cs1=<policy_id> cs1Label=PolicyID cs2=<reason_code> cs2Label=ReasonCode cs3=<source> cs3Label=Source cs4=<org_id> cs4Label=OrgID cs5=<project_id> cs5Label=ProjectID cs6=<sdk_version> cs6Label=SDKVersion deviceProcessName=<agent_id> externalId=<id>

Severity follows the decision: deny = 8, warn = 5, allow = 3. The RFC 5424 frame computes PRI = local0(16) * 8 + severity, where the syslog severity is deny = Error(3), warn = Warning(4), allow = Informational(6). All header and extension values are escaped per the CEF spec, so a crafted tool name or reason cannot break the line or inject a second event.

Audit Logs vs Application Logs

Application logs

Operational events from each service. Used for troubleshooting and monitoring service health. Contain information like startup messages, configuration changes, error stack traces, and request timing.

Audit logs

Governance decision records stored in the immutable audit trail. Every policy evaluation produces an audit record with:

Timestamp
Agent identity
Action evaluated
Resource
Decision (allow or deny)
Policy ID that matched
Token usage (if applicable)
Request metadata

Audit logs are queried through the dashboard or the management API. They are stored separately from application logs and have their own retention policy.

Field reference

Every audit row includes the fields below. The dashboard renders a subset by default; the rest are accessible by expanding a row, exporting CSV, or querying the management API.

Field	Type	Description
`id`	UUID	Stable identifier for the row. SDK-supplied to make retries idempotent.
`timestamp`	ISO 8601	When the host runtime fired the event (UTC).
`org_id`	UUID	Owning organisation. Server-enforced, not client-supplied.
`project_id`	UUID	Project the API key belongs to.
`agent_id`	string	Logical agent name (e.g. `code-assistant`, `support-bot`). Free-form; set at SDK init.
`tool_name`	string	Canonical tool name the policy engine evaluated. See the canonical tool names reference for the full list of host-tool mappings.
`method_name`	string	Sub-method on the tool (e.g. `SELECT` on `database`). `*` when not extracted.
`extracted_method`	string	Method the SDK hook extractor resolved from arguments. Empty when no extractor ran.
`decision`	enum	Policy outcome: `allow`, `deny`, `warn`.
`status`	enum	Wire status: `success`, `denied`, `error`.
`policy_id`	string	UUID of the policy rule that matched, if any.
`policy_matched`	string	Human-readable name of the matched rule.
`latency_ms`	uint	End-to-end evaluation time, in milliseconds.
`provider`	string	LLM provider for token-bearing calls (e.g. `openai`, `anthropic`).
`model_id`	string	LLM model identifier (e.g. `claude-sonnet-4`).
`input_tokens`	uint	Prompt tokens for LLM calls. Zero for non-LLM tool calls.
`output_tokens`	uint	Completion tokens for LLM calls. Zero for non-LLM tool calls.
`estimated_cost_usd`	float	Best-effort cost estimate based on the provider price card.
`client_ip`	string	Source IP the audit event arrived from.
`hostname`	string	Workstation hostname the SDK saw at runtime.
`user_email`	string	Developer email captured by enrolled SDKs (fleet attribution).
`machine_id`	UUID	Stable machine identifier from enrolment.
`client_name`	string	Free-form client label the SDK supplies (e.g. `claude-code`, `cursor`). Pre-dates the canonical `source` field; both still flow.
`client_version`	string	Version of the host CLI / agent runtime.
`sdk_version`	string	Control Zero SDK version that emitted the row (e.g. `python-1.3.0`, `gateway-0.2.0`).
`source`	enum	Canonical host runtime that emitted this event. See the table below.
`tags`	object	Free-form key/value metadata the SDK attaches.

`source` field — allowed values

source answers the question "which thing fired this event": the user's coding CLI, an MCP server, a direct SDK integration, or the proxy gateway. Values are lowercase, snake_case, and constrained to the set below. The dashboard renders an unset or unknown source as -- rather than the literal word Unknown.

Value	Emitted by
`claude_code`	Claude Code via the Control Zero Python SDK PreToolUse hook.
`gemini_cli`	Gemini CLI via the Control Zero Node SDK hook.
`codex_cli`	Codex CLI sandbox tap.
`kiro_cli`	Kiro CLI (`kiro-cli`) via the Control Zero SDK hook.
`kiro_ide`	Kiro IDE (AWS agentic IDE) via the Control Zero SDK hook.
`antigravity`	Antigravity (Google "agy") via the Control Zero SDK hook.
`cursor_ide`	Cursor IDE via the Control Zero SDK hook.
`cursor_cli`	Cursor CLI (`cursor-agent`) via the Control Zero SDK hook.
`mcp_server`	An MCP server using `@controlzero/mcp-server`.
`python_sdk`	Direct Python SDK integration (no host CLI -- application code calling `guard`).
`node_sdk`	Direct Node SDK integration.
`go_sdk`	Direct Go SDK integration.
`gateway`	The Control Zero proxy gateway (LLM-API-shaped traffic).
`unknown`	Fallback when the SDK did not supply a value and nothing in the request can be inferred. The dashboard renders this as `--`.

Resolving `source` server-side

When the SDK does not set source explicitly, the API derives one with the following ladder:

Use the wire source field if it is in the allowed set.
Else map the SDK's client_name (e.g. claude-code -> claude_code).
Else infer from sdk_version (e.g. gateway-0.2.0 -> gateway, python-1.3.0 -> python_sdk).
Else default to literal lowercase unknown.

This means rows from older SDK versions that still emit client_name continue to populate the SOURCE column correctly, even before customers upgrade.

DLP findings in audit logs

When the gateway detects PII in a request or response, the audit log entry includes a dlp_findings array with details about each detection:

{
  "timestamp": "2026-04-05T14:22:10.456Z",
  "level": "info",
  "service": "gateway",
  "message": "Policy evaluation completed",
  "fields": {
    "agent_id": "agent-prod-01",
    "action": "llm:generate",
    "decision": "allow",
    "latency_ms": 2.1,
    "request_id": "req_abc123def456",
    "dlp_findings": [
      {
        "direction": "response",
        "type": "credit_card",
        "locale": "default",
        "action_taken": "mask",
        "count": 1
      }
    ]
  }
}

Each finding includes the scan direction (request or response), the pattern type, the locale that matched, the action taken (detect, mask, or block), and the number of occurrences.

Correlation IDs in audit logs

Every audit log entry includes a request_id field containing the correlation ID for that request. This is the same value returned in the X-Request-ID response header.

Use correlation IDs to join audit log entries with your application logs and upstream provider logs for end-to-end request tracing.

# Find all audit entries for a specific request
cat /opt/controlzero/logs/gateway.log | jq 'select(.fields.request_id == "req_abc123def456")'

Prometheus metrics reference

The gateway exposes seven metric families on the /metrics endpoint in Prometheus exposition format. These metrics complement the structured audit logs with real-time operational telemetry.

Metric	Type	Labels
`cz_gateway_requests_total`	Counter	`provider`, `model`, `status_code`
`cz_gateway_request_duration_seconds`	Histogram	`provider`, `model`
`cz_gateway_policy_evaluations_total`	Counter	`decision`
`cz_gateway_pii_detections_total`	Counter	`type`, `direction`
`cz_gateway_rate_limit_hits_total`	Counter	`scope`
`cz_gateway_upstream_errors_total`	Counter	`provider`, `status_code`
`cz_gateway_active_connections`	Gauge	--

Scrape the /metrics endpoint with Prometheus and build Grafana dashboards from the counters and histograms above.

Audit log retention

Configure retention in config/audit.yml:

audit:
  retention_days: 90
  partition_by: month

Debug Mode

Enable debug mode when troubleshooting specific issues. Debug mode increases log verbosity for all services simultaneously.

Enable debug mode

czctl set-log-level debug

Enable trace mode

Use trace mode only when debug output is insufficient. Trace mode logs full request and response bodies, which can generate large volumes of output and may include sensitive data.

czctl set-log-level trace

Disable debug mode

Restore the default log level after troubleshooting:

czctl set-log-level info

Structured JSON Log Format

All services output logs in structured JSON format for compatibility with log aggregation systems (Splunk, Elasticsearch, Datadog, and similar):

{
  "timestamp": "2026-04-03T10:15:30.123Z",
  "level": "info",
  "service": "gateway",
  "message": "Policy evaluation completed",
  "fields": {
    "agent_id": "agent-prod-01",
    "action": "database:query",
    "decision": "allow",
    "latency_ms": 1.2,
    "policy_id": "pol_abc123"
  }
}

Parsing with common tools

# Filter errors from gateway logs
cat /opt/controlzero/logs/gateway.log | jq 'select(.level == "error")'

# Count decisions by result
cat /opt/controlzero/logs/gateway.log | jq 'select(.message == "Policy evaluation completed") | .fields.decision' | sort | uniq -c

Audit log argument redaction

Argument values for each governed tool call can be stored in three modes: key-only (the default), encrypted at rest with admin-only unmask, or plaintext for admin-only viewing. By default, both hosted and self-managed deployments run in key-only mode -- values are dropped at the SDK boundary and never reach the audit store. Self-managed operators who need argument-level visibility for incident response can opt in to store_full (database is single-tenant trusted) or redact_with_admin_unmask (encrypted-at-rest with audit-on-decrypt chain). See the customer-facing reference at docs/sdk/policies/audit-redaction.md for the trade-offs and the opt-in flow.

Log Levels​

Setting Log Levels​

All services at once​

Per-service​

Via runtime configuration​

Via environment variables​

Log File Locations​

Container logs (via Docker)​

Persistent log files​

Log Rotation​

Default rotation policy​

Configuring rotation​

External log rotation​

SIEM Compatibility​

Audit event export: CEF and Syslog​

Audit Logs vs Application Logs​

Application logs​

Audit logs​

Field reference​

source field — allowed values​

Resolving source server-side​

DLP findings in audit logs​

Correlation IDs in audit logs​

Prometheus metrics reference​

Audit log retention​

Debug Mode​

Enable debug mode​

Enable trace mode​

Disable debug mode​

Structured JSON Log Format​

Parsing with common tools​

Audit log argument redaction​