Troubleshooting

Common issues and their fixes for Control Zero Self-Managed deployments.

First Step: Check System Status

Before investigating specific issues, run the system status check to get an overview of all services:

czctl status

This displays the health of every service, container states, resource usage, and any active warnings. Use this output to narrow down which service requires attention.

Service Won't Start

Invalid or missing license key

Symptom: Services exit immediately after starting. Logs show license validation failed.

Fix: Verify the license key is correctly set in your configuration:

# Check if the key is present
grep LICENSE_KEY config/controlzero.env

# Re-apply the key
czctl update-license cz_ent_<your-key>

Port conflict

Symptom: Docker Compose reports port is already allocated.

Fix: Identify which process holds the port and either stop it or change the Control Zero port:

# Find the conflicting process
sudo lsof -i :8443

# Change the gateway port in docker-compose.yml
# Under the gateway service, update the ports mapping

Insufficient disk space

Symptom: Container images fail to load. Docker logs show no space left on device.

Fix: Free at least 10 GB on the Docker data partition:

# Check disk usage
df -h /var/lib/docker

# Remove unused Docker resources
docker system prune --volumes

Configuration file missing or malformed

Symptom: A specific service fails to start. Logs reference a missing or invalid config file.

Fix: Re-initialize the configuration:

czctl init

Then reapply any customizations you had made to the generated files.

Gateway Not Intercepting Traffic

Wrong base URL in client configuration

Symptom: LLM requests bypass the gateway and go directly to the provider.

Fix: Verify the client's base URL points to the gateway:

# The base URL should be the gateway address, not the provider
# Correct:
ANTHROPIC_API_URL=https://<gateway-host>:8443

# Incorrect:
ANTHROPIC_API_URL=https://api.anthropic.com

Certificate not trusted by the client

Symptom: Client reports TLS errors such as certificate verify failed or unable to get local issuer certificate.

Fix: Ensure the CA certificate is trusted by the system or application making the request:

# For system-level trust (Debian/Ubuntu)
sudo cp config/certs/ca.pem /usr/local/share/ca-certificates/controlzero.crt
sudo update-ca-certificates

# For Python requests library
export REQUESTS_CA_BUNDLE=/path/to/config/certs/ca.pem

# For Node.js
export NODE_EXTRA_CA_CERTS=/path/to/config/certs/ca.pem

Gateway health check failing

Symptom: The gateway container is running but not accepting connections.

Fix: Check the gateway health endpoint and logs:

curl -k https://localhost:8443/health

docker compose logs gateway --tail 50

Scout Not Reporting

Incorrect API URL

Symptom: Scout agent is running but no data appears in the dashboard.

Fix: Verify the Scout agent configuration points to your on-premises API:

# Check the Scout config
cat /etc/controlzero/scout.yml

# The api_url should point to your local deployment
# api_url: https://<your-host>:8080

Network connectivity between Scout and the API

Symptom: Scout logs show connection refused or timeout errors.

Fix: Verify network reachability:

# From the Scout host, test connectivity to the API
curl -k https://<api-host>:8080/health

If the hosts are on different network segments, verify firewall rules allow traffic on port 8080.

Insufficient permissions

Symptom: Scout reports partial results or permission errors.

Fix: Scout requires read access to process lists and network connections. On Linux, run Scout with appropriate capabilities:

# Verify Scout has the required capabilities
sudo setcap cap_net_raw,cap_net_admin+eip /opt/controlzero/scout

Database Connection Issues

Symptom: Audit log queries fail. Dashboard shows "Unable to load audit data."

Fix: Check the audit database health:

# Run targeted diagnostics
czctl diagnostics database

# Verify the audit database container is running
docker compose ps audit-db

# Check database connectivity
docker compose exec audit-db cz-db-check --query "SELECT 1"

# Check for disk space on the database volume
docker compose exec audit-db df -h /var/lib/audit-data

If the database volume is full, refer to Logging for log rotation configuration.

SSL Proxy Certificate Errors

Symptom: Browsers show certificate warnings when accessing inspected AI domains.

Fix: The CA certificate is not trusted on the endpoint. See SSL Proxy Setup for deployment instructions by platform (GPO, MDM, Linux system trust).

Symptom: Certificate errors only in Firefox.

Fix: Firefox uses its own certificate store by default. Either import the CA manually in Firefox settings, or enable the security.enterprise_roots.enabled policy to inherit the OS trust store.

Log Locations

Service	Log Location
Gateway	docker compose logs gateway or /var/log/controlzero/gateway.log
API	docker compose logs api or /var/log/controlzero/api.log
Audit DB	docker compose logs audit-db
Scout	/var/log/controlzero/scout.log
SSL Proxy	docker compose logs ssl-proxy or /var/log/controlzero/proxy.log
Installer	/opt/controlzero/logs/install.log

Enabling Debug Mode

For detailed diagnostic output, enable debug logging temporarily:

# Set debug mode for all services
czctl set-log-level debug

# Or for a specific service
czctl set-log-level debug --service gateway

Remember to restore the log level after troubleshooting:

czctl set-log-level info

See Logging for detailed log level configuration.