Skip to main content

Recipe: Approval (HITL) for destructive actions

escalate_on_deny does not raise an approval request

The deny fires. A rule tagged escalate_on_deny: true denies exactly as written -- the matched rule's own policy_id, effect: "deny", reason_code: "RULE_MATCH". You are protected.

The escalation does not. The tag is accepted by the policy schema and carried into the policy bundle, and no enforcer acts on it: no approval request is raised, no approver is notified, and decision.requires_approval stays false. Code that branches on decision.requires_approval in order to act on this tag therefore never runs. (A different mechanism, LLM function policies' require_approval, does set that field -- escalate_on_deny is not wired to it.) Tracking: #2391.

To request approval today, call it explicitly. client.request_approval(decision) posts a real approval request. Nothing about the tag calls it for you. See Approval callback. Per #2363 the approver-facing /approvals pages are gated in production, so requests are resolved through the API.

This applies to the published Python SDK (controlzero 1.13.14) and the published Node SDK (@controlzero/sdk 1.13.6) alike.

Status: BETA

Availability

The approval request path for approvals (human-in-the-loop, "HITL") works on every deployment, including the hosted (SaaS) plan. The feature is in BETA, needs the Teams tier, and is off by default -- you turn it on per scope in the dashboard.

The approver-facing pages are not reachable yet. The approvals inbox and request detail routes redirect to the dashboard in every shipped deployment, so an approver cannot resolve a request from the UI today; the request runs to its deadline and the SDK raises HITLTimeoutError with a synthesized deny. For this recipe that means the named destructive operations below still fail closed -- they do not run without a green light. This recipe is the policy shape; the First approval flow walkthrough wires it end-to-end, and Set up approvals is the toggle and cascade reference.

The problem

You want the agent to move freely, but a short list of operations must never run unattended: deleting files, dropping or truncating tables, mass row deletes, and privilege changes (GRANT / REVOKE). For those you want a green-light before it runs -- a human approves the specific call, in context, and only then does it proceed. Everything the operator does not approve must fail closed.

That is a human-in-the-loop (HITL) approval gate, and Control Zero ships it as a per-rule tag plus a request/approve/resume flow in the SDK.

The policy

version: '1'
# HITL approval for destructive actions.
#
# The agent works freely (settings.default_action: allow), but a short,
# named inventory of destructive operations is tagged for human review:
#
# escalate_on_deny: true -> records that this rule is one a human should
# review. The tag is accepted and stored; it does
# NOT route the call through the grants flow --
# your code does that by calling
# request_approval(). See #2391.
#
# IMPORTANT -- what this fixture proves is the fail-closed BASELINE:
# absent an approval backend (or when it is unreachable), an
# escalate_on_deny rule DENIES. The destructive call never runs on its
# own. The runtime approve -> allow / deny / timeout outcomes depend on
# live operator state and are covered by the SDK's HITL integration
# tests, not by this static input->decision fixture. See the recipe page.
#
# The other two knobs stay `deny`: a missing or tampered bundle still
# fails CLOSED, so the approval gate cannot be disabled by an outage or
# by editing the policy file.
settings:
default_action: allow
default_on_missing: deny
default_on_tamper: deny
rules:
- id: approve-rm
deny: 'Bash:rm'
escalate_on_deny: true
reason: 'File deletion needs an operator green-light.'
- id: approve-dd
deny: 'Bash:dd'
escalate_on_deny: true
reason: 'Raw disk writes (dd) need an operator green-light.'
- id: approve-db-drop
deny: 'database:DROP'
escalate_on_deny: true
reason: 'Dropping a table needs an operator green-light.'
- id: approve-db-truncate
deny: 'database:TRUNCATE'
escalate_on_deny: true
reason: 'Truncating a table needs an operator green-light.'
- id: approve-db-delete
deny: 'database:DELETE'
escalate_on_deny: true
reason: 'Row deletion needs an operator green-light.'
- id: approve-db-grant
deny: 'database:GRANT'
escalate_on_deny: true
reason: 'Granting privileges needs an operator green-light.'
- id: approve-db-revoke
deny: 'database:REVOKE'
escalate_on_deny: true
reason: 'Revoking privileges needs an operator green-light.'

Attach this policy to the project (or drop it in your local controlzero.yaml), then turn approvals on for the scope in the dashboard (/settings/hitl).

Why it works

There are two layers, and it helps to keep them separate.

  1. escalate_on_deny: true is a per-rule tag. It records which deny rules you intend a human to review. The tag is additive, and today it is purely declarative: a policy that loads it behaves exactly like a plain deny rule, everywhere, backend or no backend.

  2. The approval flow runs in the SDK, against the backend, and your code starts it. decision.requires_approval stays false on a matched escalate_on_deny rule, so nothing is surfaced as approval-eligible for you to branch on. Your code calls client.request_approval(decision) on the deny; an operator approves or denies it; the call resumes -- see the resolution table below.

The important consequence -- and the thing this recipe's fixture proves -- is the fail-closed baseline: with no approval backend wired in (local-only mode), or when the backend is unreachable, an escalate_on_deny rule simply denies. The destructive call never runs on its own. Approval is the only path from that deny to an allow; there is no path where a missing approver becomes a silent allow.

So rm -rf extracts to Bash:rm and hits approve-rm (RULE_MATCH deny). git commit extracts to Bash:git, matches no rule, and falls through to the allow default (NO_RULE_MATCH). SELECT is not in the destructive inventory, so it is allowed and audited; DROP, TRUNCATE, DELETE, GRANT, and REVOKE each hit their approval-gated rule.

What is approval-gated (and denies without a granted approval)

Agent callExtracted actionDecisionreason_code
rm -rf /var/dataBash:rmdenyRULE_MATCH
dd if=/dev/zero of=/dev/sdaBash:dddenyRULE_MATCH
DROP TABLE usersdatabase:DROPdenyRULE_MATCH
TRUNCATE usersdatabase:TRUNCATEdenyRULE_MATCH
DELETE FROM orders WHERE id = 42database:DELETEdenyRULE_MATCH
GRANT ALL ON orders TO analystdatabase:GRANTdenyRULE_MATCH
REVOKE SELECT ON orders FROM ...database:REVOKEdenyRULE_MATCH

The RULE_MATCH deny is what you observe on every deployment today, with or without an approvals-enabled backend: the tag does not route the match anywhere. Wired to an approvals-enabled backend AND with your code calling request_approval() on the deny, the same match can be taken through the request/approve/resume flow.

What gets allowed (and audited)

Agent callExtracted actionDecisionreason_code
git commit -m "wip"Bash:gitallowNO_RULE_MATCH
ls -laBash:lsallowNO_RULE_MATCH
SELECT * FROM usersdatabase:SELECTallowNO_RULE_MATCH

What the operator and agent see

When approvals are enabled and your code calls request_approval() on the deny, the agent-side SDK drives the request/approve/resume cycle. The low-level API (see First approval flow for the full walkthrough). Note the branch condition: decision.requires_approval is not set by escalate_on_deny, so the code decides which denies are reviewable:

from controlzero import Client, PolicyDeniedError

client = Client()
decision = client.guard("Bash", method="rm", args={"command": "rm -rf /var/data"})

REVIEWABLE_RULE_IDS = {"approve-rm", "approve-dd", "approve-db-drop"}

if decision.denied and decision.policy_id in REVIEWABLE_RULE_IDS:
request = client.request_approval(decision, message="cleanup for TICKET-1")
final = request.wait() # blocks until the operator decides
if final.denied:
raise PolicyDeniedError(final) # denied / timed out -> stop
# approved -> proceed

The operator approves or denies from the dashboard approval queue. The complete audit trail records every resolved request with full lineage (requestor, approver, timestamps, grant id) and distinguishes a call that did not run from one that ran and produced no result.

The request resolves to exactly one terminal outcome. Each carries a distinct machine-readable reason_code so the audit trail shows why a call did or did not proceed:

OutcomeDecisionreason_code
Operator approvedallowHITL_GRANT_APPROVED
Operator denieddenyHITL_GRANT_DENIED
Request expired (server SLA) / caller deadlinedenyHITL_GRANT_EXPIRED / HITL_GRANT_TIMEOUT
Grant revoked after approvaldenyHITL_GRANT_REVOKED
Caller cancelled the waitdenyHITL_GRANT_CANCELED
No API key / backend unreachabledenyHITL_BACKEND_UNREACHABLE
Approver not in the org's approver pooldenyHITL_IDENTITY_NOT_IN_ORG
No approver identity configured on the requestordenyHITL_IDENTITY_REQUIRED
Approved args do not match the resumed calldenyHITL_ARGS_HASH_MISMATCH

Every non-approval outcome is a deny. A timeout, a cancellation, a backend outage, or an unauthorized approver can never be mistaken for an allow -- the flow fails closed on every path except an explicit, in-pool operator approval.

What this recipe's fixture proves -- and what it cannot

The CI fixture at tests/fixtures/enforcement-spec/recipes/hitl-approval-destructive/ runs each scenario through the SDK policy evaluator as a static input -> decision check. That harness can prove:

  • Each destructive rule is approval-gated and denies (RULE_MATCH) when no approval has been granted -- the fail-closed baseline above.
  • Non-destructive calls fall through to the allow default (NO_RULE_MATCH).
  • A tampered bundle still fails closed on an approval-gated action (BUNDLE_TAMPERED), so the gate cannot be edited away.

It cannot drive the live approve/deny/timeout outcomes: those depend on a running backend and a real operator decision, which a static fixture has no way to simulate. Those paths -- approved -> allow, denied -> deny, pending -> timeout -> deny, and the approver-pool / identity gates -> deny -- are covered by the SDK's HITL integration tests: test_hitl_phase2b_protocol.py, test_hitl_6a_request_approval.py, test_hitl_6a_wait.py, test_hitl_6a_get_secret_hitl.py, and test_hitl_reason_codes.py in sdks/python/controlzero/tests/.

Test it yourself

The recipe fixtures are run by the CI recipe driver. From the repo root:

python tests/fixtures/enforcement-spec/recipes/test_recipes.py

Look for [PASS] hitl-approval-destructive in the output. The driver loads this recipe's policy.yaml, replays every scenario in scenarios.json, and asserts the decision, reason_code, and extracted method match -- so this page cannot drift from what the SDK actually does.

Caveats

  • A gate is only as good as the names on it. With default_action: allow, anything you did not tag is allowed. New tools, renamed binaries, and creative arg shapes get through. Enumerate the destructive inventory deliberately, and prefer the allow-list (default_action: deny) posture when you can name the small set of operations the agent legitimately needs -- see Read-only database.
  • The tag never starts the request/approve flow -- your code does. escalate_on_deny is accepted and stored and no enforcer acts on it (#2391), so a matching call is a plain deny even with the per-scope toggle on. Two things are required, and the tag is neither: an administrator flips the per-scope toggle in the dashboard, and your code calls request_approval() on the deny.
  • Without a backend, this is a deny-list, not an approval flow. In local-only mode there is no approver to route to, so an escalate_on_deny rule denies. That is the intended fail-closed behavior -- but it means the interactive approval only exists when the SDK is connected to an approvals-enabled backend AND your code raises the request.
  • Shell and SQL gates are extractor-based, not a sandbox. A gated Bash:rm covers the rm that the hook extractor surfaces from the command. Pair this recipe with OS-level controls for defense in depth.

Read the Approval Workflow concept for the architecture, and Enforcement Behavior for how default_action and the fail-closed knobs resolve.