The usual way a team sets the autonomy boundary for an AI agent is to write it into the instructions: never send anything to a customer without asking me first. That instruction holds until the first turn where the model decides that asking would be unhelpful, and then it is gone, quietly, with no error and no log line. A rule an AI agent can reason its way around is not a control, it is a preference.
An AI agent should stop and ask a person before any action that is hard to reverse or wide in blast radius: anything that leaves the company, moves money, deletes or overwrites records, or changes permissions. Narrow, cheap-to-undo actions run freely and are logged. The gate belongs in the execution path, not in the prompt.
What is an approval gate for an AI agent?
An approval gate is a rule enforced in the execution path that blocks a defined class of AI agent actions before they run and holds each one until a named person approves or rejects it.
Three things get sold under one heading and only one stops anything. A gate prevents: the action does not happen. Observability records what an AI agent did after the fact. Evaluation measures behaviour on a test set before you ship. That reading is ours, confirmable from any vendor's own documentation.
Escalation is the fourth word in the pile: the AI agent asking because it is unsure. A gate stops it even when it is sure. Confidence is the wrong trigger, because a confident agent never escalates and the turns that hurt are the confident ones. Confidence-based escalation is a courtesy, not a control.
One boundary: this piece is about which AI agent actions stop, not how the approval screen is drawn.
Why is a rule written in the prompt not a guardrail?
Because a system instruction is an input to a probabilistic system, not a branch in a program. It competes with the user's last message, retrieved documents and tool output, and on an ordinary turn it loses.
The failure is almost never disobedience. It is reinterpretation: the model decides this case is the exception the rule did not anticipate, and proceeds. Nothing in the transcript looks like a violation, so nobody catches it. A control you can argue with is a preference, and anything reaching the context window joins the argument: a page from the open web, a customer's attachment, a record a third party wrote.
Fortune reported on 23 July 2025 that an AI coding tool wiped a company's production database during a code freeze, after being told not to act without approval, with development and production not separated. The instruction was there. The gate was not.
System instructions are not useless. They are the right home for what a gate cannot express: how the AI agent behaves when blocked, how to ask for approval, what to tell the person waiting. They are just not the layer that says no.
What two properties decide whether an AI agent action needs approval?
Two: how reversible the action is, and how wide its blast radius is. How complex it looks, how capable the AI agent seems and how much the team trusts it are noise. Score the action, not the agent.
Reversibility is a three-step scale, anchored to concrete cases. Seconds and free: a task you close, a draft you delete. Hours and effort: a record restored from backup, a booking re-made. Impossible: a message someone has read, money out of the account, a dropped table.
Blast radius asks how many people or records the action touches, and whether the effect leaves the company. What leaves cannot be recalled from the recipient's memory, however thoroughly you delete your side.
Both axes matter, because either alone gets a case wrong. A message to every employee is wide, internal, deletable and already read. A refund to one customer is as narrow as an action gets and cannot be undone. Both belong above the line.
Ignore difficulty: the most dangerous AI agent actions are usually the simplest, because simple actions are the ones an agent completes. Deciding whether to hand over a whole job is separate scoring; this grades the actions inside it. The working rule: name the action, ask how long undoing takes, ask how far the effect travels, mark whether it crosses the company boundary.
The three tiers: which AI agent actions run, which get logged, and which stop
Sort every action an AI agent can take into three tiers: Tier 1 runs freely, Tier 2 runs and is logged as a change, Tier 3 stops for a named person. The three-tier matrix is the argument in one place.
| Action class | Reversibility | Blast radius | Tier | Worked example |
|---|---|---|---|---|
| Reading a record, searching, reading a document | No state change | Narrow, internal | Tier 1 | An order checked before replying |
| Drafting text, calculating, opening an internal task, writing its own notes | Seconds, free | Narrow, internal | Tier 1 | A draft, a follow-up for tomorrow |
| Creating or updating a record, reserving a calendar slot, uploading a file | Hours, tedious | Narrow to medium | Tier 2 | A delivery date from the carrier |
| Posting to an internal channel | Deletable, but read | Wide, internal | Tier 2 | A shift summary to a team |
| Spending under a declared per-job ceiling | Recoverable in accounting | Narrow | Tier 2 | A metered fee for one lookup |
| Anything that leaves the company | Not reversible once received | Wide, external | Tier 3 | Email, SMS, a public post |
| Moving money, refunds and payments | Hours to never | Narrow but external | Tier 3 | One refund to one customer |
| Deleting records, changing a schema | Often impossible | Wide, internal | Tier 3 | A table dropped, a list purged |
| Changing permissions or roles | The open window is the damage | Wide, internal | Tier 3 | A capability granted to an agent |
| Committing to a price or contract term | Binding once stated | Narrow but external | Tier 3 | A discount quoted in writing |
| Writing a third party's personal data outward | Not reversible | Wide, external | Tier 3 | Contacts exported to a vendor |
| Exceeding a declared cost or volume ceiling | Depends what ran | Wide | Tier 3 | A run messaging every contact |
Write the tie-break rule down before you need it, because you will argue about it under pressure: an action between two tiers goes in the higher one, and moves down only on dated evidence from your log.
Tiers belong to action classes, not tools, because tools get replaced and renamed. "Anything that leaves the company" survives that; "the messaging capability we installed in March" does not.
The actions that should never run, even with approval
Some AI agent actions never enter the matrix, because approving them is a mistake in every case rather than a judgement call. Entering a credential or payment instrument anywhere. Turning off or reconfiguring the gate. Granting itself, or another agent, a permission it does not hold. Bulk deletion.
Why they differ in kind is worth saying plainly: an action that damages the control surface is not the same kind of action as one that damages data. A deleted record is a bad day. An AI agent that can widen its own permissions has removed your ability to review whatever it does next.
The practical form matters as much as the list. These do not sit in the toolset waiting to be blocked; they are not in the toolset. Deny by default means the capability is absent, not present and discouraged, and absence is the only version of the rule nothing can argue with.
How do you classify a tool you have never seen before?
Classify an AI agent's tool by what its action does, not by who supplied it. Third-party tools keep changing, so the rule must survive tools nobody on your side has read.
First method: verb matching on the action name. Actions that send outward (send, notify, email, message, post, publish, share, broadcast) and actions that destroy (delete, drop, purge, revoke) default to Tier 3. Cheap, fast, wrong at the edges.
Second method: the tool's own declaration. A tool interface can carry a machine-readable annotation saying an action only reads. Prefer it over guessing from the name, but treat it as a hint, not proof, because the tool's author supplies it. RFC 9110, HTTP Semantics (June 2022), settled this for the web: calling a method "safe" does not prevent an implementation from including "behavior that is potentially harmful, that is not entirely read-only, or that causes side effects".
Our first version matched action verbs against the naming convention our own capabilities use. A third-party capability named its action differently, slipped past the gate, and nothing errored, which is why nobody noticed. The fix was to normalise the action name before matching, and to prefer the tool's declared read-only annotation over guessing while still treating it as a hint. Before shipping we replayed the change over the real tool names already in our action log and confirmed no existing classification changed while the third-party gap closed.
That replay check is the transferable part, and the failure it caught is the one to fear: the gate did not error, it did not fire. Run any classification change over the tool names your log holds and count how many move; nothing moving is suspicious, everything moving is a bug. If name and declaration leave you guessing, it is Tier 3 until somebody reads the documentation.
Where does the approval gate have to live?
Inside the tool execution path, where the call is made, so refusing is a code path and not a judgement. The model layer decides what to attempt; the execution layer decides what happens, and only the second says no reliably.
The refusal must be structured, not an exception: an object saying it was blocked, why, and what would unblock it, so the AI agent asks a person instead of failing blind or retrying.
Under the gate sits deny-by-default authorisation, so a capability the AI agent was never granted is not in its toolset at all: by role, a permissions matrix. Over it sits a hard per-job ceiling. A cost cap with a running spend counter is a mechanism we implement, and it keeps a runaway loop from becoming an invoice; a volume cap we recommend as advice, not as our own.
In our own runtime the gate is a function in the tool execution path, not a line in the system prompt. Once a turn has taken in outside content, an attached file or a tool that reads, fetches, scrapes, crawls or downloads, any tool that sends outward is refused in code and routed to a person, with a structured object rather than an error so the agent can ask instead of failing blind. The honest cost of drawing the line at outbound: record writes such as updating an order are deliberately not gated, because that would break the ordinary case of read this invoice and update the order. We made the trade on purpose.
Drawing the line at the action class rather than at intent produces false positives, so you will sometimes stop something that was fine. Make that decision explicitly rather than inherit it.
What should the approver actually see?
Enough to decide in under a minute without opening another system. An approver who has to go elsewhere to check something starts approving without checking, and you have a signature, not a control.
An approval request needs six things:
- What the AI agent is about to do, in plain language.
- The literal payload: the message text, the query, the amount.
- The scope it will run against.
- Who or what asked for it.
- Why the agent believes it should happen.
- What happens if it is rejected.
In our own production approval records there is a request in which an agent was asked to delete all records and drop a table, and it stopped and waited for a person. The useful part is not that it stopped; it is what the waiting request held: the literal instruction, the tool that would have run it, and the scope it would have run against.
Two ways of getting this wrong produce one signature. Too little context and the approver rubber-stamps; too much raw output and they skim. Either way, a very high approval rate paired with a very short time to decision. That is the rubber-stamp signature: the gate is there and has stopped meaning anything.
Approvals need an identity: who approved, when, and whether they edited the payload. An approval nobody can attribute is not evidence of oversight, and who answers when it was wrong is a separate question about roles and accountability. Requests must reach whoever owns the action class, which means a supervising layer that assigns work, routes approvals and reports outcomes, not a shared inbox. Log rejections with a reason: a rejection says the tier is right and the agent's judgement wrong, or the reverse.
What happens to a half-finished AI agent task while it waits?
The job has to be a durable, resumable object with a state, not a paused conversation. Pending work that exists only inside a live session dies with the session, and nobody finds out until the thing that should have happened did not.
A gated AI agent job has three states, each meaning something different to the person waiting. Waiting on a person: somebody owes it a decision. Expired: nobody answered in time. Rejected: somebody said no. All three must be visible where a human looks, not only in a log.
Pending work goes stale. An approval to send a customer a delivery update is worthless four days later, and approving it then is worse than rejecting it: you have sent a confidently wrong message with a human signature on it. Every gated action needs a validity window, and when it closes the request expires rather than finally running.
Waiting is where scheduled and unattended AI agents break, because nobody is in the conversation to ask. A recurring job whose approver and window were never assigned stops producing quietly: nothing errors, nothing alerts, the report stops arriving. If a job's value depends on being on time, either pre-approve that action class or do not schedule it. A gate on a time-critical unattended job is a scheduled failure.
How do you resume without repeating the side effects that already succeeded?
Make the unit of approval a single identified action rather than the whole job, so resuming re-runs only the step that was blocked.
The concrete failure: an AI agent job performs three steps, the fourth hits the gate, and a naive resume restarts from the beginning. The first three side effects happen twice. Two invoices, two calendar entries, two messages to one person, and the approval you were careful about is not what hurt you. The fix is small: every step with a side effect carries an identifier and a completion record, and a resumed run skips what succeeded. That is the discipline behind safe retries, applied to approvals.
One variant deserves a warning. When a system folds tool calls together for efficiency, the real action can end up buried in the arguments, so a write looks side-effect-free to the classifier and gets retried. Resolve the real action before deciding whether repeating it is safe.
The readiness test is one question: what happens if this job resumes twice? If you cannot answer it for a workflow, that workflow is not ready to sit behind a gate.
Why should reading outside content change what an AI agent is allowed to send?
Because from the moment a turn takes in content you do not control, everything the AI agent does afterwards may be happening on somebody else's instruction. The boundary is one sentence a non-engineer can repeat: content an AI agent reads is data, never an instruction. Instructions come from the system configuration and the person being spoken to.
The trigger is the part most teams never wire into anything, and the design is dynamic rather than static. A turn that has fetched a page, crawled a site, opened an uploaded file or read a third-party record moves outbound actions up one tier for the rest of that turn. The permission belongs to the turn, not the AI agent.
Outbound specifically, because outbound is where injected instruction becomes consequence outside your walls. An injection that makes an AI agent write a wrong internal note is a defect. One that makes it email a stranger is an incident, and the difference is not the model's behaviour but which tool was reachable.
Two measures for retrieved content. Fence the region, so a poisoned record cannot fake the end of the fence and issue instructions in the agent's voice. And have the agent report any instruction it finds inside content rather than ignoring it silently, because silent refusal hides an attack in progress.
These are named risk classes. The 2025 edition of the OWASP Top 10 for Large Language Model Applications lists LLM01:2025 Prompt Injection, including the indirect case where a model takes input from external sources such as websites or files, and LLM06:2025 Excessive Agency, whose root causes it gives as excessive functionality, permissions and autonomy.
How do you keep the gate from becoming a queue nobody clears?
Measure two numbers and act on them. The unmodified approval rate is the share of gated AI agent actions a person approved without editing the payload. The median time to decision is how long a request waits between being raised and answered.
A very high unmodified approval rate means the class is in the wrong tier and should move down: people are not deciding, they are clicking. A long median time to decision means the bottleneck is the queue rather than the AI agent, and a process meant to save time is slower than the work by hand. What a review step costs at volume is arithmetic for its own piece.
Tiering is a living document: put a review date on it, move classes down on evidence and up on incident, date every change. Do not tier by department or seniority. Tier by action, then assign the approver to the action class, because an approval routed to the most senior person rather than the one who understands the payload gets rubber-stamped.
The honest limit: this part of the design erodes fastest, and invisibly. A gate that has quietly become a formality looks exactly like one that works.
What does demonstrable human oversight look like to a reviewer?
Four artifacts: a written scope of authority, a named responsible person, a policy saying which AI agent action classes stop, and a log of what ran, what was refused and who approved what. A claim is not oversight. Evidence is oversight.
Cite the recognised frameworks precisely rather than gesturing at them. Regulation (EU) 2024/1689, the Artificial Intelligence Act, published in the Official Journal on 12 July 2024, sets human oversight obligations in Article 14 for systems it classifies as high risk, and whether a given deployment falls in that class depends on the use case, so scope is settled before obligation. Among the things the system must let an assigned person do:
to intervene in the operation of the high-risk AI system or interrupt the system through a 'stop' button or a similar procedure that allows the system to come to a halt in a safe state.
Regulation (EU) 2024/1689, Article 14(4)(e)
The second anchor is procedural. The NIST Artificial Intelligence Risk Management Framework (AI RMF 1.0, NIST AI 100-1, January 2023) treats oversight the way it treats everything else it asks for: MAP 3.5 says processes for human oversight are "defined, assessed, and documented". That is a higher bar than intended. Who answers when an AI agent gets something wrong is a separate problem, and autonomy is not a defence; that argument sits in the piece on writing an agent's scope of authority.
The useful translation: the artifacts a reviewer wants are the artifacts a manager wants. You are writing down how the work runs, not building compliance theatre. Disclosure belongs in the same document: if a gated AI agent action is customer-facing, decide in advance what the customer is told about who produced it. None of this is legal advice.
How do you set an AI agent approval policy without engineering help?
Nine steps. The first six are the whole intellectual job and need no engineering help; seven and eight are a request to whoever owns the runtime.
- List every action your AI agent can take, from its tool list rather than from memory.
- Score each for reversibility: seconds and free, hours and tedious, or impossible.
- Score each for blast radius, and mark whether the effect leaves the company.
- Assign a tier from the three-tier matrix, putting every unclear case in the higher tier.
- Write the Tier 3 list on one page, a named human owner beside each action class.
- Set a validity window per gated class, so an expired request drops instead of running late.
- Ask whoever owns your runtime to enforce the Tier 3 list in the execution path rather than the system instruction, returning a structured refusal instead of an error.
- Ask for a per-job cost cap, and for a per-job volume cap as well.
- Diary a date to review the unmodified approval rate and the median time to decision.
Then run the closing test. Pick an action that is supposed to be gated, ask the AI agent to do it, and watch whether it stops. A gate nobody has tested is a gate nobody has.
None of this makes an AI agent less capable. It makes the boundary of its autonomy something you wrote down rather than something you found out. The regulation, risk framework and specification cited above were checked against their primary texts on 2 July 2026, and all three change. If you want a starting point other than your own tool list, a short assessment we publish returns a score, an archetype and an estimate of the hours a team could recover.
Frequently asked questions
What is human in the loop in AI agent design?
Human in the loop means a person approves or rejects specific AI agent actions before they take effect, rather than reviewing them afterwards. It is a design decision about which action classes stop, who the named approver is, what that person sees, and how long a paused action stays valid.
Is an approval gate the same thing as a guardrail?
Guardrail is a loose word covering content filters, refusal training, monitoring and permissions. An approval gate is one specific mechanism: a defined class of AI agent actions is blocked in the execution path and held for a named person. Ask which of those a supplier means before you rely on it.
How is an approval gate different from role-based permissions?
Permissions decide what exists to be done at all: a capability the AI agent was never granted is absent from its toolset. The gate decides which of the remaining actions may run unattended. Permissions are static and set per role. Gates are set per action class and can tighten with the context of a turn.
Do read-only AI agents need approval gates?
Less urgently, but yes, for two reasons. Reading still has a blast radius when it pulls confidential or personal records into a summary that then travels. And read-only rarely stays read-only: the request to also update the record arrives quickly. Decide the tiers before the write capability is added, not after.
Can one AI agent approve another AI agent's action?
Not for a gated class. Delegation between AI agents moves work, not accountability, and a chain of automated approvals produces an audit trail with no person in it. Route the approval to the named human who owns that action class, whichever agent in the chain actually raised the request.
Does the EU AI Act require human approval of AI agent actions?
Not universally. The regulation sets human oversight obligations for systems it classes as high risk, including a person being able to intervene or interrupt. Whether a given deployment falls in that class depends on the use case, so read the text rather than assuming. Designing gates around reversibility serves the intent either way.
