Bounded Autonomy: How to control AI agents
Full AI autonomy is a recipe for disaster. Here's a framework that gives you control without killing the value.
Full AI autonomy looks great on a slide. In production, someone ends up putting out fires. Success with AI comes down to the balance between freedom and control. I call it Bounded Autonomy.
What is Bounded Autonomy?
Bounded Autonomy is a framework built on the principles of the NIST AI Risk Management Framework. An AI agent gets clearly drawn boundaries. Inside them it has full freedom. Outside them it stops and hands the case to a human (human-in-the-loop).
It's like giving your teenage son the car keys. You tell him: "You can drive around town, but not on the highway. Be home before 10."
The 4 pillars of Bounded Autonomy
THE BOUNDED AUTONOMY FRAMEWORK
-
1. Operational Limits
What can the agent do? Which actions can't it take on its own?
-
2. Escalation Triggers
When must the agent hand the decision over to a human?
-
3. Audit Trail
How do we document every decision the agent makes?
-
4. Kill Switch
How do we shut the agent down instantly when something goes wrong?
This is what I do hands-on — advising on AI strategy and building agents that survive the demo.
Operational Limits in practice
Example for a customer service agent:
ALLOWED ACTIONS
- ✓ Answering product questions
- ✓ Checking order status
- ✓ Issuing coupons up to 50 zł
- ✓ Updating contact details
REQUIRING ESCALATION
- ✗ Refunds over 500 zł
- ✗ Legal complaints
- ✗ Contract changes
- ✗ Customer explicitly asks for a human
Escalation Matrix
Not all escalations are equal. Build a matrix:
- Level 1 (Soft Escalation): The agent keeps going but flags it for review later
- Level 2 (Human Review): The agent waits for approval before it acts
- Level 3 (Full Handoff): A human takes over completely
- Level 4 (Emergency Stop): The agent is halted and an incident is reported
"Bounded Autonomy builds trust, and trust is what lets you scale. Give an agent clear boundaries and you can give it more freedom. You know it won't step over the line."
Governance Agents
The most advanced companies take it a step further. They run AI agents that watch other agents. A Governance Agent checks:
- • Are decisions aligned with company policy?
- • Are there any anomalies in behavior?
- • Is performance degrading?
- • Are escalations being handled on time?
Implementation: step by step
- 1. List every action the agent can take
- 2. For each action, decide: auto, review, or forbidden
- 3. Define escalation triggers (value, risk, sentiment)
- 4. Build an audit log for every decision
- 5. Test the edge-case scenarios
- 6. Set up alerting for anomalies
Where to start
Start with one table: the actions the agent takes on its own, the ones that need human approval, and the ones it's never allowed to touch. The whole framework grows around that list. The companies that build it scale AI calmly. The rest learn the hard way.
