We want AI to be autonomous. We just haven’t accepted what that means.
We want agents so we can delegate to them. Then we put a person back in front of every decision that matters, and assume that protects us.
One of the things people find unnerving about AI agents is their autonomy. I understand the concern, but there is something inherently contradictory about it. We want the benefit of delegating work to an agent, then try to manage the risk by putting a person back in front of every important decision.
We would never manage people that way.
Think about bringing someone new onto your team. At first, you explain the work and the context around it, answer questions, review what they produce, and check decisions that six months later you would expect them to make on their own. As they demonstrate good judgment, you give them more room. Eventually, their ability to operate without you is one of the clearest signs that they are good at their job. Nobody hires a capable person hoping to approve every action they take forever.
Yet as we start deploying AI agents, we are often trying to have it both ways. We want them to take on work, increase capacity and move much faster than people can, but we become uncomfortable when they act without a person approving what they do. For consequential decisions, the obvious control is to keep a human in the loop.
There are certainly decisions where I want a person involved. I am less convinced that the existence of a human approval gives us as much protection as we sometimes assume.
I have spent enough of my career in finance to be skeptical of approval chains. Think about an invoice or document that passes through several levels of approval. On paper, the process can look very well controlled. Six people approved it. But how many of them independently reviewed the underlying work?
Often the more senior person is relying, quite reasonably, on the work done below. You have an excellent employee who understands the process and consistently does good work. They bring something to you and tell you they have reviewed it carefully. You still look at it, but some of your confidence in the document comes from your confidence in the person bringing it to you.
It would be impossible to run a large organization if every manager completely re-performed the work of every person below them. Delegation depends on trust.
I still remember a fraud case from years ago because it brought this home in a very simple way. A manager told us he had trusted the employee who committed the fraud completely. The long-tenured employee was good at his job, and the manager almost considered him a friend. He had no reason to doubt him.
One day, the employee brought him a spreadsheet for approval. The manager reviewed it and signed it. The spreadsheet was accurate except for one missing line.
That line was the fraud.
Almost everything on the spreadsheet was right, and it came from someone who had earned the manager's trust. That case has stayed with me because trust wasn't a flaw in the process. Trust was part of how the process worked. We could not run organizations without it. But several human approvals should not automatically give us confidence that several people independently reviewed the work.
I think the same issue will emerge as we decide how to control AI agents.
“Human in the loop” sounds reassuring because a person is still there to catch a mistake. But putting a person into a workflow tells us very little about what that person is actually doing. If an agent is wrong frequently, people will check it closely. After it has been right hundreds or thousands of times, I suspect people will begin to trust it. We already do that with people, and there is no reason to assume our behavior around consistently reliable technology will be completely different.
The errors may also become harder to spot as the technology improves. The spreadsheet in my fraud example wasn't obviously wrong. It was almost entirely right. The problem was what wasn't there.
When we use human review, we need to be precise about what we expect the person to do. For a high-consequence decision, prior approval may well be the right control. The reviewer needs the information to form their own judgment and some understanding of what should cause them to disagree with the agent. An approval that is almost never withheld deserves some scrutiny of its own.
Most organizations already know a great deal about managing delegation. We give people defined authority. Someone may make routine decisions on their own and need approval above a certain threshold. We separate responsibilities where one person should not control the entire process, and we supervise someone new to a role more closely than someone who has demonstrated that they can do the job.
We can draw on that experience with agents. We need to decide what an agent can do independently, where approval adds value and where an action should remain outside its authority. The consequence of getting something wrong matters, as does how easily the action can be reversed and how well we understand the agent's performance in that particular context.
As agents take on more work, some controls will need to move. When we give up the opportunity to approve an action before it happens, we need other ways to see whether the agent is operating as expected. We can limit its authority, separate responsibilities, monitor activity and intervene when something moves outside the boundaries we intended.
Speed makes that especially important. Someone who misunderstands a process might make the same mistake several times before anyone notices. An agent could repeat it thousands of times. We need to be able to see the problem and stop it quickly.
There is a separate question about what happens to human judgment as agents do more of the underlying work. People who spend less time doing the work themselves may eventually find it harder to recognize when something is wrong. I think that deserves an essay of its own.
For this one, I keep coming back to the missing line in the spreadsheet. The manager signed the document. The employee had earned his trust. The process included the approval it was supposed to include. None of that made the missing line visible.
The fact that a human was involved doesn't actually mean that the human exercised judgment.
We want agents because we want to delegate work to them. If they have to stop and wait for us every time the work matters, much of the benefit disappears. That puts more weight on the boundaries we set, what we can see once the agent starts acting, and how quickly we can respond when something goes wrong.
Some decisions will still need prior human review. Others will depend on clearly defined authority and good monitoring. We already know that an approval doesn't necessarily mean meaningful review occurred. We shouldn't build the next generation of controls around the assumption that it does.
Continue the conversation. I read and reply to comments on Substack. The sharpest responses usually come from readers.
Get the essays in your inbox
Clear Calls publishes on Substack. No promotion. Unsubscribe anytime.