AI can't check its own homework
An AI model reviewing its own answer can make it read better, but only a check against something outside the model, such as the source document or the ledger, verifies it.
Ask an AI model to check its own answer and it will almost always find something to improve. The second version reads better and sounds more certain, which helps when I'm drafting. If the first answer was wrong, though, the revised answer is usually still wrong and now more convincing, because the check uses the same model and the same information that produced the mistake.
This series asks what still counts as audit evidence when AI does the work. The first essay was about repeatability: we can't keep a copy of the live data an AI tool reasoned over, so we keep and test the method. The second was about completeness: testing every record tells us only that the procedure ran on what it received, so we need evidence that the population was whole. This one is about independence. Evidence counts for more when it comes from a different place from what already provided the answer, and AI checking AI fails that test in a way that is easy to miss. Still to come are reproducibility, how long assurance lasts when an AI procedure never stops running, and what changes for Audit itself when it can see more.
We usually catch mistakes because something stops making sense. When a number surprises us or doesn't fit the logic we expect, we dig deeper, and we might reconcile it to the last figure that made sense. It's like using an Excel model someone else built. It has always worked and the results have looked sensible, so nobody looks closely until the outputs start to look noticeably off. Then someone digs in and finds that a cell was hard-coded or a formula was wrong, sometimes for a long time.
An AI model reviewing its own work makes this kind of error harder to find, because the review comes from the same model, usually working from the same documents that produced the answer. If the answer rests on a misreading of a contract clause, the review reads the clause the same way, and the revised answer comes back better written and more confident than the first. That makes the misreading more persuasive to whoever reads it next, and it removes the surprise that would have made them dig in.
Reasoning models now go back over their reasoning before they answer, but that rarely changes the answer. In October 2025, researchers at the National University of Singapore and MiroMind AI studied eight of them on math problems and found that more than 90 percent of the reflections that came after a model's first answer left that answer unchanged. Older models did worse when they checked themselves. In a paper presented at ICLR 2024, researchers from Google DeepMind and the University of Illinois found that without outside feedback, accuracy fell after self-correction for every model and benchmark they tested, and it improved only when the models were told which answers were wrong. I expect self-checking to keep improving, but a model that checks its reasoning more carefully is still checking it against itself.
The obvious fix is to add a second model, or a group of agents that review or debate one another. The UK's Financial Reporting Council's March 2026 guidance on AI in audit lists one model reviewing another's work, or the same model prompted with different priorities, among the ways to reduce errors in AI output. In audit terms that looks like a second reviewer, and sometimes it helps. A second reviewer is useful because they are independent of the first, and models often aren't. Researchers at Cornell evaluated more than 350 models and presented the results at ICML 2025. On one leaderboard dataset, when two models both got a multiple-choice question wrong, they chose the same wrong answer about 60 percent of the time, and larger, more accurate models had more highly correlated errors, even across providers. The shared error usually has a more ordinary cause, too. Two agents reading the same extracted data will agree about it whether or not the extraction was right.
AI review still has a place, and I use it every day. I use prompts and skills that ask my agents how something could fail, including adversarial agents set up to argue against an answer, and I always ask for a rescan for what was missed. Those reviews often find something worth checking. What they find tells me where to look, and the check itself has to come from somewhere outside the model. People tend either to assume AI can check itself or to distrust it and want a person to review everything. In my experience AI is right more often than people when its instructions and data are clear, and neither approach asks where the check came from.
The PCAOB's audit evidence standard already says that evidence from a knowledgeable source independent of the company is more reliable than evidence obtained only from internal company sources. We need to apply the same reasoning inside an AI-enabled procedure. If a model says a contract contains a termination clause, retrieving the text and comparing it settles the question. If a model calculated a total, it should agree to the ledger, and we can repeat the calculation in code, which does the arithmetic instead of predicting what it should produce. Where the answer depends on facts outside the company's own systems, the strongest check is evidence the model never saw, such as a third-party confirmation.
Self-review tends to take the place of these checks because it is fast and needs nothing beyond the model that already has the answer. A source-based check needs access to the source, which in a large company may sit in another system with its own permissions, and someone has to decide in advance what each kind of answer should be compared against. That design work is easy to skip when self-review is already producing a tidier answer. Once we build deterministic checks, they run at machine speed too, so an AI-enabled procedure doesn't have to give up its speed to be verified.
For a draft or a set of hypotheses, self-review is useful, and improving the answer is the point. When management or an audit team relies on an AI output, "the model reviewed its answer" shouldn't count as a verification step. We need a record of what each answer was checked against, and whether that source could have been wrong in the same way. Next week I'll look at a related question: whether an AI procedure has to produce the same result every time it runs.
Continue the conversation. I read and reply to comments on Substack. The sharpest responses usually come from readers.
Get the essays in your inbox
Clear Calls publishes on Substack. No promotion. Unsubscribe anytime.