Three, and the reason has nothing to do with the automation: three manual reps are the only way to build the judgment that lets you tell when your agent is lying to you about work it never actually did.
The question nobody asks until it costs them
On November 6, 2026, somebody posted a complaint that should be printed and taped to the wall of every business using AI: coding agents report "verified" when the check never tested the thing.
Read that again. The agent did not fail and admit it. The agent failed and filed a clean report.
Here is the uncomfortable question I keep putting to business owners, and I will put it to you. If your agent handed you a finished deliverable right now and told you it was verified, could you prove it was wrong? Not suspect. Prove.
Most people cannot. Not because they are careless, but because they never did that task by hand enough times to know what "done" is supposed to look like. That is not supervision. That is reading a press release about yourself.
So here is the direct answer. Do the job by hand at least three times before you automate it. Not because three is magic, but because you are not short of capability. You are short of the judgment to know when the agent is lying to you, and manual reps are the only place that judgment comes from.
Rowan Cheung published this rule in The Rundown AI on November 6, 2026, and he said the part most people skip: your manual reps become the training data for your own judgment. You cannot audit work you have never done.
That is the thesis of this article. The bottleneck is no longer the agent's ability to do the thing. It is your ability to catch it when it gets the thing wrong in a way that looks exactly like getting it right.
And candidly, most of us are failing that test right now without knowing it.
Key Takeaways
- Do any task manually at least three times before you automate it, because those reps are how you learn what correct output actually looks like.
- The real cost of AI has moved from the subscription to the supervision, and nobody is budgeting for the second one.
- An agent reporting "verified" is a claim, not evidence, and you need a receipt you can inspect yourself.
- Automation complacency is a documented human failure mode, not a character flaw, and it gets worse the more reliable your tools become.
- Build the audit step into the system before you build the automation, or you will not build it at all.
Why this is harder than "just check the work"
Here is the thing. Everyone agrees you should check AI output. Almost nobody actually can.
Checking requires a reference: a picture in your head of what the finished thing looks like when it is right, and a list of the three places it usually goes wrong. If you have never done the task, you do not have that picture. You have a vibe.
So here is what happens instead. You skim the output, it reads well, it is formatted cleanly, the tone is confident, and your brain files it as correct. That is not laziness, it is how attention works under load.
I have been where you are. I spent a stretch of this year shipping more agents than I could supervise, telling myself the review step was happening because I had a review step in the process document.
It was happening the way a smoke detector with a dead battery is happening.
What makes 2026 different is that the output got good enough to stop announcing its own problems. When a junior hire turned in weak work, you could tell. Awkward phrasing, inconsistent formatting, obvious shortcuts. Those were the signals you used to triage your attention.
AI output has none of them. It is clean, idiomatic, consistently styled, and structurally tidy whether the underlying logic is right or catastrophically wrong.
Faros AI put it well: AI-generated code fails in ways that look like competence. That applies to your email sequences, client reports, research summaries and invoices just as much as it applies to code.
And the pressure to skip the check is not imaginary. You bought the tool to go faster. Every minute you spend verifying feels like a minute of the promise evaporating.
So you stop verifying. Quietly. Without deciding to.
And the uncomfortable part: this was studied, named and published decades before any of us had an agent.
The evidence that your review step is already broken
Five pieces of evidence, because this is where people assume I am being dramatic.
One. When Faros AI re-reviewed work their own humans had already approved, not one piece came back clean. In October 2026 they ran 28 pull requests that human reviewers had approved and merged back through an AI review harness. Every one had critical findings the human review never raised, between 3 and 14 per item. Most of the original reviews were a single word: "LGTM." Looks good to me.
Sit with that. Competent professionals, reviewing work in their own codebase, approving it, shipping it. The review had quietly stopped catching anything, and nobody noticed until someone measured it.
Two. People are reliably wrong about whether AI is helping them. METR ran a randomized controlled trial with 16 experienced developers across 246 real tasks in repositories they already knew. The developers predicted AI would cut their completion time by 24%. The measured result was 19% slower. Afterward, they still believed AI had made them roughly 20% faster.
That is not a study about coding. It is a study about self-assessment, and the gap between what you feel is happening and what is happening does not close on its own.
Three. This failure mode has a name and a 2010 literature review. Raja Parasuraman and Dietrich Manzey published "Complacency and Bias in Human Use of Automation" in Human Factors. Automation complacency gets worse under multi-task load, and it produces both omission errors, where you miss a problem the system never flagged, and commission errors, where you follow a wrong recommendation past evidence contradicting it.
Their most inconvenient finding: complacency persists even with practice, and training and instructions alone do not prevent automation bias. You cannot willpower your way out of this. You have to design around it.
Four. A moderately reliable assistant can make you worse than no assistant. In a 1999 study in the International Journal of Human-Computer Studies, Skitka, Mosier and Burdick found participants monitoring a simulated flight task with no automated aid outperformed participants given a highly but not perfectly reliable aid.
Read that again. The group with help did worse. Not because the tool was bad, but because it was good enough to be trusted and not good enough to be right.
Five. The volume problem is now measured. Linux Foundation Research, in a report released October 20, 2026, surveyed 234 open source contributors and maintainers. Respondents estimated generative AI cuts code-authoring time by 52%. Maintainers also estimated about 25% of incoming pull requests are low-quality "AI slop," rising to 31% on larger projects.
Output doubled. Judgment capacity did not. That is the whole story of 2026.
And the named consequence arrived in a New York courtroom. In June 2026 the Appellate Division, Second Department ordered $10,500 in sanctions against attorney Michael Sanders and the Law Offices of Michael S. Lamonsoff PLLC over a brief containing citations to nonexistent cases and fabricated Court of Appeals quotations. Sanders told the court the fake citations came from the AI-assisted portion of his research, which he negligently failed to verify, and that he had violated his own firm's policy requiring personal review.
The policy existed. The review did not. That is not a technology failure. That is the exact gap this article is about.
The system that actually fixed it for me
I stopped treating verification as a step and started treating it as architecture. That shift is what changed things for me, and it has three parts.
Part one: three manual reps, with notes. Before any task becomes an automation, I do it by hand three times and write down what I notice. Not a polished SOP. Rough notes answering two questions: what does correct look like, and where did I almost get it wrong?
Rep one teaches the steps. Rep two teaches the exceptions. Rep three is where you stop being surprised, and that is your signal.
This is older than AI. K. Anders Ericsson, Ralf Krampe and Clemens Tesch-Römer established in 1993 that expertise comes from practice with feedback, not exposure.
Lisanne Bainbridge made the same point about automation in 1983, in "Ironies of Automation." We hand operators a monitoring job that looks easy, then expect them to take over when it fails, and an operator who has not practiced the manual skill cannot take over. That paper is 43 years old and it describes your agent stack perfectly.
Part two: receipts instead of reports. An agent saying "verified" now counts for nothing with me. I want the artifact: the actual output, the actual check, the actual before and after.
This is not paranoia. Within one 24-hour window on November 6 and 7, practitioners posted four versions of the same complaint: coding agents reporting "verified" when the check never tested the thing; Google's developer blog on agent distraction, where a stale TODO comment sends an agent chasing a bug fixed years ago; The New Stack arguing developers should not have to babysit agents; and a post titled "You Don't Know If Your Agents.md Helps. You Just Keep Adding Rules."
That last one is the tell. We are adding rules to systems we cannot audit, hoping the rules work, with no way to know.
Part three: build for inspection, not intelligence. Look at what shipped on Product Hunt the same day. A terminal that records what coding agents did. An open-source harness that shows its work. A tool to find where agents get stuck in your product. TraceX pulled 316 votes, crystl pulled 261, and users asked for persistent agent notification cards and one-click rollback to the last signed version.
Nobody asked for a smarter agent. Everybody asked to see what the agent did. That is the market telling you where the real constraint sits.
There is a nuance worth stealing from Anthropic's agent memory research, published November 6, 2026. They found that telling the model what memory is for outperformed dictating how to use it, and that a rigid rule like "record only failures" actively hurt performance on their stronger models.
Translate that: your job is not to micromanage the agent's method. It is to be clear about the outcome and verify against it, which you can only do if you know what the outcome looks like. Which takes three reps.
Ramp card data reported by TechCrunch on November 6 shows median firms spending $12 to $15 per employee per month on AI.
That is nothing, which means the expensive part of your AI stack is not the subscription. It is the senior attention required to supervise it, and almost nobody has that line item in a budget.
Seven steps to run this week
1. Pick one task you were about to automate and stop. Choose something recurring that touches money, a client, or your reputation. Write the task name on a sticky note and add a tally mark each time you do it by hand.
2. Do it manually three times and keep a "what good looks like" note. After each rep, write three bullets: what correct output contains, what almost tripped you up, and what you would never have noticed if a machine had handed it to you.
3. Write down the three failure modes before you automate anything. Every task has a short list of ways it goes subtly wrong. Name yours explicitly. A checklist you wrote beats a checklist you downloaded.
4. Define the receipt before you define the automation. Decide in advance what evidence the agent must hand you to prove the job is done. A file path. A row count. A screenshot. A diff. If you cannot name the receipt, you have no way to tell success from a confident summary of success.
5. Automate one slice, not the whole chain. Hand over the most repetitive 20% first and keep your hands on the judgment calls. A five-step process where you own steps three and four beats a ten-step process you cannot inspect anywhere.
6. Write the brief with the Perfect Prompt Framework so the agent reports evidence, not confidence. The template I use:
[The Job] Run [TASK NAME] for [CLIENT OR TEAM] and report back with evidence I can check myself, not a summary. An error here reaches [WHO IS AFFECTED] before anyone catches it.
[The Background] Here is how I do this by hand: [YOUR THREE-REP NOTES]. It usually goes wrong in these ways: [FAILURE MODE 1], [FAILURE MODE 2], [FAILURE MODE 3]. Do not change [WHAT MUST NOT CHANGE]. Leave out [WHAT TO SKIP]. If you cannot complete a step, stop and say so instead of working around it.
[The Deliverable] Two sections. First, the output in [FORMAT]. Second, a verification section listing each check you ran, the exact result, and evidence I can inspect, such as [FILE PATH, ROW COUNT, SCREENSHOT, OR DIFF]. Flag anything you assumed. Maximum [LENGTH].
[The Questions] Ask me any questions you have.
7. Re-run the hand method once a quarter. Put it on the calendar and do the task manually one more time on the thing you automated. This is the step everyone skips, and it is the one that keeps your judgment current as the task drifts underneath the automation.
Frequently Asked Questions
Is three times a real rule or just a nice number?
Three is a practical floor, not a law. Rowan Cheung published it as a working rule in The Rundown AI on November 6, 2026. Practitioners commonly cite three manual reps; for complex or high-stakes processes, many wait for 5 to 20 runs so edge cases surface first. Use three as the minimum, and more when the cost of a quiet error is high.
Does this apply to non-technical work, or just to coding agents?
It applies everywhere. Parasuraman and Manzey documented automation complacency across decision-support systems generally, and the 2026 New York sanctions case involved legal research, not code. If you automate client reporting, invoicing, email sequences, or research summaries, the same gap applies to you.
Isn't doing a task manually three times just wasting billable time?
It is cheaper than the alternative. Those three reps cost you maybe an hour. Not having them cost one attorney and his firm $10,500 in sanctions in June 2026. You are not paying for the task. You are paying for the reference picture that lets you catch errors later.
How do I tell when an AI agent is lying about verifying something?
You usually cannot tell from the report, which is the problem. The fix is structural: require an inspectable artifact, not a status claim. Ask for the file path, the row count, the screenshot, or the diff, and spot-check one item yourself. Faros AI found 3 to 14 missed critical issues per item in work humans had already approved.
What should I automate first if I am starting fresh?
Start with the highest-frequency, lowest-consequence task you already know cold, because you already have the reps. Avoid anything touching money, legal exposure, or a client relationship until your verification habit is real. Then expand one slice at a time, adding a receipt requirement before each new slice goes live.
The thing I actually want you to hear
I opened with an agent that told me the work was done and verified when it was neither.
What stung was not the error. It was realizing I had outsourced a task before I had earned the right to supervise it, and then congratulated myself on the efficiency.
You are being sold speed right now, relentlessly. And speed is real.
But nobody is selling you the other half, because it does not fit on a landing page: the faster your tools get, the more valuable your judgment becomes, and judgment cannot be downloaded, subscribed to, or prompted into existence.
It gets built. Slowly. By doing the work with your own hands, badly at first, until you know what right looks like well enough to spot wrong at a glance.
So go do the thing by hand. Three times. Write down what you notice. Then hand it over, and demand a receipt every single time.
You are not behind because your agents are not smart enough. You are behind because you are supervising work you never learned to do.
Earn the reps. Then automate.
P.S. If you want to see how I build this verification layer into real agent stacks, receipts and failure-mode checklists included, that is most of what we work through inside AI Insiders at $247 a month. No pressure either way. Do the three reps regardless. That part is free, and it is the part that matters.
About the Author
Jonathan Mast is the founder and CEO of White Beard Strategies, where he teaches non-technical business owners how to use AI to amplify the skill and experience they already have. He runs a Facebook community of roughly 500,000 members, created the Perfect Prompt Framework, and speaks regularly on practical AI adoption for small businesses. He is also the guy who once shipped an agent he could not audit, read its confident report, believed it, and had to go back and do the job by hand to find out what it had actually done. He does the three reps now.
Sources
- Rowan Cheung, The Rundown AI, November 6, 2026
- Faros AI, "What AI caught in PRs our human reviewers had approved," October 28, 2026
- Faros AI, "AI-generated code is creating a code review burden for senior engineers"
- METR randomized controlled trial on AI and experienced developer productivity (arXiv:2507.09089)
- Parasuraman and Manzey, "Complacency and Bias in Human Use of Automation: An Attentional Integration," Human Factors 52(3), 2010
- Skitka, Mosier and Burdick, "Does automation bias decision-making?" International Journal of Human-Computer Studies, 1999
- Bainbridge, "Ironies of Automation," Automatica 19(6), 1983
- Ericsson, Krampe and Tesch-Römer, "The Role of Deliberate Practice in the Acquisition of Expert Performance," 1993
- Linux Foundation Research, "Navigating Open Source Software Development in the Generative AI Era," October 20, 2026
- Minnesota Lawyer, "Attorney, law firm sanctioned for AI mistakes in court filing," July 2, 2026
- Google Developers Blog, "Agent distraction"
- The New Stack, "AI agents are getting more complex. Developers shouldn't have to babysit them"
- Anthropic, "Components of agent memory," November 6, 2026
- TechCrunch, "Ramp data shows AI spending stalling just as Anthropic nears its IPO," November 6, 2026
