Why Does My AI Keep Getting Things Wrong Even When I Watch It Closely?

Because watching is not checking, and the fix is a written spec plus one test you already know the answer to. Here is the process, and the evidence behind it.

The Hook

"I supervised the agent for four nights before I saw that it had no way to check its results."

That sentence was published yesterday by Swami Sivasubramanian, one of the people who architected the data systems underneath Amazon Web Services. He was using a coding agent to rebuild a replication engine he had originally built with Al Vermeulen about twenty years ago. The original build took roughly four months.

Four nights of close supervision got him nothing.

Then he gave the agent two things: a written specification and a test environment. The rebuild took about two hours.

If you have been frustrated with AI lately, read those two numbers again, because they are the same model, the same task, and the same person watching. The only thing that changed was whether the agent had a way to find out it was wrong.

Here is the direct answer to the question. Your AI keeps getting things wrong under close supervision because supervision catches bad writing and misses wrong facts. What fixes it is giving the AI something external to check itself against: a plain-English definition of a correct result, plus one example where you already know the right answer. Write those two things and you move from four nights to two hours. Skip them and no amount of attention will save you.

Key Takeaways

Supervision and verification are different activities, and only one of them catches a wrong answer that sounds confident.

A written specification plus a single known-answer test case is the smallest version of verification that works, and it takes about an hour to build.

On November 11, 2026, AWS shipped an emulator whose only purpose is giving coding agents something to test against, and Google DeepMind published roughly 6.5 million lines of machine-checked proof verifying Signal's encryption.

Four of the top ten Product Hunt launches on November 10, 2026 sold inspection and review rather than new capability.

A 2010 review of automation bias research reported that making people socially accountable was associated with less automation bias and more verification, while cautioning that the evidence is not conclusive and higher effort may explain it.

The Problem

You already know the shape of this, because you have lived it.

You hand a task to AI. The output comes back fast and reads beautifully. You skim it. Nothing jumps out. You use it.

Two weeks later a client points at a number in it and you go cold, because you cannot remember where that number came from and neither can the tool.

So you adjust. You start reading more carefully. You spend longer on each output. You tell yourself you are being diligent.

Candidly, that is exactly what I did for most of this year, and I called it quality control. What I was actually doing was reading more slowly. I never once wrote down what a correct answer would look like before I asked for one.

That is the trap, and it is not a character flaw. It has a name in the research literature.

In 2010, Raja Parasuraman and Dietrich Manzey published an integrative review of automation complacency and automation bias in human factors research. Their finding was that these are not two separate problems but overlapping effects, with attention sitting at the center of both. People stop checking automated systems not because they are lazy but because attention is a finite resource and a system that has been right ten times in a row stops feeling like something that needs checking.

The studies they reviewed found something more uncomfortable. In process-control experiments with decision aids, researchers observed insufficient verification of the automation, and the errors people made were associated with high levels of complacency. And here is the part worth sitting with, with the caveat the authors themselves attach. Reviewing earlier work by Skitka, Mosier and Burdick, they report that making participants socially accountable was associated with less automation bias and more verification. They are careful to add that the results are not fully conclusive, and that simple extra effort and motivation might explain the same outcome.

Read that as a business owner rather than a researcher. The thing that made people check was not better training or a better interface. It was somebody waiting to ask them how they knew.

Right now, for most of the work you hand to AI, nobody is asking. Including you.

But what if the problem is not your attention at all? What if the problem is that your agent has no way to fail out loud?

The Evidence

Four things landed this week that all point the same direction, and not one of them is about building a smarter model.

One. AWS shipped a referee and said so out loud.

On November 11, 2026, AWS released AWS Emulator, a local offline copy of more than 200 AWS services built from more than 400 public API models, released into the public domain under CC0. Its entire purpose is to give a coding agent something to test against on your own machine before it touches anything real. Note whose framing that is: the claim that all of its code was written and tested by agents is Amazon's own, and humans built the specifications, the fidelity-test scaffolding, the merge gates and the release process around those agents. Which, conveniently, is exactly the point of the article.

Every change to it passes what Amazon calls a merge gate, which includes several security agents that each review a change for a different attack and a general code reviewer alongside them. The repository keeps a public folder named for the things that do not work yet, holding the issues the agents found where the emulator still does not match real AWS.

Think about that last detail. A company with Amazon's incentives chose to publish its own list of known gaps. That is what confidence in a verification system looks like, and it is the opposite of how most of us present our AI setups.

Swami's framing in that post is the sentence to keep: when agents write the code and check their own work, the engineer's job is to build the specs, tests, and merge gates that decide when that work is correct.

Two. Google DeepMind spent four weeks proving something rather than generating it.

On November 10, Google DeepMind's AlphaProof Code formally verified security properties of Signal's encryption, including the post-quantum PQXDH handshake and the Triple Ratchet. To be precise about what was proven, the target was libsignal, the reference implementation, compiled to 64-bit Arm machine code, and the properties were forward secrecy, authentication, post-compromise security and handshake secrecy for one-to-one session establishment. Five people directed agents to produce roughly 6.5 million lines of Lean proofs against about 26,000 lines of trusted specification and processor semantics, over four weeks. The researchers also checked that the system correctly rejected older versions of the software with known bugs.

That last check is the tell. They did not just confirm the thing passed. They confirmed the checker could catch a known failure. That is the difference between a test and a formality.

Two honest caveats. These figures come from DeepMind's own technical paper, and the scope is narrower than the headlines suggest: specified properties on specified code, one build, no multi-device state, no quantum adversaries, and some environment assumptions left informal. It is not a proof that Signal cannot be broken. But the direction is unambiguous: the most sophisticated AI lab in the world spent a month building a referee.

Three. The market priced it the same day.

On the November 10 Product Hunt leaderboard, four of the top ten launches sold inspection and nothing else. Latch, at number four, pitches that it builds a sidekick by watching you do the task and talk through the exceptions no SOP captures, and that when something does not fit, it stops and asks you, with every step logged for review. Seldon's QA agents find the tests you did not know to write by taking the paths customers actually take, like going back to change an address halfway through checkout. Co-Review lets you open your whole repository and ask the agent that built it why it did something, free and open source. None of these makes an agent smarter. Every one of them makes an agent reviewable.

Four. Practitioners spent the same 24 hours building cages.

Inside one day on Hacker News, developers posted a self-hosted agent with a permission kernel, two separate agent sandboxes, a tutorial on building one from scratch, a tool for tracking every agent session across machines, and a write-up on bypassing agent guardrails using legalistic language. Most of those sat at three to five points because they were hours old, so treat them as evidence of what people were building rather than what the community endorsed.

And the human half of it came from a developer named Mario Zechner on November 9, quoted in Ben's Bites: "the most import skill in 2026 is still being able to tell your agents 'wtf is this crap, no, not like this.'"

He is right, and he is describing only half the system. You bring the judgment. The agent needs a scoreboard. Swami had the judgment for four nights and it bought him nothing until the scoreboard showed up.

The Solution

Here is how this works together, because it is a system and not a tip.

Three pieces, in order.

The spec is the standard, written down before the work starts. Not a prompt. A description of what a correct result looks like, in plain language, including the exceptions. The exceptions are the whole game, because that is where an agent confidently invents something. When Swami says the AWS agents worked from public API models written in Smithy, that is what a spec is in their world. In yours it is a page of plain English that says what goes in, what comes out, and what is true of a right answer.

The test is one example where you already know the answer. This is the piece almost nobody builds, and it is the piece that moves four nights to two hours. You need one case you have personally done, where you know the correct output cold. The agent has to reproduce that before you let it near anything new. Amazon's version of this was the real AWS cloud itself: the agents corrected each test until it passed against the real service, so the test recorded how AWS actually behaves, and only then did they build the emulator until it passed that same test.

The gate is the thing that runs automatically before work reaches you. Once you have a spec and a test, you can turn the spec into a standing instruction the agent applies to its own output, and you can add an adversarial pass where a second AI attacks the first one's work. That is Amazon's merge gate, scaled down to a small business. It costs you nothing but the time to write it once.

Here is what changed for me when I actually did this. I had a recurring research task I had handed to AI for months, and every few weeks something in the output turned out to be wrong in a way I could not trace. I wrote one page describing what a correct result contained, and I built a single test from a piece of research I had done by hand in the spring, where I knew every number.

The tool failed the test. Not on style. On two specific claims it had been confidently producing for months.

That hour of work found a failure that four months of careful reading had not, and the reason is simple. Careful reading compares output to my expectations. The test compared output to reality.

The research points this direction, carefully. Parasuraman and Manzey write that specific experience of automation failures may reduce the effect, and list practical experience among the promising places to intervene. They also report that the evidence is inconsistent across studies, and that experience and practice generally do not eliminate complacency, since skilled pilots and controllers still show it. So take it as a reason to build the test, not as a promise that one failure cures you.

Practical Steps

Here is the work, in order. The whole sequence takes about ninety minutes and you only do it once per task.

1. Pick the task where being wrong costs the most. Not the task you use AI for most. The one where a wrong answer reaches a customer, a number, or a decision. Frequency times cost of failure, and nothing else. If two tasks tie, pick the one you would struggle to explain if it went wrong.

2. Write the definition of a correct result before you write another prompt. One page, plain English, and it must include the exceptions. Use this:

[The Job]
Write the definition of a correct result for [THE TASK].
This is for: me, so I can tell good output from confident-sounding bad output.
It matters because: I cannot review work when I have never written down the standard.

[The Background]
Here is what you need to know: [WHAT THE TASK PRODUCES], [WHO RECEIVES IT], [WHAT HAPPENS IF IT IS WRONG], [THE LAST TWO TIMES THE OUTPUT DISAPPOINTED ME AND WHY].
Do not use: generic quality words like "high quality," "professional," or "engaging."

[The Deliverable]
Return: a one-page checklist of 7 to 10 pass or fail checks, in plain text.
Must include: at least three checks a stranger could run without knowing my business, and at least two that require my specific context.
Optimize for: accuracy.

[The Questions]
Ask me any questions you have.

3. Build one test case from work you already did by hand. Find a past example where you know the right answer. Write down the input, the expected output, and three specific ways a wrong answer would show itself. Save it somewhere you can find it in a month. This is your test environment, and it does not need to be anything fancier than a note.

4. Run the test on the tools you are already paying for. Same input, same expected answer, two or three tools. Score each against your checklist rather than your gut. You will likely learn two things: that one tool is clearly better at this specific task, and that you did not need both subscriptions.

5. Turn the checklist into a standing self-check. Paste your checks into the instructions for whichever tool you use, with one addition that matters more than all the rest: tell it to state which checks failed and stop, rather than guess. An agent that will not admit a gap is an agent that fills it.

6. Add the adversarial pass for anything that leaves your building. Before work reaches a client, have a second AI attack it. Give it the confirmed facts, tell it to ignore style entirely, and ask it to mark every claim as confirmed, unsupported, or wrong. Amazon runs several security agents in parallel on every change for the same reason: work nobody argued with is work nobody checked.

7. Keep your own list of things that do not work. One line per failure: what it was, how you noticed, how often it happens. Amazon publishes theirs. Yours can live in a note. An honest list of known gaps is worth more than a claim that your setup works, and it is the only way the knowledge survives your own memory.

Frequently Asked Questions

How long does this actually take for one task?

About ninety minutes the first time, split roughly into forty minutes writing the spec, thirty building the test case from past work, and twenty running it. After that, rerunning the test takes under five minutes. You do the ninety minutes once per task, not once per use.

What if my work is subjective, like writing or design?

Then your spec splits into two lists, one for correctness and one for voice. The correctness list is objective even in creative work: are the facts right, are the names spelled correctly, is the offer described accurately, is the price current. Run that pass first. Most of the damage in creative work is factual, not aesthetic.

Can I just use a better prompt instead?

You can try, and many people spend months there. The diagnostic is simple: if your outputs are inconsistent in quality, that is often a prompt problem. If they are consistently smooth but occasionally wrong in ways you catch late, that is a verification problem, and no prompt fixes it. Rewriting wording does not give the tool a way to know it was wrong.

Do I need special software for the test environment?

No. For most small business tasks a test environment is a saved note holding one input, one expected output, and the pass or fail criteria. AWS needed to build an emulator because they were testing against 200 cloud services. You are testing against one past job you already did correctly.

What if the AI fails my test?

That is the point, and it is the best outcome available on day one. A failure found against a known answer costs you nothing. The same failure found by a client costs you the relationship. The human factors literature is more cautious than I would like here, reporting that experience of failure may reduce complacency while noting the evidence is inconsistent. Build the test anyway. A cheap failure is still cheaper than an expensive one.

The Close

Four nights of watching. Two hours with a spec and a test.

That gap is not about Amazon, and it is not about coding. It is about the difference between paying attention and building a way to be wrong.

Here is the thing I keep coming back to. The most capable AI organizations on earth spent this week building referees. AWS built an emulator so agents could check themselves. DeepMind spent a month generating proof rather than output. Four of the ten most popular new products on Product Hunt sell nothing but the ability to see what your AI did.

Every one of them reached the same conclusion, and none of them reached it by getting a better model.

You are running the same experiment Swami ran, and you have been running it for months. You have been reading harder. You have been checking more carefully. And somewhere in that work is a wrong answer you have not found yet, sitting in a document you already sent, because careful reading compares output to what you expected and never once compares it to what is true.

So stop supervising. Go write the one page that says what correct looks like, find the one job you already did right, and make your AI prove it can reproduce it.

You cannot audit work against a standard you never wrote down.

About the author

Jonathan Mast is the founder and CEO of White Beard Strategies, where he teaches non-technical entrepreneurs how to use AI to amplify the skill and experience they already have. He runs a community of more than 500,000 entrepreneurs, created the Perfect Prompt Framework 2.0, and speaks regularly on practical AI adoption for small business. He has built a seven-figure agency, lost it, and rebuilt, which is a reasonably good education in the difference between watching a problem and checking it.

Why Does My AI Keep Getting Things Wrong Even When I Watch It Closely?

Contents

The Hook

Key Takeaways

The Problem

The Evidence

The Solution

Practical Steps

Frequently Asked Questions

The Close

Sources

Should I Let AI Agents Buy From My Business?

Why Isn’t AI Recommending My Business When Someone Asks It For One?

How many times should I do a task manually before I automate it with an AI agent?

How Much Should A Small Business Spend On AI Per Month?

About the Author

Jonathan Mast