Token prices are falling, your invoice is rising, and the reason is that your AI is re-reading the same background on every single turn. Here is the mechanism, the research behind it, and what to do about it this week.
The Bill Nobody Warned You About
A friend of mine runs a six-person agency. He called me in September because his AI spend had tripled in two months and nothing about his work had changed. Same clients. Same output. Same tools.
He had already done the obvious thing. He went shopping for a cheaper model.
Here is the direct answer, and it is the one almost nobody gives: your AI bill is not primarily a function of the price per unit. It is a function of how much background the system has to re-read on every single step of every single task. In agent-style work, the input side dominates the cost, and the input grows with every turn because each new step re-sends everything that came before it.
Stanford's Digital Economy Lab put a number on it this year. In a study of eight frontier models on real coding tasks, researchers including Erik Brynjolfsson found that agentic tasks consume roughly 3,500 times more tokens than a single-round reasoning task and 1,200 times more than a multi-round chat. Their explanation is one sentence long and it is the whole ballgame: "the same context gets fed into the models repeatedly."
So when the industry announces that prices dropped by half, and your volume per task went up by a factor of twelve hundred, you have not saved anything. You have been handed a discount on a much larger purchase.
That is the bad news. The good news is better than you think. This is the one part of AI cost you fully control, it requires no new software, and in most small businesses the fix takes an afternoon.
Key Takeaways
- In agent-style AI work, input tokens drive cost more than output tokens, because every step re-sends the full conversation that came before it.
- Stanford researchers measured agentic tasks consuming roughly 3,500 times the tokens of a single reasoning prompt, and found that higher spend does not buy higher accuracy.
- Prompt caching is a billing discount, not a quality fix, and a badly structured prompt can make caching cost more than not caching at all.
- Longer context also degrades answer quality, a problem Anthropic's own engineers say will persist regardless of how large context windows get.
- The fix is writing down the minimum your AI needs to know, not shopping for a cheaper model.
The Problem: You Are Being Charged for Reading, Not Thinking
Most owners picture AI cost the way they picture a phone bill. You use it, you pay for what you used, and if you use it twice as much you pay twice as much.
That mental model is wrong in a specific and expensive way.
When you run a single chat message, you pay once for what you sent and once for what came back. Fine. But when you run anything that takes multiple steps, which includes most of what the industry now calls an agent, each step sends the entire history again. Step one sends your instructions. Step two sends your instructions plus step one. Step three sends your instructions plus steps one and two. By step forty, you have paid for your instructions forty times.
The security company ProjectDiscovery published the clearest description of this I have seen anywhere. Their agent, Neo, averages 26 steps and 40 tool calls per task, with a system prompt running over 20,000 tokens. They wrote it plainly: "On a 40-step task, you're sending that 20K-token system prompt 40 times." They named the effect the agentic tax, and described it as cost that compounds with task complexity.
I have been in business long enough to recognize this shape. It is the same shape as a bad lease where the escalator clause is buried on page nine. Nothing about it is dishonest. It is just that nobody put it in the brochure, and most owners sign without knowing the mechanism.
Here is where it gets worse. The thing that drives your cost up also drives your quality down.
Anthropic's own engineering team published this in September 2025, and I want you to read it as a concession rather than a sales pitch, because that is what it is. They wrote that "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases," and that "this characteristic emerges across all models." They describe an attention budget, where "every new token introduced depletes this budget by some amount."
So the long instructions file you pasted in to be thorough is costing you money and making the answer worse at the same time.
And before you ask whether a bigger context window solves this: Anthropic addressed that directly. They wrote that "for the foreseeable future, context windows of all sizes will be subject to context pollution and information relevance concerns."
Candidly, this is the least-discussed and most expensive misunderstanding in small business AI right now.
The Evidence: Four Numbers That Should Change How You Work
Paying more often buys you a worse answer. The Stanford study found that "higher token usage does not translate into higher accuracy; instead, accuracy often peaks at intermediate cost and saturates at higher costs." The expensive models in their test were burning tokens on redundancy. For the highest-cost models, around half of all file operations were repeat actions on the same file. Half. The system was re-reading things it had already read.
Two near-identical workloads, sixty times apart in cost. This is the single most striking figure in my research. ProjectDiscovery published two real tasks from their own logs. One consumed 67.5 million input tokens at a 91.8% cache hit rate. The other consumed 66.8 million input tokens at a 3.2% cache hit rate. Almost the same volume of work. They state the second cost roughly sixty times the first. The difference was not the model and not the task. It was whether the stable parts of the prompt were positioned so they could be reused.
Accuracy degrades in the middle of long context, and it is well documented. The foundational paper here is "Lost in the Middle" by Nelson Liu and colleagues at Stanford and Berkeley, published in Transactions of the ACL. Their finding: performance is highest when the relevant information sits at the beginning or end of the input, and "significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models." Burying the important fact in paragraph six of your instructions is not thoroughness. It is a design error.
This is killing real projects, not just annoying developers. Gartner forecast in June 2025 that more than 40% of agentic AI projects will be canceled by the end of 2027, naming spiraling costs first among the causes. S&P Global Market Intelligence found the share of companies abandoning most of their AI initiatives rose from 17% to 42% in a single year, with the average organization scrapping 46% of proofs of concept before they reached production. Cost was named among the top obstacles in both.
One more, because it closes the loop on where the money actually goes. The Stanford team broke agent token spend down by phase: setup 9.98%, explore 30.37%, fix 33.53%, validate 16.59%, closeout 9.53%. Add explore and validate and you get roughly 47% of spend going to reading and re-checking rather than producing. Nearly half your bill is the system looking things up.
A fair caveat before we move on. The ProjectDiscovery numbers come from a company that sells an AI product, so they are a vendor disclosure rather than an independent audit, though they state their figures come from actual reported costs. The Gartner and S&P figures are both 2025-dated, and I have not found 2026 updates. I would rather hand you a dated number with its date attached than a fresh number I cannot source.
The Fix: Write Down the Minimum, Not the Maximum
Here is what changed for me.
I used to treat the context file as a place to be generous. More background meant better output, I assumed, so I pasted in everything: the full offer list, every price, the team roster, the brand voice guide, the history of why we do things a certain way. Thorough. Professional. Expensive and worse.
The shift was realizing that a context file is not a briefing document for a new hire. A new hire reads it once. An AI reads it on every turn, pays for it on every turn, and spends a little of its attention budget on every irrelevant word.
So I started treating it as an architecture problem, which is how I think about most things anyway. Three layers.
The first layer is the core, and it is small. The task, the hard constraints, and the required output format. Nothing else. If the task would still succeed without a line, that line is not core.
The second layer is reference material that gets attached on demand. Pricing goes in one file. Brand voice goes in another. Team and process in a third. Most tasks need one of them. No task needs all of them.
The third layer is position. This is the part almost nobody knows, and it is the one that produced ProjectDiscovery's sixty-fold difference. Caching on these systems is prefix-based, which means anything that changes early in your prompt invalidates everything after it. They put it bluntly: "anything that changes in the middle of the prefix invalidates everything after it." Their working memory was sitting in the middle of the prompt and, in their words, "silently killing our cache hits." Moving the dynamic content to the end took them from a 7% hit rate to 74% in a single deployment, and cut billed cost by 59%.
A per-request timestamp is enough to do this to you. They had to freeze their datetime to date-only to stop it.
There is a real limit worth stating. Caching helps the bill and does nothing for the quality problem. Anthropic's documentation is explicit that prompt caching has "no effect on output token generation. The response you receive is identical to what you would get if prompt caching were not used." Cached tokens also still count toward your prompt length. So caching is a discount on re-reading, not permission to re-read more.
Which brings it back to the same place. Shorter context costs less and answers better. Caching makes the remaining re-reading cheaper. In that order.
One thing I want to name before we go further, because it is the thing that got my agency friend. He had an automation that ran unattended overnight with no ceiling on it. That is how a documented case ended up with an agent retrying a failing tool 31,000 times in five hours. The fix for that is not context engineering. It is a limit sheet, and it is step six below.
Practical Steps You Can Take This Week
1. Find out what you are actually re-sending. Open the standing instructions or project file that rides along on every request you make. Count the words. Then go line by line and mark whether the task would fail without that line. Most people find half of it is reference material that one task in ten needs.
Here is a Perfect Prompt for it:
[The Job]
Audit the background information I hand my AI every time I ask it to do [RECURRING TASK].
This is for: me, the owner, trying to cut AI cost without cutting output quality.
It matters because: I am paying for every word of context on every single turn, not just once.
[The Background]
Here is what you need to know: [PASTE THE FULL INSTRUCTIONS OR PROJECT FILE YOU CURRENTLY GIVE IT].
Do not use: assumptions about my industry I have not stated.
[The Deliverable]
Return: a two-column table. Left column, every distinct piece of context I provide. Right column, whether the task would fail without it.
Must include: a word count for the whole thing and a word count for only the items marked essential.
Optimize for: accuracy.
[The Questions]
Ask me any questions you have.
2. Split it into a small core plus on-demand reference files. One core file with the task, the constraints, and the output format. Then no more than four reference files, each with a one-line note on when to attach it. If you cannot remember which to grab, you made too many.
3. Move anything that changes to the very end of your prompt. Dates, names, session details, today's numbers. All of it goes last. This is the change that produced a 59% cost reduction for ProjectDiscovery in one deployment, and it costs you nothing but ten minutes of rearranging.
4. Ask your vendor for a cache hit rate, not whether they use caching. This is the single best question on the list. ProjectDiscovery ran a 7% hit rate while caching was fully enabled. Anthropic prices cache writes at 1.25 times normal input and reads at 0.1 times, which means a badly structured prompt makes caching cost you more than skipping it. "Do you use caching" gets you a yes. "What is your cache hit rate, measured per task rather than averaged" gets you the truth.
5. Shorten what you ask for, not just what you send. Look honestly at the output you request and the parts you actually read. If you skim to one section every time, stop paying for the rest. You are billed for every word it writes, including the words you skip.
6. Put a hard ceiling on anything that runs unattended. Maximum retries per tool. Maximum total steps per run. Maximum spend per run. And a defined behavior when it hits one, including who gets told. This is not advanced practice. This is the smoke detector.
7. Measure before and after, then write down what worked. Baseline your input size, output size, and runs per month before you change anything. Compare after. Then write a one-page standard for your team, because the savings evaporate the moment someone pastes their whole notes file back in. If you have team members building AI workflows, they need the mechanism explained, not just the rule handed down.
Frequently Asked Questions
Token prices keep dropping. Will my AI bill fall on its own?
No. Volume per task is rising faster than unit price is falling. Stanford measured agentic tasks consuming roughly 3,500 times the tokens of a one-shot prompt because the same context gets re-fed repeatedly. A fifty percent price cut against a twelve-hundred-fold volume increase is not a saving.
Can I just use a model with a bigger context window and skip this?
No. Bigger windows grow the bill faster without fixing accuracy. Anthropic's engineering team wrote that context windows of all sizes will remain subject to context pollution and relevance problems for the foreseeable future, and that recall decreases as tokens increase across all models.
If I pay more per task, do I at least get a better answer?
Often not. Stanford's study of eight frontier models found that higher token usage does not translate into higher accuracy, and that accuracy frequently peaks at intermediate cost. In their tests, roughly half of the high-cost models' file operations were repeat actions on files already read.
My vendor says they use prompt caching. Am I covered?
Not necessarily. Caching can be switched on and still deliver almost nothing. ProjectDiscovery ran a 7% hit rate with caching enabled because dynamic content sat mid-prompt. Ask for the measured hit rate per task, since cache writes cost more than ordinary input tokens.
Is this actually ending projects, or is it just a developer complaint?
It is ending projects. Gartner forecast in June 2025 that more than 40% of agentic AI projects will be canceled by end of 2027, naming spiraling costs first. S&P Global found companies abandoning most AI initiatives rose from 17% to 42% in one year.
The Close
My agency friend did not need a cheaper model. He needed to delete four paragraphs and move one line to the bottom of the file.
That is not a satisfying answer. Nobody wants the fix to be that small. We would rather believe the problem is the vendor, the pricing, the industry, anything with a villain in it. I understand the pull. I have spent money on tools to solve problems that were actually discipline problems, and I have the receipts to prove it.
But here is the thing. Every one of the expensive mistakes in this article comes from the same instinct, which is a good instinct used wrong. You were being thorough. You pasted in everything because you wanted the AI to have what it needed. You left the automation running because you wanted it to finish the job. Nobody told you that thoroughness has a per-turn price and a quality cost attached to it.
Now you know the mechanism. And once you know a mechanism, you are not at anybody's mercy on it.
So go open the file that rides along on every request you make. Count the words. Mark what the task would fail without. Move the changing parts to the bottom. That is an afternoon, and it is worth more to your business than any tool you will buy this quarter.
The expensive part was never the thinking. It was the re-reading, and you are the only one who can shorten it.
About the Author
Jonathan Mast is the founder and CEO of White Beard Strategies, where he teaches non-technical entrepreneurs how to use AI to amplify the skill and experience they already have. He runs a 500,000-member community of business owners learning AI in public, built the Perfect Prompt Framework 2.0 that this article's prompts are built on, and speaks regularly on practical AI implementation for small business. He has built a seven-figure agency, lost it, and rebuilt, which is partly why he is allergic to any AI advice that depends on an unlimited budget.
