Your agent is not broken because the model is weak. It is failing because you never decided what it should see, when it should see it, and when it should stop. This article answers the question every entrepreneur is quietly asking: why does an AI agent that looked brilliant on Tuesday fall apart in real work on Thursday?
The Two-Hour Agent That Cost Me a Weekend
I built an agent last year that I was genuinely proud of. It pulled our customer emails, sorted them by intent, drafted replies, and flagged the ones that needed me. I tested it for two hours. It was flawless. I told my team we had just bought back six hours a week.
Four days later I found out it had been quietly answering questions with pricing from a document we retired in 2023. Not occasionally. Consistently. It never errored. It never warned me. It just confidently handed customers information that was two years stale, and it did it in my voice.
Here is the direct answer to why your agent keeps failing: in almost every case the model is not the problem. The problem is context. Your agent is either seeing the wrong information, seeing too much information, seeing information in the wrong order, or seeing nothing at all when a tool call silently fails and it keeps going anyway. The fix is not a better model. The fix is deciding, on purpose, what your agent gets to see.
That discipline has a name. It is called context engineering, and it is the least glamorous and highest leverage skill in AI right now. It is not about clever prompts. It is about information architecture: what goes in the window, what stays out, what gets refreshed, and what makes the agent stop and ask.
Most people never learn it because it does not demo well. Nobody claps for a well-scoped file path. But it is the entire difference between an agent that impresses your friends and an agent you can actually put in front of a customer.
Key Takeaways
- Your agent is most likely failing because of context problems, not model weakness, and no model upgrade will fix a context problem.
- Research shows that every frontier model gets less reliable as you stuff more into its context window, long before you hit the token limit.
- Silent tool failures are more dangerous than loud errors because your agent keeps working confidently on missing or stale information.
- Context engineering means deciding what your agent sees, when it sees it, and in what order, and it is a skill you can learn without being an engineer.
- If you cannot trace what your agent did step by step, you are not running an agent, you are running a slot machine.
The Problem Nobody Warns You About
Here is the pattern I see every week in our community, and it is remarkably consistent.
Someone builds an agent. The demo is spectacular. They show it to their team, everybody gets excited, and they roll it into a real workflow. For the first day or two it holds. Then something shifts. Answers get vaguer. It starts repeating itself. It forgets a rule it followed perfectly an hour ago. It calls a tool, the tool returns nothing, and the agent invents a plausible answer instead of stopping.
And the person who built it draws the wrong conclusion. They think the model is not smart enough yet. So they swap to a bigger model, and it gets marginally better, and then it degrades again in a slightly different way.
I have been exactly where you are. After the stale pricing incident I spent a full weekend rebuilding that email agent. My first instinct was to give it more. More documents. More history. More examples. I dumped every policy file, every past thread, every FAQ into its working memory because I assumed that if it had everything, it could not get anything wrong.
It got worse. Measurably worse. It started missing rules that were sitting right there in the context I had given it.
That was the moment the thing finally clicked for me. I had been treating context like storage, as if the goal were to load the agent up like a truck. Context is not storage. It is attention. Everything you add competes with everything else for the model’s focus. When I gave the agent forty pages, I did not make it smarter. I made the two paragraphs that actually mattered harder to find.
There are three failure modes underneath almost every broken agent I have looked at.
The first is silent tool failure. Your agent tries to read a file, hit an API, or search a database. The call fails or returns empty. The agent, trained to be helpful, keeps going and fills the gap with something reasonable-sounding. You never see an error. You just see a wrong answer delivered with total confidence.
The second is context degradation over long sessions. The longer an agent runs, the more junk accumulates: old tool outputs, abandoned reasoning paths, superseded instructions. By turn forty, the agent is wading through the wreckage of turns one through thirty-nine.
The third is no tracing. When it breaks, you have no idea which step broke it. You are debugging by vibes.
Here is the reframe that changes everything: stop asking “how do I make my agent smarter” and start asking “what does my agent actually need to see right now, and how would I know if it did not get it.”
What the Research Actually Shows
I do not want you to take my word for this. The data is unusually clear, and it points away from model capability and toward engineering discipline.
Roughly 95 percent of enterprise AI pilots produce no measurable return. MIT’s NANDA initiative published “The GenAI Divide: State of AI in Business 2025” in August 2025, based on 150 leader interviews, a survey of 350 employees, and analysis of 300 public AI deployments. About 5 percent of pilots delivered rapid value. The rest stalled with little or no measurable impact on the bottom line. The researchers’ conclusion was not that models are too weak. It was that organizations are mismanaging how AI gets integrated into real work.
Gartner projects that over 40 percent of agentic AI projects will be canceled by the end of 2027. That prediction, published June 25, 2025, cites escalating costs, unclear business value, and inadequate risk controls. Gartner also noted a widespread “agent washing” problem, where vendors rebrand existing chatbots and automation as agents, and estimated that only around 130 of the thousands of agentic AI vendors in the market are offering something real. Worth noting: the same firm projects that by 2028, 15 percent of day-to-day work decisions will be made autonomously by agents, up from zero in 2024. This is not a prediction that agents fail. It is a prediction that undisciplined agent projects fail.
Every frontier model degrades as context grows, well before it hits the limit. In July 2025, Chroma Research published “Context Rot: How Increasing Input Tokens Impacts LLM Performance,” testing 18 frontier models including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3. Every model tested got less reliable as input length increased, even on simple retrieval tasks. Critically, this is not the same as running out of room. A model with a 200,000 token window can start slipping at 50,000 tokens. Adding more context does not reliably help. Past a point it actively hurts.
Teams that ship agents successfully invest heavily in tracing. LangChain’s State of Agent Engineering report, published June 2026 from a survey of 1,340 professionals, found that 57 percent of respondents had agents in production. Quality was the top barrier to production, cited by about a third. And here is the part that matters: 89 percent had implemented some form of observability, with 62 percent running detailed tracing that lets them inspect individual agent steps and tool calls. Among teams already in production, that jumped to 94 percent with observability and 71.5 percent with full tracing. Respondents at large enterprises specifically named context engineering and managing context at scale as their ongoing challenge.
And the real-world consequences are not hypothetical. In Moffatt v. Air Canada, the British Columbia Civil Resolution Tribunal ruled in February 2024 that Air Canada was liable for incorrect bereavement fare information its chatbot gave a customer, awarding $812.02 in damages and fees. The airline argued the chatbot was effectively responsible for its own statements. The tribunal disagreed. You own what your agent says.
Then there is Klarna, which in February 2024 announced its AI assistant was doing the work of 700 agents, handling 2.3 million conversations and cutting resolution time from 11 minutes to under 2. By spring 2025 the company was publicly walking that back and rehiring human staff, with the CEO acknowledging that the push for cost savings had degraded service quality. Klarna now runs a hybrid model. The lesson is not that the AI could not work. It is that they deployed it without a clear boundary around what it should and should not handle.
Context Engineering: Deciding What Your Agent Sees
Anthropic’s engineering team frames context engineering as the natural evolution of prompt engineering: instead of optimizing how you phrase a request, you optimize the entire set of information the model holds during a task. Their framing is the one I use with clients. Context is a finite resource with diminishing returns. Your job is to find the smallest set of high signal information that gets the outcome you want.
That is the opposite of what most people do. Most people load the truck.
When I rebuilt my email agent, I did four things. Total build time was about a day. It has run reliably since.
I built a context file and kept it short. One document, under two pages, containing only what the agent must never get wrong: current pricing, our refund policy, the three things we never say, and who gets escalated to a human. Not a knowledge base. A constitution. Everything else the agent fetches on demand.
I scoped what it could reach. Previously it had access to our entire shared drive. Now it can read exactly four locations. If the answer is not in those four, it does not guess, it escalates. Narrowing access made it dramatically more accurate, which sounds backwards until you remember that every extra document is a distraction competing for attention.
I wrote explicit stop conditions. This is the single highest impact change and almost nobody does it. My agent now has written rules: if a tool call returns empty, stop and flag. If the customer mentions a refund over a certain amount, stop and flag. If you are not confident, stop and flag. Before this, “not confident” produced a confident guess. Now it produces a notification to me.
I turned on tracing. Every run logs each step: what it was asked, what tools it called, what came back, what it decided. When something goes sideways now, I do not theorize. I read the trace and find the exact step where it went wrong, usually in under two minutes.
Microsoft’s team building the Azure SRE Agent hit the same wall at enterprise scale and solved it the same way. Rather than pushing 50,000 tokens of raw metrics into the context window, they sent the data to a code interpreter and had the model write analysis code against it. Same information available. A fraction of the context consumed. That is context engineering in one move: not less capability, less noise.
You do not need to be an engineer to do any of this. You need to be willing to make decisions about information that most people avoid making because it is boring.
Seven Steps to Fix Your Failing Agent
1. Write down what the agent must never get wrong. Before you touch any settings, list the five to ten facts, rules, and boundaries that would cause real damage if the agent got them wrong. That list becomes your context file. If it runs longer than two pages, you are including things that belong somewhere else.
2. Cut the agent’s access down to what it genuinely needs. Go through every document, folder, tool, and integration your agent can reach and remove anything not required for its specific job. Broad access feels safe and produces worse results. Narrow access forces the agent toward the sources you actually trust.
3. Define your stop conditions in plain language. Write explicit rules for when the agent must halt and involve a human: missing data, dollar thresholds, anything touching legal or medical or financial advice, anything where it is uncertain. An agent without stop conditions will always choose to keep going, because that is what helpfulness looks like from the inside.
4. Test for silent failures on purpose. Deliberately break something. Rename a file the agent depends on. Disconnect a tool. Then watch what it does. If it produces a confident answer instead of an error, you have found the exact failure mode that will embarrass you in production.
5. Turn on tracing before you need it. Whatever platform you are using, find the logging or run history feature and switch it on now. When your agent misbehaves in three weeks, the difference between a ten minute fix and a lost afternoon is whether you can see what it actually did.
6. Keep sessions short and start fresh often. Do not let one agent conversation run for hours across dozens of unrelated tasks. Start a new session per task or per day. This single habit sidesteps most context degradation without any technical work at all.
7. Review the trace weekly for the first month. Set a recurring thirty minute block. Read through what your agent actually did, not what you assume it did. Every review I have ever run has surfaced at least one thing I would not have predicted.
Frequently Asked Questions
What is context engineering in simple terms?
Context engineering is deciding what information your AI agent sees, when it sees it, and in what order. Prompt engineering is about how you phrase a request. Context engineering is about the whole information environment around that request: documents, tool outputs, memory, and history. It is the difference between a demo and a system.
Will a better AI model fix my agent’s problems?
Usually not. If your agent is failing because it is reading a stale document, missing a silent tool error, or drowning in accumulated session history, a stronger model will make the same mistakes more articulately. Upgrade the model when reasoning quality is genuinely the bottleneck. Fix context first, because it is cheaper and faster.
Why does my agent get worse the longer it runs?
Because context accumulates. Old tool outputs, abandoned reasoning, and superseded instructions all stay in the window and compete for the model’s attention. Chroma Research tested 18 frontier models and found every one degrades as input length grows, well before hitting the token limit. Shorter sessions and periodic fresh starts fix most of it.
Do I need a developer to do context engineering?
No. The highest impact moves are decisions, not code: writing a short context file, narrowing what folders the agent can read, defining when it must stop and ask a human, and reviewing run logs. Most modern agent platforms expose all four in settings. The bottleneck is judgment about your business, not technical skill.
How do I know if my AI agent is actually working?
Turn on tracing and read it. A working agent shows a clean chain: it was asked something, it called the right tools, the tools returned real data, and it decided accordingly. If you cannot see that chain, you do not know whether your agent is working or getting lucky. Verified output beats confident output every time.
The Unglamorous Skill That Actually Wins
That stale pricing incident cost me two customer relationships and a weekend I will not get back. What stings is that it was entirely preventable, and not by anything sophisticated. I just never decided what that agent should see. I gave it access to everything and assumed it would sort out what mattered.
It could not. It never can. That was my job.
I want you to hear the good news buried in all of this. If 95 percent of pilots stall and over 40 percent of agentic projects get canceled, and the cause is not model capability but engineering discipline, then this is a solvable problem. Model capability you have to wait for. Discipline you can apply this afternoon.
You do not need a bigger model. You do not need a bigger budget. You need to sit down with a blank page and answer four questions: What must this agent never get wrong? What is it allowed to look at? When must it stop and ask me? And how will I see what it actually did?
That is it. That is the work. It takes an afternoon, it produces no screenshot worth sharing, and it is the difference between an agent that impresses people in a demo and an agent you trust with your customers on a Thursday.
The people winning with AI right now are not the ones with the best prompts. They are the ones who decided what their agents get to see.
About the Author
Jonathan Mast is the founder of White Beard Strategies, where he provides AI coaching and mentorship to entrepreneurs and small business owners who want practical results instead of hype. He serves tens of thousands of entrepreneurs through his training, community, and content, and is the creator of the Perfect Prompt Framework, a plain-English system for getting reliable output from AI. He is a frequent speaker on applied AI for small business, and his approach is consistently the same: skip the theater, build the thing that works.
Sources
- Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027
- MIT report: 95% of generative AI pilots at companies are failing (Fortune)
- Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma Research)
- Effective context engineering for AI agents (Anthropic)
- State of Agent Engineering (LangChain)
- Context Engineering for Reliable AI Agents: Lessons from Building Azure SRE Agent (Microsoft)
- How can I mislead you? Air Canada found liable for chatbot’s bad advice on bereavement rates (CBC News)
- Klarna Reverses AI Push, Says Customers Prefer Human Support (Forbes)
- Klarna changes its AI tune and again recruits humans for customer service (CX Dive)
