The Paradox No One Saw Coming
The token cost paradox is real — and it's reshaping how companies think about AI infrastructure.
HHere's a number that should make no sense: the price of running a GPT-4-equivalent task has fallen from roughly $20 per million tokens to roughly $0.40 - a 98% reduction in under two years. Measured a different way, Stanford's AI Index put the drop in inference cost for GPT-3.5-level performance at 280x between late 2022 and late 2024. Those two figures measure different things - one is the price of a token, the other is the cost of a fixed level of capability - but they point the same direction. Every unit of AI got dramatically cheaper.
By any normal economic logic, enterprise AI bills should be falling.
They're not. Average enterprise AI budgets have gone from around $1.2 million a year in 2024 to around $7 million in 2026 - close to a sixfold increase. Some Fortune 500 companies now report monthly inference bills in the tens of millions.
Welcome to the AI inference cost crisis of 2026 - a textbook case of Jevons paradox playing out in real time. When a resource becomes cheaper per unit, consumption doesn't just increase proportionally. It explodes.
The Casualties Are Piling Up
This isn't theoretical. It's already claiming budgets across Silicon Valley.
Uber burned through its entire 2026 AI coding tools budget in four months. The company had encouraged staff to use AI "as much as possible" and even ranked teams on internal leaderboards by total usage. By April, the money was gone. Uber's COO Andrew Macdonald now admits it's "very hard to draw a line" between AI token consumption and consumer features shipped. The company has since capped employees at $1,500 per month per coding tool.
Microsoft revoked internal Claude Code licences for most employees in May 2026 - six months after distributing them to thousands of engineers, product managers and designers. The reason was straightforward: token costs were outpacing what the company could justify against employee productivity gains. NVIDIA's VP of Deep Learning, Bryan Catanzaro, put it bluntly: "For my team, the cost of compute far exceeds the cost of employees."
Amazon shut down its internal "Kirorank" leaderboard after employees gamed the system with pointless AI activity to climb the rankings - every meaningless task burning capacity the company had to pay for. Senior VP Dave Treadwell told staff: "Please don't use AI just for the sake of using AI."
These aren't startups running out of runway. These are trillion-dollar companies discovering that AI adoption without cost discipline is a form of self-inflicted financial damage.
Why Agents Eat Tokens for Breakfast, Lunch, and Dinner
The problem isn't that AI is expensive per token. It's that modern AI workflows consume tokens in ways no traditional budget model anticipated.
The agentic loop multiplier. A simple chatbot query triggers one inference call. An agentic workflow - where an agent reasons iteratively, breaks down a task, calls tools, verifies outputs and self-corrects - triggers 10 to 30 calls to complete a single user-initiated task. Gartner's 2026 analysis puts agentic models at 5 to 30 times more tokens per task than a standard chatbot.
The context tax. Retrieval-Augmented Generation, the industry standard for grounding AI in enterprise data, sends thousands of pages of documentation to the model with every query. A RAG-enhanced enterprise query typically consumes 3 to 5 times more tokens than a simple query on the same model. When that context is poorly targeted - when you're sending entire codebases to find one relevant function - the waste compounds.
Always-on agents. The shift from on-demand to always-on AI means monitoring agents scan emails, logs, code repositories and operational systems in real time, consuming compute continuously even when no human has asked a question. Goldman Sachs estimates these will account for over 70% of all token usage by 2040, and projects global token consumption multiplying 24 times by 2030.
The overthinking problem. Research from Amazon Science found that reasoning models generate 7 to 10 times more tokens than necessary on simple tasks. A model will spend seventeen seconds deliberating "what is 1 + 1?", applying the same chain-of-thought it would use for multi-step planning. At scale across billions of queries, that computation costs tens of millions annually.
Tokenmaxxing: The Culture That Made It Worse
A strange culture emerged in early 2026: tokenmaxxing - treating token consumption as a badge of honour. Developers competed over who could burn the most compute. Companies built leaderboards rewarding raw usage rather than outcomes.
The data shows how badly this misfired. Jellyfish looked at 12,000 developers in Q1 2026 and split them by token spend. Engineers in the top 20% spent $1,822 on tokens over the quarter and shipped 23 merged pull requests. Engineers in the bottom 20% spent $3 and shipped 11. Roughly twice the output for several hundred times the spend.
Two other findings from the same period are worth putting alongside it. Code churn - lines deleted versus lines added - rose 861% under high AI adoption, per Faros AI and Waydev. And developers accepted AI-generated code at 80 to 90% rates initially, but real-world acceptance fell to between 10 and 30% once revision cycles kicked in. Per-developer token consumption rose 18.6x in nine months.
The tools generate volume. A disproportionate amount of that volume doesn't stick.
The Linux Foundation has now launched the Tokenomics Foundation — a standards body specifically created to bring cost discipline to AI tokens, the same way FinOps brought discipline to cloud spending. The fact that this body needs to exist tells you everything about the state of the market.
The Real Problem: Wasteful Retrieval
Strip away the cultural dysfunction and the budget meltdowns, and you find a common technical root cause: AI coding agents spend the overwhelming majority of their token budgets not on writing code — but on trying to understand where things are.
Recent research into agentic coding workflows reveals a striking breakdown of where tokens actually go:
Context loading — 62%. Reading files, re-loading previously seen code, ingesting directory structures. Every session starts from scratch, so the agent re-reads the same modules it explored yesterday.
Search & retrieval — 18%. Grep operations, file discovery, tool calls to find relevant code. The agent is lost in your codebase, flipping through files like someone handed a stack of loose pages with no table of contents.
Reasoning & review — 15%. The model thinking, critiquing, and verifying its own work.
Actual code output — 5%. The new code the agent writes — the only part that creates value.
That means 80% of every token dollar goes to orientation and retrieval — the AI equivalent of a developer spending their entire day re-reading the company wiki before writing a single line of code. Research from CodeConductor confirms that read operations (cat, grep, head) account for 76.1% of all tokens used by Claude Sonnet 4.5 on complex coding benchmarks. Microsoft's FastContext research found that reading and searching operations account for 56.2% of all tool-use turns. And as one developer put it: "I watched Claude Code read 25 files to answer a question about three functions — not because it's bad at coding, but because it had no idea which three it actually needed."
When a coding agent needs to understand a function, modify a module, or trace a dependency, the typical workflow looks like this:
Pull in large chunks of the codebase (expensive context loading)
Ask the LLM to identify what's relevant (expensive reasoning)
Re-read and re-process context across multiple tool calls (expensive iteration)
Discard most of what was retrieved and try again (pure waste)
This is the agentic loop multiplier in its most concrete form. The agent isn't thinking harder because the task is hard — it's thinking harder because it can't find what it needs efficiently. Every irrelevant file loaded into context is a token burned for nothing. Every re-read of documentation the agent has already processed is money evaporating.
The problem isn't the intelligence of the model. It's the precision of the retrieval feeding it. Fix the 80% waste layer, and the economics of AI coding transform overnight.
What If Agents Could Find Code Without Burning the Budget?
This is the problem we built Miru to solve.
Miru is a code-retrieval tool powered by ds1-code, an embedding model purpose-built for understanding code at the semantic level. Instead of dumping entire repositories into an LLM's context window and hoping it figures out what's relevant, Miru delivers precisely the code an agent needs — the exact function, the specific module, the relevant dependency chain — in a single retrieval step.
The economics shift fundamentally:
Fewer tokens per task. When retrieval is precise, agents don't need to iterate. They don't need to re-read. They don't need to load 50 files to find the one that matters. The agentic loop multiplier collapses.
Smaller context, better reasoning. LLMs reason better with focused context than with massive context windows full of noise. Precision retrieval doesn't just save money — it improves output quality.
Agents that scale without the budget exploding. If Goldman Sachs is right that token demand will multiply 24x by 2030, the only sustainable path forward is dramatically reducing tokens-per-task. Not by limiting what agents can do — but by eliminating the waste in how they find information.
Where This Goes
The inference cost crisis isn't going away. Token prices will keep falling and consumption will keep rising faster. If Goldman Sachs is right that demand multiplies 24 times by 2030, capping employee usage and revoking tools is not a strategy - it's a holding action.
The companies that come through this won't be the ones that used AI least. They'll be the ones that stopped paying for their agents to get lost.
Miru is available now as a CLI and an MCP server that plugs into Claude Code, Cursor, Copilot, VSCode, Codex and Kiro. It's open source under the MIT licence.
Sources
Oplexa, "AI Inference Cost Crisis 2026" (March 2026)
TechCrunch, "Uber caps employee AI spending after blowing through budget in 4 months" (June 2, 2026)
TechCrunch, "Is this the dawn of the Tokenpocalypse?" (June 7, 2026)
The Next Web, "Token prices fell 98%. Enterprise AI bills tripled." (June 5, 2026)
TechCrunch, "Tokenmaxxing is making developers less productive than they think" (April 17, 2026)
Jellyfish, "Is Tokenmaxxing Cost Effective?" (Q1 2026)
Fortune, "Uber's COO says it's getting harder to justify the company's AI spend" (May 26, 2026)
KuCoin/The Verge, "Microsoft halts internal use of Claude Code" (May 2026)
HR Chief Magazine, "Amazon Axes AI Leaderboard After Tokenmaxxing Backfires" (June 4, 2026)
Enterprise DNA / Goldman Sachs, "Decoding the Agentic Economy: 24x Token Demand" (May 2026)
Amazon Science / Firat Elbey, "The Overthinking Problem in AI" (November 2025)
Let's Data Science / Apollo Global Management, "Cheaper Tokens Drive Higher AI Token Spending" (June 2026)
Stanford HAI AI Index Report / Search Engine Journal, "AI Costs Drop 280x" (2025)
Gartner, "Worldwide IT Spending Forecast" (April 2026)
CodeConductor, "AI Coding Agent Memory" (June 2026) — read operations account for 76.1% of tokens
Microsoft Research, "How Do AI Agents Spend Your Money?" (May 2026) — agentic tasks consume 1000x more tokens than code chat
Jake Nesler / Medium, "Your AI Coding Agent Wastes 80% of Its Tokens Just Finding Things" (Feb 2026)
Vector Labs / Microsoft FastContext, "How to Stop Your Coding Agent from Burning Its Token Budget on Repo Search" (June 2026)