Token Economics¶
Token prices are falling faster than ever, yet enterprise AI bills are rising. That is not a contradiction; it is the Jevons paradox applied to intelligence.
The Jevons paradox¶
In the 1860s William Stanley Jevons observed that when steam engines became more efficient at burning coal, coal consumption rose, not fell. Cheaper energy made new applications profitable, and total demand grew faster than efficiency improved. The same thing is happening with LLM tokens.
Today the price per token has fallen by roughly an order of magnitude per year in some segments. At the same time, enterprise spending on generative AI is climbing. Menlo Ventures reported that enterprise generative-AI spend reached $37 billion in 2025, roughly three times the $11.5 billion spent in 2024 [Menlo Ventures, 2025]. A separate index tracking total LLM token expenditure roughly doubled in late 2025 even as the price per million tokens fell more than 90% since 2023 [Apollo / Daily Spark, 2026; Fortune, 2026].
By mid-2026 the trend had intensified. GPT-4-equivalent performance was available for roughly $0.40 per million tokens, down from $20 per million in late 2022 — a 98% drop — while enterprise AI bills were estimated to have risen about 320% [Stan, 2026]. The average enterprise AI budget grew from $1.2 million in 2024 to $7 million in 2026 [Stan, 2026]. The Linux Foundation responded by launching the Tokenomics Foundation to build open standards, cost metrics, and billing transparency for AI tokens, with metrics such as cost-per-intelligence and tokens-per-watt [Stan, 2026].
Why bills grow when prices fall¶
Three forces drive the gap:
- More sessions. A pilot with thousands of calls becomes a production system with millions.
- More tokens per session. Agents send long contexts, tool results, and reasoning traces. An agentic workload can consume 10–100× more tokens than a simple chat exchange.
- Reasoning models emit hidden tokens. Models with chain-of-thought or extended-thinking modes bill for intermediate reasoning tokens. A response that looks like 300 tokens may have cost tens of thousands of reasoning tokens behind it.
Piper Sandler noted in mid-2026 that even a 50% drop in output-token pricing can be overwhelmed by the volume increase from agentic workloads. For a workload of 10 million sessions, reducing input tokens from 20,000 to 2,000 per session can cut monthly cost by about 83% — roughly $54,000 — without changing the model [Piper Sandler, 2026]. Context architecture is the largest single budget variable. Early enterprise context-layer deployments have cut token consumption by 50–75% in tested use cases by pre-materializing and indexing domain knowledge [Piper Sandler, 2026].
Per-developer token consumption has reportedly risen roughly 18.6× in nine months as agentic tools spread, and the cost per interaction for an orchestrated agentic system is about 30× that of a simple 2023 workflow [Stan, 2026].
Hidden cost levers¶
The posted per-token price is only the headline. The real unit cost depends on several mechanics:
| Lever | Effect on bill | How to use it |
|---|---|---|
| Prompt caching | ~90% discount on repeated input prefixes | Cache stable system prompts, RAG context, and conversation history; watch TTL and cache-write premiums. |
| Batch API | Up to 50% discount for asynchronous processing | Use for overnight evals, bulk ingestion, and backfill jobs. |
| Output length | Output is often the dominant cost for short answers; input dominates for long-context agent loops | Cap output tokens, request structured short answers, and right-size context. |
| Tokenizer differences | Models can count 20–35% more tokens for the same text | Compare cost per real document, not per posted price. |
| Reasoning / tool tokens | Hidden reasoning and multi-step tool calls multiply tokens | Count intermediate tokens in traces; set per-step budgets. |
| Cache switching | Switching models mid-session can break provider KV-cache prefixes | Measure full-session cost, not per-call price. |
Cost math examples¶
A few worked examples show where the money actually goes:
| Workload | Key assumption | Monthly cost implication | Biggest lever |
|---|---|---|---|
| 10M sessions, 20K → 2K input tokens, frontier model | Input tokens dominate; output held constant | ~83% reduction, ~$54,000 saved in the cited scenario [Piper Sandler, 2026] | Shrink context |
| 100K agent loops, 20 steps, 4K input / 500 output per step | Sonnet 4.6 ($3 / $15 / MTok), no caching | ~$39,000 per month | Enable prompt caching → 70–80% input-cost cut [Respan, 2026] |
| 3M classification calls per month | GPT-5.4-nano ($0.20 / $1.25 / MTok), 500 in / 50 out | ~$488 / month | Model selection is the dominant lever; frontier model would be 10–100× more [Respan, 2026] |
What this means for agents¶
- Per-request cost is the wrong metric. A cheap request that loops 30 times or spawns reasoning tokens is not cheap.
- Context is the dominant cost. Input tokens are billed on every loop, so a bloated context multiplies quickly.
- Observability is a cost-control tool. You cannot optimize what you cannot see: hidden reasoning tokens, repeated tool calls, bad loops, and wrong model routing all hide in the trace.
- Efficiency and quality are coupled. A smaller, focused context often produces better answers and lower bills.
- Model routing is the primary cost lever. Routing simple requests to small models and reserving frontier models for hard requests is now the first-line cost control, alongside context optimization [Stan, 2026].
Failure modes and metrics¶
| Failure mode | What to watch | Mitigation |
|---|---|---|
| Runaway agent loop | Tokens per session, step count, cost ceiling | Per-step token budgets, max iteration counters, human escalation |
| Hidden reasoning-token blowout | Reasoning-token ratio, time-to-first-token vs total tokens | Gate reasoning models behind need, expose reasoning budgets |
| All-you-can-eat subscription shock | Monthly spend vs budget, per-user spend | Per-user quotas, team budgets, alerts at 50/80/100% |
| Cache misconfiguration | Cache hit rate, effective cost per token | Cache stable prefixes, monitor TTL and write costs |
| Wrong model routing | Cost per resolved task, routed-vs-gold quality gap | Router confidence thresholds, shadow traffic |
Key metrics to track:
- Cost per resolved task (blended cost / successful outcome).
- Tokens per outcome (input + output + reasoning + tool tokens).
- Cost-per-intelligence and tokens-per-watt (emerging standards) [Stan, 2026].
- Cache hit rate and effective cost per cached token.
- Routed-vs-frontier quality gap when using smart routing.
Summary¶
Falling token prices do not guarantee lower bills. They enable new workloads, and those workloads consume more tokens than the old ones. The way to control spend is to control context, loops, tool usage, model routing, caching, and batching — in other words, to engineer the harness, not just buy a cheaper model.
The harness effect: orchestration as the decisive cost lever¶
A 2026 controlled swap from Writer isolates the orchestration layer as the dominant cost variable. The same 22 locked enterprise tasks ran on the same six foundation models (Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, Palmyra X6) under two orchestrators: a frozen conventional production-agent loop and the Writer Agent Harness [Sayed Ali et al., 2026].
| Metric | Frozen baseline | Writer Agent Harness | Change |
|---|---|---|---|
| Blended cost per task | $0.21 | $0.12 | −41% |
| Median wall-clock | 48 s | 27 s | −44% |
| Tokens per task | 14.2k | 8.8k | −38% |
| Task completion quality | 0.78 | 0.81 | +0.03 (directional) |
| Quality per dollar | baseline | +82% | — |
| Task-completions per million tokens | 54.9 | 92.0 | +68% |
Crucially, the harness savings were model-invariant: every model got cheaper by 33% to 61%, while quality gains correlated almost perfectly with the model's baseline strength (r = 0.99). The authors call this harness leverage: stronger models convert better orchestration into better outcomes; weaker models can be overwhelmed by it. On this workload, changing the harness moved cost per task more than switching from the most to the least expensive model did [Sayed Ali et al., 2026].
Provider pricing tiers and hidden multipliers (March 2026)¶
Per-token list prices are only the headline. Stabilarity's March 2026 pricing survey shows three orders of magnitude between commodity and frontier endpoints, and several mechanisms that multiply or divide the effective bill [Ivchenko, 2026].
| Tier | Example models | Input $/M | Output $/M | Notes |
|---|---|---|---|---|
| Frontier premium | GPT-5.2, Claude Opus 4.6 | $1.75–$5.00 | $14.00–$25.00 | Output/input ratios of 5:1 to 8:1 |
| Mid-tier workhorse | GPT-4.1, Claude Sonnet 4.5 | $2.00–$3.00 | $8.00–$15.00 | Handles ~70–80% of enterprise workloads |
| Commodity/open | DeepSeek V3, Grok 4.1, Qwen | $0.075–$0.80 | $0.30–$1.10 | Cost floor; quality varies by task |
Key multipliers:
- Prompt caching: repeated stable prefixes billed at ~10% of standard input cost on Anthropic (and a smaller discount on OpenAI), yielding up to 80% cost reduction for high-reuse workloads.
- Batch API: 50% discount for asynchronous processing with a 24-hour turnaround, ideal for backfills and document pipelines.
- Context-length surcharges: Google doubles input pricing above 200K tokens; OpenAI charges 2× input and 1.5× output beyond 272K tokens.
A workload with 60% prompt reuse processed through batch APIs can see effective costs 70–85% below naive synchronous usage [Ivchenko, 2026].
Cache anti-patterns that silently double bills¶
Prompt caching only works when the request prefix is byte-stable. Several production incidents show how easy it is to accidentally invalidate the cache.
| Anti-pattern | Why it breaks caching | Detected by |
|---|---|---|
| Dynamic live-budget line in the system prompt | The dollar figure changes every turn, invalidating the whole prefix | Agentic Control Plane saw cache hit drop from 57% to 2% and cost per run rise from 18.7¢ to 41.5¢ on Gemini 2.5 Pro [Crowe, 2026] |
| Timestamp in the system prompt | The first bytes differ every request | Basedash found a per-second clock turned every request into a cache miss; moving it to the per-turn message restored the discount [Basedash, 2026] |
| Full tool/RAG context prepended to the prompt | Different or reordered content every turn breaks prefix matching | Move stable schemas into the cached prefix and fetch volatile data on demand [Basedash, 2026] |
| Gateway-side cache regression | Claude Code 2.1.211 fixed a bug where Bedrock, Vertex AI, Mantle, and Azure Foundry billed the trailing system context as fresh input every turn | Audit cache_creation_input_tokens vs cache_read_input_tokens ratios at the gateway [TheRouter.ai, 2026] |
The general rule: keep system prompts, tool schemas, stable history, and durable instructions at the front of the message list; put anything that changes per turn — budgets, timestamps, live memory, recent tool results — at the end. Measure cache_read_ratio per agent and model; a sudden drop is usually a prefix-regression alarm [Basedash, 2026; Crowe, 2026].
Cost attribution: the 340% overrun pattern¶
A 2025 analysis of multi-tenant SaaS products found that teams exceeded LLM cost budgets by 340% on average. The root cause was rarely the per-token rate; it was the absence of per-tenant, per-feature, and per-prompt-template cost attribution [Tianpan, 2026]. An invoice shows what you spent; it does not show which customer or which prompt template is burning 80% of the budget.
| What to attribute | Why it matters | Implementation hint |
|---|---|---|
| Per request / per resolved task | Blended cost is the real unit economics | Tag every trace with task ID, outcome, and model route |
| Per tenant / per user | Prevents a single customer from consuming the entire quota | Gate usage with per-user budgets and alerts at 50/80/100% |
| Per prompt template | Reveals which templates bloat context or trigger reasoning | Version templates and compare cost per resolved task across them |
| Cache read vs. fresh input | Catches prefix-stability regressions | Split input tokens into cache_read and cache_write in telemetry |
Without this granularity, falling token prices hide rising bills because the organization cannot see which workloads are token-maxing.
Deeper dive: why cheaper tokens can mean larger bills¶
Between 2021 and early 2026, the cost of GPT-3-class capability fell roughly 1,000×—from about $60 per million tokens to about $0.06 per million—while enterprise AI spending grew 320%, from $11.5 billion to $37 billion. The organizations spending the most were not victims of price gouging; they were the ones that gained the most from the price drop. As Tianpan frames it, the rebound effect is the Jevons paradox operating in real time: cheaper useful work expands the frontier of economically viable work faster than efficiency can shrink the old work [Tianpan, 2026].
The expansion is structural. When an API call drops from a budget constraint to a rounding error, the design question changes from "can we afford this?" to "should we run three agents or five?". Tianpan identifies four mechanisms that now dominate token consumption: reasoning chains, agent loops, context-window saturation, and multi-agent parallelism [Tianpan, 2026]. A reasoning-mode response can turn a 7-token answer into hundreds of billed internal tokens; a 10-turn ReAct loop can consume roughly 50× the tokens of a single-pass response; and context windows that grew from 4K to 1M tokens encourage teams to load whole repositories, session histories, and document sets by default. Aggregated usage data—over 100 trillion tokens—shows average prompt tokens per request grew 4× and completion tokens nearly tripled in 13 months, while reasoning models exceeded 50% of total token consumption by mid-2025 [Tianpan, 2026].
At the team level the savings are quickly reinvested. A cheap, single-purpose valuation calculator can cost under $0.0002 per use, while a pitch-deck analyzer running five sequential calls over multi-megabyte documents can cost $0.20 per use—three orders of magnitude more per resolved intent, even though both ride the same falling per-token prices [Tianpan, 2026]. The same price collapse that makes the calculator trivial to operate makes the analyzer economical to build.
| Mechanism | How it multiplies the bill |
|---|---|
| Reasoning chains | Internal reasoning tokens are billed before any visible answer is produced |
| Agent loops | Planning, tool selection, parsing, retries, and final response each add tokens |
| Context saturation | RAG contexts, repo-level prompts, and long session histories fill large windows |
| Multi-agent parallelism | Several models each receive their own context for the same user intent |
This is why cost control cannot be delegated to model-price forecasts. The budget moves when the architecture changes; the harness—context design, routing, loop limits, and attribution—determines how much of the cheaper-token dividend is retained and how much is converted into new workload capacity.
References¶
- Apollo / Daily Spark. "Cheaper Tokens, Bigger Bills." Jun 2026. https://www.apollo.com/wealth/the-daily-spark/cheaper-tokens-bigger-bills
- Fortune. "Tokens are getting cheaper, but AI costs keep climbing anyway." Jun 2026. https://fortune.com/2026/06/17/why-is-ai-spending-increasing-as-tokens-get-cheaper-jevons-paradox/
- Jevons, W.S. "The Coal Question." 1865.
- Menlo Ventures. "2025: The State of Generative AI in the Enterprise." Dec 2025. https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/
- Piper Sandler / Stockwirex. "AI Token Costs Surge: Why Enterprise Bills Keep Climbing." Jul 2026. https://stockwirex.com/analysis/enterprise-ai-token-costs/
- Respan. "OpenAI vs Anthropic Pricing (2026): The Real Cost Math, Side by Side." May 2026. https://www.respan.ai/articles/openai-vs-anthropic-pricing-2026
- Stan, A.M. "AI token prices fell 98% but enterprise bills tripled." The Next Web, Jun 2026. https://thenextweb.com/news/token-prices-fell-98-enterprise-ai-bills-tripled-now-the-industry-wants-a-standards-body-to-explain-why
- Tianpan. "The Inference Cost Paradox: Why Your AI Bill Goes Up as Models Get Cheaper." Apr 2026. https://tianpan.co/blog/2026-04-14-the-inference-cost-paradox