Cheaper tokens, bigger bills: how to control AI token costs

The price of using AI keeps falling, and the bills keep going up. Both of those are true at the same time, and the gap between them is where a lot of money is quietly going.

The falling part is real. By one venture firm’s measure, the cost of getting a fixed amount of AI work done has dropped roughly tenfold a year, so the same task that cost dollars in 2023 costs cents now. The rising part is just as real. Enterprise spending on generative AI tripled to about $37 billion in 2025, and Gartner projects that by 2028 what a company spends on AI coding tools will pass what it pays the average developer. Nearly a quarter of technology leaders already spend $200 to $500 per developer every month just on the tokens their coding tools burn through.

So the invoice climbs while the unit price collapses, and for a lot of companies the result is a bill that grows every quarter with little to point to that pays for it. It does not have to, whether you are a two-person firm or a company with several teams and its own engineers.

Cheaper tokens, bigger bills

First, the one piece of vocabulary you need, in plain terms. AI tools charge by the token, a chunk of text about three-quarters of a word. You pay for every token you send in and every token the model writes back. A short question and answer might be a few hundred tokens; a long document stuffed into a request can be tens of thousands.

Here is the trap. When the price per token falls, using more of them gets easier to justify, so people send longer prompts, keep more history in the conversation, and turn on automated tools that call the model again and again. Usage climbs faster than the price drops, and the total goes up. Cheaper tokens invite bigger usage, and bigger usage is a bigger bill.

The numbers bear it out. Even as the per-token price falls, Menlo Ventures found that total spending on generative AI keeps rising, because volume more than makes up the difference. The lesson for anyone watching the invoice is uncomfortable but freeing: you cannot buy your way out of a usage problem with a cheaper price. Switching to a cheaper model feels like the obvious fix, and it is the one most teams reach for first, but if the underlying usage is undisciplined the bill just climbs again from a lower base.

Treat it like any other budget

Before any of the technical moves, the mindset matters more, and it is the one thing an owner or executive can set that an engineer cannot.

Most companies do not manage AI spend the way they manage other money. As MIT Sloan Management Review put it recently, “few companies apply the same financial discipline to artificial intelligence as they would to a new factory or piece of machinery.” A new machine gets a business case, an owner, and a check on whether it earned its keep. AI spend often gets none of that. It shows up as one line on a cloud bill that grows on its own, with no one able to say which use is responsible for which dollar.

The reframe is to treat AI spend as ordinary capital discipline, not a new and mysterious technical chore. Deloitte’s guidance to finance leaders is to “govern AI with the same rigor as any other strategic investment,” which means tying spend to a business case, giving each use an owner, and modeling where the cost is heading before it gets there.

That reframe cuts both ways, and this is the part worth slowing down on. The goal is not the smallest possible bill. Gartner’s finance analysts put it bluntly: “simply spending more on AI does not, by itself, equate to better business outcomes.” The reverse is just as true. Cutting spend on a use that clearly pays for itself, only to hit a budget number, is as much a mistake as pouring tokens into one that never will. The target is the most value per dollar, not the fewest dollars.

One more piece of discipline protects you from a common overreaction. AI investments tend to pay back over a longer horizon than ordinary software. Harvard Business Review notes that a typical AI payback runs two to four years, against the seven to twelve months companies expect from most technology. So a use that has not paid back in its first two quarters is not automatically a failure to be cut. The skill is telling a slow, strategic bet apart from a genuine leak, and that requires seeing your spend clearly, which is where the practical work starts.

Where the money actually goes

Your AI bill is not one number you can only accept or reject. It is a product of four things you can each control:

what you spend ≈ (how many calls) × (how much you send and get back each time) × (the price of the model you pick) − (what you reuse instead of re-sending)

Every real cost-control move maps to one of those four. And the reason this framing helps is that most teams pull only one lever, the price, by switching to a cheaper model, when the biggest savings usually sit in the other three. Take them one at a time, roughly in the order that matters most.

The biggest lever: how much you send and get back

This is the one almost everyone underuses, so start here.

A model has no memory of its own. Every time you continue a conversation, the whole history so far is sent again for the model to re-read, and you pay for all of it, every turn. A chat that has run for twenty exchanges is re-sending nineteen exchanges’ worth of text on the twentieth. Long documents pasted into a request are the same story: you pay to send the whole thing whether the answer needed all of it or not.

Automated agents make this dramatically worse, and it is where the scary bills come from. An agent works in a loop, calling the model over and over to take steps toward a goal, and each loop re-reads the growing history. Anthropic’s own engineers measured it: “agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats.” A single automated workflow left to run can quietly cost fifteen times what the same question would in a chat window.

There is a second, smaller edge here worth knowing. The text a model writes back costs several times more per token than the text you send in, commonly four to five times more on the major providers. So a habit of asking for long, chatty answers when a short structured one would do is more expensive than it looks.

None of this requires an engineer to explain to the person paying the bill. The instruction from the top is simple: send less, and get back less. In practice your team does that by keeping only the relevant history rather than the whole conversation, pulling in only the part of a document the task needs instead of the whole file, asking for short and structured answers, and putting firm stopping conditions on any agent so a loop cannot run forever. Cloud providers now build this into their guidance; Amazon’s well-architected cost advice for AI explicitly calls for limiting prompt and response sizes and setting exit conditions on agents to avoid runaway consumption. For teams that need to go further, research tools such as Microsoft’s LLMLingua can compress a prompt several-fold before it is ever sent, though for most companies trimming habits get most of the win.

The next lever: the price of the model you pick

Only after you have the size of your requests under control does swapping models become a clean win rather than a temporary one.

Here the key fact is that model prices span an enormous range for the same task, and the most expensive model is rarely as much better as it is pricier. A Stanford study known as FrugalGPT found that prices across models differ by up to two orders of magnitude, and that you can match the quality of a top model at up to 98 percent lower cost by sending easy cases to a cheap model and only escalating the hard ones to the expensive model. A Berkeley project called RouteLLM reported the same shape of result: routing each request to the right-sized model held about 95 percent of the top model’s quality while cutting cost by well over half.

The strategy behind that is old, not new. Most tasks do not need the frontier model any more than most trips need a race car. The frontier model overshoots what the job actually requires, and a cheaper, good-enough model does the work for a fraction of the price. Matching the model to the value and difficulty of the task, rather than defaulting every request to the best and most expensive option, is where the price lever pays off.

The obvious worry is quality, and it is the right worry. A cheaper model that fails quietly on important work is not a saving; it is a hidden cost. This is the same trap covered in our piece on why a cheaper coding model can cost you more: once you count the failure tax, the time your people spend catching and fixing a model’s mistakes, a cheap model that fails often can be the most expensive choice on the table. The answer is not to avoid cheaper models. It is to decide which model handles which work by testing them on your own tasks before you trust them, rather than by guessing. Reserve the expensive model for the work where a silent mistake is costly, and send the routine, easy-to-check work to the cheap one.

One free variant of this lever: for work that is not time-sensitive, such as processing a backlog overnight, the major providers offer a batch option at roughly half price. If the answer does not have to come back in the next few seconds, it should not be paying the premium for speed.

The easiest lever: stop re-paying for the same thing

This one is close to free money, and many companies leave it switched off.

A great deal of what you send an AI is the same text over and over: the standing instructions that set up the assistant, the policy document it has to follow, the product catalog it answers from. Normally you pay full price to send that repeated block on every single request. Prompt caching lets the provider recognize the part that does not change and charge you far less for it after the first time. Anthropic discounts a cached read by around ninety percent; OpenAI applies a similar discount automatically once a request is long enough. Amazon lists it as a named cost practice in its guidance precisely because it changes nothing about the quality of the answer while cutting the bill on everything you send repeatedly.

We will be honest that this one is easy to miss, because we missed it ourselves. In a three-week field study of our own agent system, we noted that not one of the run’s requests used caching, a clear saving we had left on the table. It is the first thing to turn on, and the cheapest, because there is no quality trade to weigh.

The safety lever: cap it so it cannot run away

The moves above lower the bill. This one keeps a bad day from becoming a bad month.

The single fastest way an AI bill balloons is an automated process with no ceiling: an agent that gets stuck in a loop, a misconfigured job that retries forever, a test left running over a weekend. The protection is a hard cap, set in two places. First, a limit on any single automated run, so one process cannot spend past a point no matter what it does. Second, a monthly limit per team or per use, with alerts as it approaches the line, so nothing surprises you at the end of the month.

This is exactly how we run our own system. Every automated run carries a cost ceiling: at 85 percent of its budget the system nudges the agent to wrap up, and at 100 percent it stops the run outright, so a loop physically cannot spend past its allowance. On top of that, each team and department has a monthly cap that is checked before every charge, with alerts at 80, 95, and 100 percent. The FinOps Foundation, the industry body for cloud cost management, now recommends exactly this pattern for AI: strict usage limits paired with automatic alerts on unusual spending. A cap is not a tax on ambition. It is the seatbelt that lets you drive faster.

But first, meter it

Everything above depends on one thing that has to come first: you cannot manage a cost you cannot see.

Most companies see AI as a single monthly total, not a breakdown of which team, which feature, or which automated agent spent what. Without that breakdown you are guessing at which lever to pull. The first move, before any optimization, is to meter: record what every use costs, so spend can be attributed to the team or feature responsible for it. The FinOps Foundation now defines cost per token and cost per request as standard measures to track, and its 2026 survey found that 98 percent of organizations now actively manage AI spend, up from 31 percent two years ago. This has gone from a nicety to standard practice in about the time it takes to write a budget.

Our own control system records the cost of every single generation it runs, with the tokens, the price, and which agent was responsible. One honest caveat, since it matters for how much precision to expect: we estimate those costs locally rather than reading the provider’s exact billed figure, which is accurate enough to decide what to do but not an accountant’s ledger. For a finance-grade number, read the usage figure the provider itself reports. As the engineer and writer Simon Willison keeps demonstrating with his own running tally of model prices, the discipline is simply to always know what a request costs. You do not need perfect numbers to act. You need to stop flying blind.

If you are a small business

The right approach depends on your size, because a two-person firm and a company with several teams face genuinely different problems. Start with the small end.

The most important decision a small business makes about AI cost is to buy and configure rather than build. The evidence is stark: MIT’s 2025 study found that buying AI tools from vendors succeeds about 67 percent of the time, while building them in-house succeeds only about a third as often. Your edge as a small company is not a custom cost-control platform, and building one drains the scarce attention you should spend on the work that actually differentiates you. Buy the capability and configure it well.

Concretely, that means:

• Turn on caching. It is usually a setting or a small amount of setup, and it cuts the cost of everything you send repeatedly.
• Pick a cheaper default model, and reserve the expensive one for the hard work. Do not run every request through the priciest option out of habit.
• Use the batch option for anything that is not urgent. Roughly half price for work that can wait.
• Put a hard cap on anything automated. Especially any agent that runs in a loop. This one control prevents the worst-case bill.
• Look at one dashboard once a month. Just enough to see whether spend is tracking with value, and to catch a surprise early. This is the small-business version of what larger firms call showback: showing what was spent, so the spending has an owner.

The number to watch is not your total bill. It is whether the spend is producing something worth more than it costs, one use at a time.

If you are a mid-sized organization

With several teams, multiple use cases, and your own engineers, the problem changes. Now the risk is not one runaway process but many small, uncoordinated ones, each team solving the same cost problem slightly differently or not at all. The answer is to make the good defaults the path of least resistance.

That means going a step further than the small-business checklist:

• Attribute spend to each team, feature, and agent. Not one company total, but a breakdown that tells you where the money goes and who owns it.
• Set a house rate card and route to the right-sized model. Give teams a default cheap model and a clear rule for when a task earns the expensive one, so routing is a standard, not a per-engineer judgment call.
• Enforce the rules in one shared place. Run requests through a single internal gateway that applies caching, trimming, and caps for everyone, so each team does not re-solve it, and a new project inherits the discipline automatically.
• Watch for anomalies. Automatic alerts when a use suddenly costs several times its normal amount catch the misconfiguration before the invoice does.
• Move from showing to billing. Start by showing each team what it spent. As the numbers become trusted, move to charging each team’s own budget for what it uses, which is the point at which people start economizing on their own. Deloitte’s finance guidance frames this progression, from showback to internal chargeback, as the way a growing organization gets AI spend under control.

This is the level at which a real control system earns its keep, the kind of thing we build into our own platform. It is also the level at which a small business should not try to build one. Match the effort to the size of the problem.

Which lever first

Put together, the moves have a natural order, and following it saves you from optimizing the wrong thing:

1. Meter. See where the money goes, per team and use. You cannot manage what you cannot see.
2. Cap. Put ceilings on automated runs and monthly team spend, so a bad day cannot become a bad month.
3. Cache. Turn on reuse for the parts of every request that repeat. It is the free win, no quality trade.
4. Trim. Send less history and shorter answers; put stopping conditions on agents. This is usually the biggest lever.
5. Right-size. Match the model to the value of the task, tested against your own work, not guessed.

And a short list of what not to do, because these are the common ways companies waste the effort:

• Do not just switch to a cheaper model and call it done. It is one lever of four, and the savings evaporate if the underlying usage is undisciplined.
• Do not cut spend on a use that pays for itself to hit a budget number. That is the value-per-dollar point in reverse.
• Do not build a cost-control platform if you are small. Buy and configure. Save your build effort for what makes you different.
• Do not chase the last cent. Past a point, the engineering time spent optimizing costs more than the tokens it saves. Cost control is a means to value, not a hobby.

What we have learned running our own

We are not writing this from the sidelines. We run a standing system of automated agents, and we instrument every part of its cost the way this article describes: a ceiling on every run, monthly caps with alerts, caching built in by default, history trimmed automatically as a conversation grows, cheaper models for the routine work, and alerts when a use spends more than it should.

The result is that the economics stay boring, which is the goal. In our three-week field study, a team of twenty agents working around the clock cost about $12 in modeled token terms, a little over a cent per run. Those are modeled figures, not a billed invoice, and as noted above that same run left caching switched off, so even our own numbers had room to improve. The point is not the exact dollar figure. It is that when you meter, cap, and control usage from the start, the bill stops being a source of surprise and becomes a number you can plan around. That is the same discipline covered in our work on designing agents as reliable loops rather than open-ended processes.

Spend for value, not for its own sake

The point of controlling cost is not a smaller bill. It is to free money up for what works.

Bessemer, the venture firm, calls the goal value density: how much useful output you get per dollar, rather than how few dollars you spend. That is the right target for a buyer too. Starve a use that pays for itself to hit a budget number and you have cut the wrong thing.

So keep two questions in view. First, judge a use by its true cost, the volume side in this article plus the quality side in the failure tax, what its mistakes cost to fix, rather than by its invoice alone. Second, spend the way careful investors do on anything unproven: start small, prove the value on one use, then scale what earns it, the discovery-driven approach the strategist Rita McGrath described long before AI made it urgent.

This is the work we do with clients: finding where AI spend actually goes, which of it pays for itself, and what to change first. If your bill is climbing faster than the value you can name, our AI strategy and roadmap work puts a real number on where the money goes and where it should, and our evaluation work tells you which cheaper models are safe to trust before you switch. If that is on your desk now, start a conversation.