Why your AI bill doesn't match what you typed

AI usage is billed by the total context the model reads on every turn, not by the number of messages you send. Your entire conversation history, attached files, system instructions, and tool definitions get re-processed each time you hit enter. That is why a single message in a long chat can consume a disproportionate share of your usage limit, and why your bill feels disconnected from your typing.

That's the answer. The rest of this piece is the mechanics, because the mechanics are where the money is.

First, the moment that made me write this. I was mid-session in a long working chat when one message... one... burned through roughly 30% of my usage limit. I hadn't pasted a novel. I'd typed two sentences. If that has happened to you and you concluded the meter was broken: it isn't. It's working exactly as designed. The design is just not what most people assume.

What actually counts as a "token" in an AI conversation?

A token is a chunk of text, roughly three-quarters of a word in English. "Structural engineering" is about four tokens. Every provider (Anthropic, OpenAI, Google) meters usage the same fundamental way: tokens in, tokens out. Input tokens are everything the model reads. Output tokens are everything it writes.

The part that surprises people is what counts as input. It is not your message. It is:

• Your message
• The entire conversation history before it
• Every file you attached, still sitting in context
• Images (converted to token equivalents, and they are not cheap)
• The system prompt and any tool or function definitions the product loaded for you

You pay for the whole stack. Every turn. This principle is identical across Claude, GPT, and Gemini. The differences between vendors are pricing tiers and caching mechanics, not the underlying model of what gets counted.

Why does reopening an old chat cost so much?

Because the model has no memory. Not in the way you'd assume.

Each API call is stateless. When you send message forty in a conversation, the model does not "remember" messages one through thirty-nine. It re-reads them. All of them. Think of an actor who must re-read the entire script from page one before delivering their next line. Every line, every time. The longer the play runs, the more expensive each new line becomes.

So the cost of a conversation compounds:

• Message 5 in a chat might process a few thousand tokens.
• Message 40 in the same chat, with two PDFs attached along the way, might process a hundred thousand.
• Same two sentences typed. Wildly different meter reading.

This is the whole explanation for my 30%-in-one-message moment. The message was small. The context it rode in on was enormous.

Practical consequence: long-running chats are a cost decision, not just a convenience decision. Starting a fresh conversation with a short summary of prior context is often 10x cheaper than continuing a bloated one.

How do model tier and reasoning settings change the cost?

Two multipliers sit on top of the context problem.

Model tier. Every provider ships a ladder: small and fast, mid-range, frontier. Anthropic's Haiku/Sonnet/Opus ladder and OpenAI's mini-to-frontier structure follow the same logic. The price spread between the bottom and top of a ladder is roughly an order of magnitude, sometimes more. Sending a formatting task to a frontier model is paying surgeon rates for a bandage.

Reasoning effort. Modern models can "think" before answering: extended thinking in Claude, reasoning models in the GPT line, thinking budgets in Gemini. Those internal reasoning tokens are billed as output, even though you never see most of them. A hard question with high reasoning effort can produce thousands of billed tokens of invisible work. Sometimes that is exactly what you want. It should be a choice, not a surprise.

Caching, the partial rescue. All three vendors now offer some form of prompt caching: the stable prefix of your context (system prompt, tool definitions, that PDF you keep referencing) gets stored and re-read at a steep discount, provider-dependent but typically 50 to 90% cheaper than fresh input. Caching is why the same long conversation can cost noticeably different amounts depending on how the product you're using structures its requests. If you are building on the APIs directly, cache structure is one of the biggest cost levers you control. The vendors document this well: Anthropic, OpenAI, and Google each publish their own mechanics. Read the primary source, not a reseller's summary.

Can you reduce this with local models?

Sometimes. Honesty required here, because this is where consultants tend to oversell.

I run local models myself: open-weight models on my own hardware, served over my own network, reachable from anywhere I work. Not because it's fashionable. Because certain workloads never need a frontier model and never need to leave my machines.

What local models are documented and proven good at:

1. Embeddings. Vectorising documents for search and retrieval is a solved local problem. Paying API rates to embed a large corpus is usually money on fire.
2. Classification and routing. "Is this page a drawing or a text document?" A small local model answers this all day for the cost of electricity.
3. Privacy-constrained preprocessing. If documents cannot leave your infrastructure (a GDPR requirement, a client confidentiality clause), local isn't a cost optimisation. It's the only compliant option. I built an enterprise search system for a European engineering firm under exactly this constraint: nothing leaves their servers, full stop.

What local models are not, yet: a like-for-like replacement for frontier reasoning. On hard synthesis, long-context reliability, and agentic tool use, the gap is real. Anyone telling you to swap your frontier model for a local one across the board is optimising your bill at the expense of your output.

The pattern that actually works is hybrid routing: local models for the high-volume cheap work (embeddings, classification, extraction triage), frontier APIs for the judgment calls, with caching structured deliberately on the API side. The economics of that split depend on your volumes, and I'd be speculating if I gave you a universal number. Run your own arithmetic on your own workload. The point is that the routing decision exists, and most teams never make it consciously.

What does this have to do with answer engine optimization?

Everything, and this is the part almost nobody selling "AEO services" will tell you.

Answer engines (ChatGPT, Perplexity, Google's AI Overviews, Claude with web search) are the same models, under the same token economics, reading your website instead of your chat history. When a model retrieves your page to answer someone's question, your content is now someone else's context window. The same constraints apply:

• Context is expensive, so retrieval is selective. Models work with extracts, not whole sites.
• A page that answers the question in its first two sentences is a cheap, high-confidence extract. A page that winds up for four paragraphs before the point is an expensive, low-confidence one.
• Clear structure (descriptive headings, direct answers, real HTML text instead of text baked into images) makes your content parseable at low effort. Parseable at low effort is what gets cited.

I'll be precise about epistemic status here: vendors do not publish their citation-selection logic, so the last step is inference from mechanics rather than documentation. But the mechanics are not speculative. Token economics shape what gets read, and what gets read shapes what gets cited. If someone claims to optimise your content for AI systems and cannot explain how those systems meter and allocate context... that tells you something.

Practical takeaways

1. Long chats compound cost. Start fresh conversations with a short summary instead of riding one chat for weeks.
2. Attachments persist. That PDF you dropped in twenty messages ago is still being re-read every turn.
3. Match the model tier to the task. Frontier models for judgment, small models for mechanical work.
4. Reasoning effort is billed output. Turn it up deliberately, not by default.
5. If you build on the APIs: structure your prompts for caching. Stable prefix first, volatile content last.
6. Embed and classify locally where volumes are high; keep frontier models for what they're uniquely good at.
7. If your data can't leave your infrastructure, local isn't optional. Design for it from day one.
8. For your website: answer the question in the first two sentences. The machines reading it are on a meter too.

Frequently asked questions

Why does my AI usage cost so much when I barely typed anything? Because billing is based on the full context re-read on every turn: conversation history, files, images, and system instructions, not just your new message. A short message in a long chat can process a hundred times more tokens than the same message in a fresh chat.

Do Claude, GPT, and Gemini all bill the same way? The principle is identical: input tokens plus output tokens, with the full context counted as input on every turn. The differences are in pricing tiers, caching mechanics, and how each vendor's products manage context behind the scenes.

What is prompt caching and does it actually save money? Caching stores the stable part of your context (system prompts, tool definitions, reference documents) so it can be re-read at a discount, typically 50 to 90% off input rates depending on the vendor. For API builders it is one of the largest single cost levers available.

Can I run open models locally instead of paying for GPT or Claude? For embeddings, classification, and privacy-constrained preprocessing: yes, and you probably should. For frontier-level reasoning and agentic work: not at parity yet. The proven pattern is hybrid, with local models handling the high-volume cheap work and frontier APIs handling the judgment.

What is answer engine optimization (AEO)? The practice of structuring content so AI answer engines can retrieve, parse, and cite it. Since those engines are LLMs operating under token economics, AEO fundamentals are the fundamentals in this article: direct answers, clear structure, machine-readable text.

How do AI models decide what content to cite? Vendors don't publish their selection logic, but the mechanics constrain it: retrieved content must be parsed as context, context is metered, and clearly structured pages that answer questions directly make cheaper, higher-confidence extracts than sprawling ones.

Where this goes next: the citation question deserves its own piece. How answer engines choose their sources is the closest thing the industry has to a new PageRank, and almost nobody is examining it from the mechanics up. That's the follow-up.

If you're weighing this kind of infrastructure decision for your own stack (hybrid routing, caching architecture, or an AI system your data governance team will actually approve), this is the problem class I build for.