LLM API pricing explained: what you are actually paying for

Every major provider bills the same basic way, but the headline per-million-token price is the least useful number on the rate card. Here is what the bill is actually made of.

Pricing fundamentals · 9 min read · Last reviewed 2026-07-28

Key takeaways

Every provider bills the same four things: input tokens, output tokens, cached input reads, and cache writes. Output is the expensive one.
The headline price per million tokens tells you almost nothing without an estimate of your input:output ratio.
Reasoning models add a fifth line item you cannot see or count in advance, and it often dominates.
Two discounts — batch processing and prompt caching — are large, widely available, and routinely left on the table.
Cost per task, not cost per token, is the number that decides which model is cheapest.

Model providers publish pricing that looks trivially comparable. Model A is $2.50 per million input tokens, model B is $3.00, therefore model A is cheaper. This reasoning is wrong often enough to be dangerous, and the reasons why are worth understanding before you commit to an architecture.

What follows is every line item that can appear on an LLM API bill, roughly in order of how much it usually matters.

Input tokens

Input tokens — sometimes called prompt tokens — are everything you send to the model in a request. That includes the system prompt, the full conversation history, any documents or retrieved context, tool and function definitions, and the user's current message. All of it is billed at the input rate on every request.

Two things about input tokens surprise people. The first is that tool definitions count. If you pass a set of twelve function schemas with the request, those schemas are input tokens, on every single call, whether or not the model uses any of them. Verbose JSON schemas can quietly add a thousand tokens to every request in your application.

The second is that you are billed for the conversation history, not the provider. LLM APIs are stateless: they retain nothing between calls. A chat application maintains the illusion of memory by resending the entire transcript each turn, and each resend is billed in full. This is the single largest source of underestimation in cost modelling, and it has its own guide: why multi-turn conversations grow quadratically.

Output tokens

Output tokens are what the model generates. They are priced at roughly three to five times the input rate across essentially every provider — look down any rate card and you will see the same asymmetry, whether the model is a $0.15 budget option or a $5.00 flagship.

This is not a pricing convention. Reading a prompt and writing a reply are genuinely different operations on a GPU: the prompt can be processed in parallel in a single pass, while each output token must be generated sequentially, one forward pass at a time, bounded by memory bandwidth rather than compute. The mechanism is explained here, and it is worth understanding because several non-obvious optimisations fall straight out of it.

The practical consequence: your input:output ratio determines which model is actually cheapest for you, and it varies enormously by workload. A summarisation task might send 10,000 tokens and return 200 — overwhelmingly input-weighted. A code generation task might send 500 and return 3,000 — the opposite. Two models can swap places in the cost ranking depending on which of those you are doing, which is why comparing headline input prices in isolation is meaningless.

Cached input tokens

Every major provider now offers prompt caching: if a request begins with a prefix the provider has recently processed, it can reuse the cached internal state instead of recomputing it, and charges you a fraction of the normal input rate for that portion. Discounts are steep — commonly a 50% reduction, and as much as 90% for explicit cache reads.

For applications with a large stable prefix — a long system prompt, a fixed set of tool definitions, a document being asked many questions about — this is usually the largest single cost lever available. It is also the easiest to get no benefit from at all, because caching requires the prefix to be byte-identical and stable, and a surprising number of applications defeat it by injecting a timestamp or a session ID at the top of the prompt.

The details differ meaningfully between providers, including a case where writing to the cache costs more than not using it. See the prompt caching guide.

Reasoning tokens

Reasoning models — OpenAI's o-series, and the extended-thinking modes of several other model families — generate internal chain-of-thought before producing their visible answer. Those tokens are billed at the output rate. In most cases they are not returned to you.

So the bill contains a quantity you cannot count in advance, cannot see afterwards in full, and which is charged at the most expensive rate. It commonly runs two to five times the visible output, and scales with how hard the model finds the problem, which means it is workload-dependent in a way that is difficult to predict from the prompt alone.

This is the most common cause of a genuinely shocking first invoice. If you are considering a reasoning model, read how to bound the invisible half of the bill before you deploy.

The discounts most people miss

Batch processing

Every large provider offers an asynchronous batch endpoint at roughly half the synchronous price, in exchange for a completion window measured in hours rather than seconds. The discount is around 50% and applies to both input and output.

The question to ask is not "is my application real-time?" but "which parts of it are?" Plenty of production workloads that feel interactive contain large batch-eligible components: overnight enrichment, classification of a backlog, generating embeddings, evaluation runs, report generation, content moderation sweeps. Moving those to batch is a straight halving of their cost for no quality change whatsoever. It is the highest ratio of saving to effort available anywhere in LLM cost optimisation.

Model tiering

The price spread across a provider's own lineup is often more than an order of magnitude — and across providers, the cheapest capable models cost one-twentieth of the flagships. Most applications route everything to one model because that is simpler, and most applications contain a mix of tasks with wildly different difficulty.

Classification, extraction, routing, formatting and simple rewriting rarely need a flagship. Reserve the expensive model for the genuinely hard step. This requires actually evaluating the cheap model on your task rather than assuming — but when it works, it is a larger saving than any prompt-level optimisation.

Committed-use and enterprise rates

Published rates are list prices. At meaningful volume, providers negotiate. If you are spending five figures a month, you are probably paying more than you need to.

What is not on the rate card

Several real costs do not appear as line items but land on your invoice anyway.

Retries. A request that fails validation, gets rejected by your own guardrail, or produces malformed output that you retry is billed the first time as well as the second. Applications with strict output-format requirements and no schema enforcement can spend a surprising fraction of budget on discarded generations.

Tool-call round trips. An agent that calls three tools before answering has made four billable requests, and each one resends the entire accumulated context. Agentic architectures multiply request count, and each request is larger than the last. This is where cost estimates fail most badly in practice.

Non-text modalities. Images, audio and video are billed on separate schedules — typically converted to a token-equivalent at rates that make a single high-resolution image comparable to a page of text. If your application handles images, text-token arithmetic will understate it.

Failed context-window overflows. Exceeding a model's context limit produces an error, not a truncation, at most providers. You have still paid nothing for a rejected request — but you have paid for whatever work led up to constructing it, and the retry with a shortened prompt costs again.

How to actually compare two models

Compare cost per completed task, not cost per token. The procedure:

Take a real prompt. Not a representative sample — an actual one, including the system prompt and tool definitions, at realistic length.
Measure the output length you actually need, from logs if you have them. Do not guess this; it is usually the dominant term.
Count the requests per task. A single completion is one. A three-turn conversation with two tool calls is closer to eight, each larger than the last.
Apply your realistic cache hit rate, not the theoretical maximum.
Add a reasoning multiplier if the model is a reasoning model — assume 3× visible output as a starting point and measure the real figure early.
Then compare totals, and only among models you have verified can actually do the task.

That last point is the one that matters most. A model that is 95% cheaper and fails 20% of the time is not cheaper, once you count the retries, the fallback calls to the expensive model, and the cost of the failures you did not catch. Establish capability first, then optimise price within the set of models that work.

Try it

The calculator handles steps 1, 2, 4 and part of 3: paste a real prompt, set your expected output length and monthly volume, enable multi-turn mode if you are building anything conversational, and compare all thirteen models at once. The methodology page documents exactly what it does and does not account for.