Costing an LLM feature starts easy and gets difficult fast. The published rates are public and precise. What is not public - and this is the finding worth leading with - is almost everything downstream of them. When we went looking for citable figures on tokens consumed per request, or cost per user per month, we found that essentially every number in circulation traces back to blog posts performing arithmetic on the same public price lists. There is no observed data. Anyone quoting you a benchmark for cost per user is quoting a model, not a measurement.
The rates themselves, as published in August 2026, per million tokens. Anthropic: Claude Sonnet 5 at $2 input and $10 output; Claude Opus 5 at $5 and $25; Claude Haiku 4.5 at $1 and $5. OpenAI: GPT-5.6 Luna at $0.20 and $1.20; Terra at $2 and $12; Sol at $5 and $30. Google: Gemini 3.6 Flash at $1.50 and $7.50; Gemini 2.5 Flash-Lite at $0.10 and $0.40. Check them before relying on them - these move.
The structural fact underneath those tables matters more than any individual row: output tokens cost five to six times input tokens, at every provider, across every tier. That single ratio determines most bills. A feature that reads a large document and returns a short answer is cheap. A feature that takes a short prompt and generates pages is expensive, even though it feels like less work. If you are optimising cost, constrain output length before you consider changing model.
Two discounts are real, published, and worth designing around. Batch processing is fifty per cent off input and output, confirmed independently by all four major providers - it applies to any workload that does not need an immediate answer, which includes every evaluation run you will do. Prompt caching reads cost a tenth of the input rate. But the caching headline hides an asymmetry: writing to the cache costs more than a normal input token - 1.25 times for a five-minute time-to-live and twice for an hour. A cached prefix has to be read several times before it breaks even. Cache a system prompt served to thousands of requests and the saving is enormous; cache something read twice and you have paid more. Google additionally charges $1 per hour for cache storage, which the other providers do not.
On self-hosting, the verified numbers say something unexpected: the cost varies more by vendor than by anything else. Eight H100s cost about $88 an hour on Google Cloud's A3 High, $55 an hour on an AWS p5.48xlarge on demand, and roughly $32 an hour on Lambda at $3.99 per GPU-hour. That is a spread of nearly three times for identical hardware. Anyone telling you self-hosting costs a fixed amount has not priced it properly. As for the break-even utilisation against API pricing - the threshold at which running your own beats paying per token - we could not find a single methodologically grounded published figure. Every number in circulation is an unsourced blog. Compute it yourself from your own throughput, and treat anyone quoting a universal threshold with suspicion.
The costs that surprise teams are not the generation costs. Embeddings are cheap per token - OpenAI's small embedding model is $0.02 per million - but the cost story is re-embedding: change your chunking strategy or your embedding model and you pay to reprocess the entire corpus, and teams do that several times in the first year. Vector database hosting has a floor rather than a marginal cost, with Pinecone's enterprise tier starting at $500 a month. Retries bill at full price with no discount, and a failed request still charges you for the input plus whatever output was generated before it failed.
The practical approach, given that nobody can give you a credible benchmark: instrument before you scale. Log tokens in and out per request from the first day, tagged by feature. Within a fortnight of real usage you will have better data than any published figure, because it will be your prompts, your documents and your users. That measurement is also what turns a cost conversation into an engineering one - you cannot shorten a prompt you have never measured.
