How do I estimate LLM costs?

Step by step

1. 1List each AI feature and how often it runs: per user per day, per document, per ticket.
2. 2Measure or estimate tokens per call for the system prompt, context, user input and output. A token counter on real examples beats a guess.
3. 3Split input into the part that repeats (cacheable with prompt caching) and the part that changes.
4. 4Count calls per task: an AI agent or a RAG pipeline may make several model and embedding calls per user request.
5. 5Apply each model's current input, cached input and output prices from the provider's price page.
6. 6Add a margin for retries, errors and longer-than-average inputs, then multiply by expected volume.

Check it against reality

Estimates are often wrong because conversations get longer, agents take more steps than planned and outputs run long. Log token counts and cost per feature from the first prototype, and recheck the estimate with real traffic.

Cost depends on the token mix, not only the total: an output-heavy feature and an input-heavy one with the same total can cost very different amounts.