How do I reduce LLM API costs?

Start by measuring the token mix

Log token counts per feature, split into fresh input, cached input and output. Cost depends on that split, not just the total: an output-heavy feature and an input-heavy one with the same total can cost very different amounts, and the cheapest model for one may be the dearest for the other.

Once you can see which feature and which part of the prompt uses the most tokens, you know which lever to pull first.

The levers, roughly in the order to try them

Each one is independent, so you can combine them.

1. 1Prompt caching: put fixed instructions, tool definitions and shared documents at the start of the prompt and changing content after it, so repeated input is billed at the cached rate.
2. 2Send less context: retrieve the few relevant passages with RAG instead of whole documents, and trim old chat history.
3. 3Cap output: ask for short answers, set a maximum output length and use structured output instead of long prose where code reads the result.
4. 4Batch API: move work nobody is waiting on, such as tagging, summaries and nightly reports, to batch jobs, which providers usually price lower.
5. 5Model routing: send easy requests to a smaller model and escalate only the hard ones.
6. 6Cache whole responses for repeated questions, so identical requests do not hit the model at all.
7. 7Cut wasted calls: retries on errors, agents looping without a step limit, and duplicate calls from the front end.
8. 8Compare providers on your own token mix and check for off-peak or discounted pricing that suits your workload.

Protect quality while you cut

Every saving that changes the model, the prompt or the context can change answers. Keep a set of evals built from real cases and run it before and after each change. A cheaper setup that fails your evals is not cheaper once support tickets and rework are counted.