Token (LLM)
The unit an AI language model reads and writes, and the unit most LLM APIs bill by.
Last updated
What it is.
A language model does not read words or letters. A tokenizer first splits text into tokens, which are often parts of words, whole short words, punctuation or spaces. Images and audio are also turned into tokens when a model accepts them.
Each model family has its own tokenizer, so the same text can be a different number of tokens on different models. APIs usually price input tokens and output tokens separately, per million tokens, and output is often the dearer of the two.
Why it matters
Tokens are the meter on every LLM bill and the unit of the context window. Knowing roughly how many tokens a feature uses per call, split into input, cached input and output, is the starting point for any cost estimate.
Common mistakes.
- 01
Estimating cost from word counts. Count tokens with the model's own tokenizer.
- 02
Ignoring the token mix. Two features with the same total can cost very different amounts if one is output-heavy.
- 03
Forgetting hidden tokens: system prompts, tool definitions, retrieved documents and reasoning output all count.
Work it out with a tool.
Related terms.
- Context windowThe most tokens a model can take in and produce in one request.
- Prompt cachingReusing the processed start of a prompt across calls, so repeated input is billed at a lower rate.
- Batch APIA way to send many LLM requests to run later, usually at a discount.
- Inference (AI)Running a trained AI model on new input to get an output, such as an answer, a label or an embedding.
Guides and pages
Browse the full glossary or read answers to common questions.
