Context window
Also called context length.
The most tokens a model can take in and produce in one request.
Last updated
What it is.
The context window covers everything the model sees in one call: the system prompt, tool definitions, any documents or chat history you send, and the reply it writes. It is measured in tokens. If a request does not fit, it fails or has to be cut down.
A bigger window allows longer inputs, such as whole contracts or long conversations. It does not make the model remember anything between calls; each request starts fresh with whatever you send.
Why it matters
Every input token is billed, so filling a large window on every call is expensive, and some providers charge a higher rate above a length threshold. Very long inputs can also make answers worse, because the important part is buried. Sending the right few passages, as in RAG, is often cheaper and better than sending everything.
Common mistakes.
- 01
Treating the window as memory. The app has to send the history again on every call.
- 02
Sending everything because it fits. Fitting is not the same as being useful or cheap.
- 03
Leaving no room for the reply. Output tokens share the same window.
Work it out with a tool.
Related terms.
- Token (LLM)The unit an AI language model reads and writes, and the unit most LLM APIs bill by.
- Prompt cachingReusing the processed start of a prompt across calls, so repeated input is billed at a lower rate.
- RAG (retrieval-augmented generation)A pattern that finds relevant passages from your own data and gives them to a language model along with the question, so it answers from your content.
- LLM (large language model)An AI model trained on very large amounts of text that predicts the next piece of text, which lets it read, write and reason in language.
Guides and pages
Browse the full glossary or read answers to common questions.
