Temperature (LLM setting)
Also called sampling temperature.
A setting that controls how random a language model's output is: lower is more predictable, higher is more varied.
Last updated
What it is.
At each step a model has a ranked list of possible next tokens. Temperature reshapes that list before one is picked. Low values make the top choice win almost every time, so answers are consistent. Higher values give less likely tokens more of a chance, so answers vary more.
It is one of several sampling settings, alongside top-p and top-k. Some models, especially reasoning models, fix or ignore these settings, so check the provider's docs.
Why it matters
Extraction, classification and code usually want consistency, so they run at low temperature. Brainstorming and creative writing can benefit from more variety. Even at the lowest setting, outputs are not always identical, so tests and checks are still needed.
Common mistakes.
- 01
Raising temperature to fix wrong answers. It makes them less predictable, not more correct.
- 02
Assuming low temperature means fully repeatable output.
- 03
Tuning temperature before fixing the prompt and the data the model is given.
Work it out with a tool.
Related terms.
- LLM (large language model)An AI model trained on very large amounts of text that predicts the next piece of text, which lets it read, write and reason in language.
- Hallucination (AI)When an AI model produces information that is false or made up, presented as if it were true.
- Structured outputGetting a language model to reply in a fixed, machine-readable format, such as JSON that matches a schema you define.
- Token (LLM)The unit an AI language model reads and writes, and the unit most LLM APIs bill by.
Guides and pages
Browse the full glossary or read answers to common questions.
