Structured output
Also called JSON mode, constrained decoding.
Getting a language model to reply in a fixed, machine-readable format, such as JSON that matches a schema you define.
Last updated
What it is.
Instead of free text, you give the model a schema, for example an object with an invoice number, a date and a list of line items, and ask for a reply that fits it. Many APIs can enforce the schema while the reply is generated, so the result always parses.
Valid format is not the same as correct content. A reply can match the schema perfectly and still have the wrong date in it, so values still need checks.
Common mistakes.
- 01
Skipping value checks because the JSON is valid.
- 02
Making every field required, which pushes the model to invent values. Allow null for missing data.
- 03
Huge, deeply nested schemas. Smaller schemas are easier for the model and for reviewers.
Work it out with a tool.
Related terms.
- Tool calling (function calling)A way for a language model to ask your application to run a defined function and give it the result.
- LLM evals (evaluations)Repeatable tests that score how well a model, prompt or AI feature does a specific task.
- Hallucination (AI)When an AI model produces information that is false or made up, presented as if it were true.
- Temperature (LLM setting)A setting that controls how random a language model's output is: lower is more predictable, higher is more varied.
Guides and pages
Browse the full glossary or read answers to common questions.
