{"head":{"title":"What Is Model Routing for LLMs? Definition | OceanLab","description":"Model routing sends each AI request to the model best suited to it, often a cheaper model first. How routing works, an example and the mistakes teams make.","canonical":"https://oceanlab.in/glossary/model-routing"},"crumbs":[{"name":"Home","path":"/"},{"name":"Glossary","path":"/glossary"},{"name":"model routing","path":"/glossary/model-routing"}],"slug":"model-routing","term":"Model routing","aka":["LLM routing","model cascade"],"category":"LLM costs","short":"Sending each AI request to the model best suited to it, instead of using one model for everything.","body":[["A router looks at a request and picks a model. The rule can be simple, such as task type or user plan, or it can be a small classifier. A common pattern is a cascade: try a smaller, cheaper model first and escalate to a larger one only when the answer fails a check."],["Routing can also be about providers, not just model size: sending traffic to a backup when one provider is down, or to a provider in a specific region for data rules."]],"why":["Many requests in a product are easy and do not need the most capable model. Routing them to a smaller model lowers ",{"href":"/glossary/inference","text":"inference"}," cost and often latency, while hard requests still get the strong model. The key is having ",{"href":"/glossary/llm-evals","text":"evals"}," that show the cheaper path is good enough."],"example":["A support bot answers order-status questions from a template and passes billing disputes to a stronger model with access to account tools. A check on each cheap answer sends it to the stronger model if it does not cite an order record."],"inBuild":null,"mistakes":[["Routing without measuring quality on each path. Savings that lower answer quality are not savings."],["Building a router more complex than the problem. A few clear rules often beat a learned router."],["Forgetting that each model needs its own prompt tuning and ",{"href":"/glossary/structured-output","text":"structured output"}," checks."]],"related":[{"href":"/glossary/inference","label":"Inference (AI)","detail":"Running a trained AI model on new input to get an output, such as an answer, a label or an embedding."},{"href":"/glossary/llm-evals","label":"LLM evals (evaluations)","detail":"Repeatable tests that score how well a model, prompt or AI feature does a specific task."},{"href":"/glossary/token","label":"Token (LLM)","detail":"The unit an AI language model reads and writes, and the unit most LLM APIs bill by."},{"href":"/glossary/llm","label":"LLM (large language model)","detail":"An AI model trained on very large amounts of text that predicts the next piece of text, which lets it read, write and reason in language."}],"tools":[{"href":"/tools/llm-model-picker","label":"LLM model picker","detail":"Free tool"},{"href":"/tools/llm-api-cost-calculator","label":"LLM API cost calculator","detail":"Free tool"}],"links":[{"href":"/pricing/categories/llm-api","label":"LLM API pricing","detail":"Model prices side by side, with sources."},{"href":"/ai","label":"AI by industry","detail":"AI use cases, costs and risks per industry."}],"sources":[],"legal":false,"updatedAt":"2026-10-25"}