Model Routing
Also called LLM router, Model router, AI gateway
Automatically sending each request to the AI model best suited to it, often a cheap, fast model for easy tasks and a stronger one for hard tasks.
What is model routing?
Model routing means choosing, for each request, which AI model should handle it. Instead of sending everything to one expensive model, a router sends simple requests, like short summaries or classification, to a small, fast model, and saves the most capable (and costly) model for the requests that need it.
How does it work?
Routers decide in different ways:
- Rules: by task type, user plan, input length or language.
- A classifier: a small model predicts how hard the request is and picks a model to match.
- Fallbacks: if one model or provider is down, slow or rate-limited, the request goes to another.
Routing can be built into your own code, offered by an AI gateway that sits between your app and several providers, or done inside a product. Some assistants automatically switch to a reasoning model when a question looks hard.
What is it commonly confused with?
- Mixture of experts. That routes tokens between parts of a single model. Model routing chooses between separate models.
- An agent orchestrator. Orchestration splits one task across several agents. Routing picks one model per request.
Why does it matter?
Most requests in a real product are easy, so routing can cut inference costs and latency a lot without users noticing a drop in quality. The risk is sending a hard question to a weak model, so test routing decisions on real examples and watch quality after launch. Routing across providers also reduces your dependence on any single one.
