Cost
Models are priced differently. Sending every request to the most capable model spends that rate on tasks a smaller model may already handle.
Map · Foundations · 05 · Model Selection & Routing
Concept card · Foundations
Model Selection & Routing
Model selection is choosing which model handles a request, and routing is sending different requests to different models.
Picking a model, and sometimes a route between models, for the job in front of you.
What changed
Oct 17: First card. The picture routes three kinds of task. Sources are the OpenAI model selection guide, the OpenAI deployment checklist, and Anthropic's guide to choosing a model.
1 What it is
The API call names a model. Selection is that choice for one job. Routing is doing it on purpose across jobs: a short labeling task can go one way, and a long draft another.
2 Why it matters
Quality
A faster model can be enough for a narrow task and not enough for a hard one. The way to tell is to run the same examples on each.
Risk
A different model can change the reply, which tools the request can use, and which settings the API accepts.
3 How to apply · try today
Pick one real task. Run it on two models you can already call, with the same prompt, and keep the reply that does the job.
Common mistake
Routing every request to the most capable model, including the ones a smaller model already handles.
Self-check
You have a short labeling task and a long drafting task. What is routing?
Coming soon. One task you already run, compared on two models you can call.
Coming soon. What changes besides the reply when the model name changes, including tools and settings.
Questions
- What is model selection?
- Model selection is choosing which model handles a request.
- What is routing?
- Routing is sending different requests to different models, based on the job.
- What is a common mistake?
- Sending every request to the most capable model, including tasks a smaller model already handles.