Back to the AI Product Lab
AI Product Lab, AI platform Planned
Model Routing & AI Cost Optimizer
Routes each LLM request to the cheapest model that meets the quality bar, with spend by team and a savings case.
- Setting
- A fictional European online shop, shared by all four lab builds
- Users
- Internal teams using AI, the CFO and the head of platform
- Status
- Planned. Results will appear here once measured.
- Scope
- A small working build, a measured test and a short write-up
The problem
Every team sends every request to the most expensive model. AI spend grows faster than usage, and Finance can't see who spends what.
What I'm building
Real requests
100 sample requests from the shop's teams, sent to a cheap, a mid and a strong model.
Score each answer
Quality, cost and speed measured for every model.
Routing rule
Easy requests go to the cheap model, hard ones to the strong model, with a fallback if one fails.
Spend view
Spend by team and model, and savings against always using the strongest model.
How I'll measure it
—Cost saved, vs always using the strongest model
—Quality kept, vs the strongest model
—Speed, per route
No numbers are shown until the tests have run. Code and results will be linked here.