Back to the AI Product Lab
AI Product Lab, Data platform Planned
Self-Serve Analytics Assistant
Text-to-SQL over the shop's data, using agreed metric definitions, showing the query it ran, and tested for accuracy.
- Setting
- A fictional European online shop, shared by all four lab builds
- Users
- Business teams who need numbers without waiting for an analyst
- Status
- Planned. Results will appear here once measured.
- Scope
- A small working build, a measured test and a short write-up
The problem
Business teams wait days for simple numbers, and the same metric is calculated differently by different teams.
What I'm building
Sample data
A small orders database for the shop.
Agreed metric definitions
One definition for revenue, return rate and other key metrics, which the assistant must use.
Ask in plain language
The assistant writes the query, runs it, and shows the answer with the query it used. It says so when it can't answer.
Accuracy test
30 business questions with known answers, plus a review of every failure. Personal data columns are blocked.
How I'll measure it
—Answer accuracy, on 30 test questions
—Correct refusals, vs confident wrong answers
—Cost, per question
No numbers are shown until the tests have run. Code and results will be linked here.