Back to the AI Product Lab
AI Product Lab, Customer AI Planned
Support Agent with Human Approval
An AI agent for order and refund requests that cites the policy it used and asks a person before risky actions.
- Setting
- A fictional European online shop, shared by all four lab builds
- Users
- Online shoppers, and support staff who approve risky actions
- Status
- Planned. Results will appear here once measured.
- Scope
- A small working build, a measured test and a short write-up
The problem
Support is overloaded with order-status and refund tickets. Leadership wants AI, but Legal worries about wrong refunds and data leaks.
What I'm building
Agent with tools
Looks up orders, searches the return policy and cites the clause it used, and issues small refunds.
Human approval for risk
Refunds above a set limit go to a person, with a full action log.
Guardrails
Personal data is masked, prompt attacks are tested and blocked, and the refund limit is enforced in code.
A/B test
A small test with student volunteers comparing the agent with a human-only flow.
How I'll measure it
—Tickets resolved, without a human
—Risky actions blocked, of attempted
—Policy answers, citing the correct clause
No numbers are shown until the tests have run. Code and results will be linked here.