Skip to content
← Back to workCase study 01 / 04 · Protected

AI Agent Training: when should the agent act on its own?

Year
2025
Company
Meta
Role
Lead product designer
Key details
Human oversight, AI systems, Agent training, Craft, Model behavior
Snapshot

I set out to simplify the interface. The interface wasn’t the problem.

4x
More saved corrections
6 weeks
To rethink and rebuild the experience
3
Spun out projects

Businesses could teach their AI agent when it got something wrong.

The problem was that most never made it to the end. They’d start a correction and drop off before saving, so the changes they wanted to make never reached their agent.

I started by redesigning how businesses corrected their agent. I built a working prototype connected to a live LLM and tested it with real businesses. It showed that a better interface wasn’t enough.

So I went beneath the UI and rethought the agent’s behavior: what it could handle on its own, and when it needed a person involved.

Businesses finally finished what they started. They could correct their agent and clearly see what it had learned, with saved corrections increasing 4x.

The work also uncovered three new opportunities that moved onto the roadmap.

Joined live customer calls to understand where businesses were getting stuck
Redesigned the correction experience around what I learned
Built and tested a working prototype connected to a live LLM
Dug beneath the UI to understand what was making the experience so complex
Worked with research science, PM, and engineering to rethink the agent’s behavior
- Designed the shipped experience and how the agent communicates what it learned
Protected case study

Enter a passcode to keep reading.