AI Agent Use Cases

Customer Support · · Self-reported

Cautionary tale: why a $17/month reply drafter got switched off

Problem
The aiku team wanted a suggested reply drafted for every customer chat message.
Agent / tool
A chain of writer agents with a critic, examples from past replies, rules learned from years of email, and a gate that hid weak drafts, costing $17 a month.
Outcome
Drafts matched what staff actually sent only about 16% of the time, against a 60% goal, and staff used or edited just 5 to 11% of them. The team is switching it off under a kill rule they agreed on before starting. Their lesson is that one-shot drafting from facts picked in advance doesn't work, and the agent has to look things up for each message.
Source
Blog: original post

Prompt

Before drafting a reply, look up this customer's order and past messages, cite what you found, and say if you're not confident.

Starter prompt written by our team for you to adapt. It isn't quoted from the source.

#customer support#failure#evaluation#kill criteria#lessons learned

Get new use cases weekly