AI agents for customer support · Part 2 of 14
What an AI support agent can and can’t do today
The work it handles well, the work to keep with people, and four ways it goes wrong.
The short version
An AI support agent is good at work that is written down: answers, lookups, and actions within a rule. It is poor at judgement, at upset customers, and at anything nobody wrote down. Plan for four ways it goes wrong:
What it does well, and what to keep with people
Sort the work by how much judgement it takes. The agent is strong on the left and weak on the right, and the line moves only as far as you write things down.
The agent’s work
A written answer
“What’s your return window?”
A lookup
“Where is my order?”
An action within a rule
“Refund €25 for a damaged item.”
Your team’s work
A judgement
“Is this case an exception?”
Feelings
“I want to talk to a person.”
As you write answers and rules, the line moves down this list.
Keep these with people for now: judgement calls, upset or vulnerable customers, disputes and anything with a legal side, and anything your knowledge doesn’t cover.
Four ways it goes wrong
Each one has a cause you can see before launch, and a check that catches it.
1
It invents an answer
How it happensYour knowledge has a gap, and the model fills it with something that sounds right.
What catches itAnswering only from your knowledge, and handing over when the answer isn’t there.
New York City’s business chatbot told owners they could take workers’ tips (2024).
2
It promises what you don’t offer
How it happensIt states a policy, a price or an offer in its own words.
What catches itPolicy answers that quote their source, and offers checked like any other action.
Part 1 has the tribunal that held an airline to its chatbot’s answer.
3
It loops
How it happensIt keeps asking or rephrasing, and the conversation goes nowhere.
What catches itA limit on turns without progress, then a hand-over. Watch how often the same customer comes back.
4
It’s talked into things
How it happensA customer tells it what to say, or claims an approval it can’t see.
What catches itChecking anything that moves money against rules outside the chat, where nothing said in the conversation can change a limit.
A car dealer’s chatbot was talked into agreeing to sell a new SUV for $1. No sale happened (2023).
Is a ticket reason ready for the agent?
Run each reason through four questions before you hand it over.
1 Is the answer written down?
If not: write it down first (part 4).
2 Is it the same answer every time?
If not: keep it with people, or write the rule (part 8).
3 Can the agent see the facts it needs?
If not: connect the system first (part 6).
4 If it moves money, is there a rule with a limit?
If not: write the rule first (part 8).
Ready for the agent
Start in suggest mode (part 11).
Next in this series
Part 3: build or buy, and what each option costs you. See all 14 parts.
Where Himmel fits
Himmel is the check in the fourth failure: whatever the customer says in the chat, an action that moves money is checked against your rules before it happens.
Sources
- The Markup, “NYC’s AI chatbot tells businesses to break the law”, 29 March 2024.
- The $1 SUV, December 2023: AI Incident Database; Gigazine.