AI agents for customer support · Part 13 of 14
Raising CSAT with an AI support agent
How to compare the agent’s ratings fairly with your team’s, which customers are asked, where ratings drop, and a weekly habit that turns low ratings into fixes.
The short version
An AI agent’s CSAT is easy to misread. It may take the easier tickets, and your settings decide which customers are asked. So compare it one ticket reason at a time, split by who answered, and know who gets the survey. Research shows what customers mind: their first worry is reaching a person, and angry customers rate a humanlike chatbot lower unless it resolves their problem. Then read the low ratings every week, and fix each one where its cause is. Four steps:
Step 1Compare one ticket reason at a time
When the agent takes the easier tickets and people keep the disputes, its overall score can beat your team’s while it trails on every kind of ticket. So compare one ticket reason at a time. Then split each reason three ways: the agent alone, the agent then a person, and a person alone. The middle group shows what your hand-overs cost the customer (part 9).
| Positive ratings | Agent alone | Person alone |
|---|---|---|
| All tickets | 85% | 77% |
| Delivery | 90% | 92% |
| Returns | 70% | 84% |
Most of the agent’s ratings were about delivery. People also took 150 disputes, rated 60% positive.
Outcomes differ this much by reason. In a survey of 497 customers, published in June 2023, Gartner found: “Just 17% of billing disputes are resolved by customers who used a chatbot at some stage in their journey”, against 58% of returns and cancellations.
Gartner, customer service survey, June 2023In Intercom, use the CX Score’s three groups as your split: Fin alone, a teammate alone, and both, sorted by who actually wrote in the conversation. In Ada, set one survey for the agent’s conversations and another for hand-overs, so the two scores stay apart.
Step 2Know who gets asked
CSAT counts only the customers who answered, and your settings decide who is asked. Intercom can show Fin’s survey when a customer replies with positive feedback, at a hand-over, or once the customer goes quiet, and works the score out “only from conversations that received a rating”. Ada asks when the customer closes the chat, or when a person leaves the conversation. Write down which triggers you have switched on, and count how many conversations each one reaches.
Intercom’s own help page lists “Surveyed CSAT is incomplete and biased” among the problems its CX Score addresses, and says customers who don’t respond, “including those who had a neutral or frustrating experience”, are often left out.
Intercom help, on the CX ScoreTo see past the survey, read a sample of the conversations nobody rated, or use a score that rates conversations without a survey, such as Intercom’s CX Score, which needs Fin and the Pro add-on.
Step 3Fix what customers mind most
Customers’ first worry about AI in service is reaching a person: in a Gartner survey of 5,728 customers, published in 2024, the top concern was that “it will get more difficult reaching a person”. A 2022 study shows where ratings fall: customers who arrive angry rate a humanlike chatbot lower, unless it resolves their problem. And Intercom’s CX Score counts customers repeating themselves as effort. Each has its fix in an earlier part of this series.
Reaching a person is hard
Customers’ top worry, Gartner, 2024
Change: pass the customer on the first time they ask. Part 9, step 1.
An angry customer meets a humanlike chatbot that can’t resolve the problem
Journal of Marketing, 2022
Change: hand angry customers to a person early, and give the agent the actions it needs, within a rule. Parts 9, 7 and 8.
The customer explains it all again
Intercom’s CX Score counts it as effort
Change: send the case along with the hand-over. Part 1, step 6.
A 2022 study found that “when customers enter a chatbot-led service interaction in an angry emotional state, chatbot anthropomorphism has a negative effect on customer satisfaction”, anthropomorphism meaning how humanlike the chatbot seems. The harm to how customers rated the company went away when the chatbot resolved the problem.
Crolic, Thomaz, Hadi and Stephen, “Blame the Bot”, Journal of Marketing, 2022Step 4Read the low ratings every week
Once a week, read every low-rated conversation the agent took part in, and give each one a cause: a missing answer, an action it couldn’t take, a slow or blind hand-over, a rule that said no, or its tone. Fix each where its cause lives, and count the same causes again the next week. A rule that said no may be right: check it is still the rule you want before you change anything.
Let your tool bring the low ratings to you. In Gorgias, install the rule “Re-open low CSAT AI Agent tickets”: it re-opens the closed tickets AI Agent handled when they get a low score, and adds an internal note, so your team has the context when they pick it up.
Next in this series
Part 14: keeping a record of what the agent did, and why. See all 14 parts.
Sources
- Gartner, “Gartner Survey Reveals Only 8% of Customers Used a Chatbot During their Most Recent Customer Service Interaction”, 15 June 2023, read on 21 October 2026.
- Gartner, “Gartner Survey Finds 64% of Customers Would Prefer That Companies Didn’t Use AI For Customer Service”, 9 July 2024, read on 21 October 2026.
- Crolic, Thomaz, Hadi and Stephen, “Blame the Bot: Anthropomorphism and Anger in Customer–Chatbot Interactions”, Journal of Marketing 86(1), 2022, abstract read on 21 October 2026.
- American Marketing Association, press release on “Blame the Bot”, read on 21 October 2026.
- Intercom help, “Analyze and report on Fin AI Agent CSAT”, read on 21 October 2026.
- Intercom help, “Surveyed CSAT reporting”, read on 21 October 2026.
- Intercom help, on the CX Score, read on 21 October 2026.
- Ada, “CSAT survey configuration”, read on 21 October 2026.
- Gorgias, “Use rules with AI Agent”, read on 21 October 2026.