The support automation playbook: getting to 90% without wrecking CSAT

Support is the first place most e-commerce brands automate, and for good reason: volume is high, patterns are repetitive, cost scales linearly with growth, and the work is genuinely unpleasant for the people doing it.

It's also the place most likely to be done badly, because a bad support bot is worse than no support bot. Customers remember it.

This is the sequence we run. It's deliberately boring.

Step 0: Measure the queue before you touch it

You cannot automate what you haven't counted. Before writing a line of code, pull 90 days of tickets and classify them — a cheap model does this in an afternoon over a few thousand tickets.

For each category, get four numbers:

• Volume — how many per month
• Handle time — median minutes from open to resolve
• Resolution complexity — how many systems does an agent touch to resolve it?
• Judgement required — does resolution need a decision, or just a lookup?

Multiply volume by handle time to get the hours. That column, sorted descending, is your roadmap. It will not match anyone's intuition — teams consistently overestimate the cost of the tickets they hate and underestimate the cost of the boring ones they process quickly and constantly.

A typical distribution for a D2C brand:

Category	Share of volume	Automatable
Where is my order (WISMO)	30–40%	Almost fully
Returns & refunds	15–20%	Mostly, with policy encoded
Product questions (sizing, materials, compatibility)	10–15%	Mostly, with good retrieval
Order modification (address, size, cancel)	8–12%	Depends on fulfilment timing
Discount & payment issues	5–8%	Partly
Damaged / wrong item	5–8%	Partly — needs photo verification
Everything else	10–20%	Case by case

Step 1: WISMO first, always

Thirty to forty percent of volume, near-zero judgement required, and the answer lives in systems you already have.

The critical detail: do not just send a tracking link. That's what the automated email already did, and the customer is writing to you because the link didn't answer their question. It says "in transit" and they want to know if it'll arrive before the weekend.

A good WISMO agent:

1. Identifies the order from the message — number, email, or by matching context
2. Pulls the live carrier status, not the cached one
3. Interprets it against typical transit times for that lane
4. Gives a specific answer: "It cleared the Rotterdam hub yesterday and is on track for Thursday"
5. Detects the exception cases — stalled scans, customs holds, delivery attempts failed — and takes the right action for each rather than reading the status aloud

Point 5 is where value concentrates. A parcel that hasn't scanned in six days needs a claim opened, not a status update.

Realistic outcome: WISMO ticket volume drops by 80–90%, and the residue is genuine exceptions.

Step 2: Returns and refunds, with the policy in code

Higher stakes, and the discipline is different: the policy goes in code, not in the prompt.

Write a function that takes an order and returns an eligibility decision — window, item condition, category exclusions, market-specific rules, whether the item was on final sale. The model calls that function and communicates the result. It does not read a policy document and decide.

This matters because return policies are full of exceptions that are trivially expressible in code and genuinely ambiguous in prose. "Sale items are final sale except during the January window, and personalised items are always excluded except for manufacturing defects" is three lines of logic and a paragraph that no two people will read the same way.

Set a value threshold above which a human reviews. Start it low — say €150 — and raise it as the agent's accuracy record accumulates. Your finance team will thank you for the audit log.

Step 3: Product questions, which is really a retrieval problem

Sizing, materials, care instructions, compatibility, allergens. Success here is entirely determined by whether the information exists in a retrievable form.

It usually doesn't. It lives in the PDP as marketing copy, in a spec sheet nobody indexed, in the head of the merchandiser, and in 4,000 past ticket replies.

The fix is a content project, not an AI project:

• Extract structured attributes from spec sheets into a queryable form
• Mine past ticket resolutions for genuine answers — this is your densest source and it's already written in your brand voice
• Index reviews for real-world fit and quality feedback ("runs small" is answered in reviews, not the PDP)
• Write down what only exists in someone's head

Do this once, properly, and it pays into support, PDP content, ads and SEO simultaneously.

Step 4: The escalation layer, built in parallel

Not last. Build it alongside step one, because it's what makes the rest safe.

Escalate on category. Legal threats, safety issues, bereavement, chargeback disputes, press enquiries, anyone who's asked for a human twice. These leave automation regardless of whether the agent could technically answer.

Escalate on sentiment shift. Watch the trajectory within a thread, not just the opening message. A conversation that starts neutral and turns is the one you must catch. This runs as a separate classifier with its own eval set — don't fold it into the main agent's judgement.

Escalate on confidence. No retrieval above the relevance floor, or a decision outside the coded boundaries, means handoff.

Make handoff genuinely good. The human should receive: a written summary of what the customer wants, the retrieved context, the actions already taken, and the specific reason for escalation. If handoff is excellent, you can set thresholds conservatively without your team resenting it — and conservative thresholds are what protect CSAT.

The metrics that matter

Deflection rate is the number everyone reports and the easiest to game — an agent that answers unhelpfully and closes the ticket deflects beautifully. Track these instead:

True resolution rate. Percentage of automated conversations with no reopen within seven days and no follow-up on the same issue. This is the honest version of deflection.

CSAT, segmented. Automated vs. human-handled vs. escalated. Watch the escalated segment especially — if escalated conversations score lower than pure-human ones, your handoff is dropping context.

Time to resolution, not time to first response. Automation makes first response instant by definition. That's not the win. The win is the customer's problem being solved faster.

Escalation precision and recall. Of the tickets escalated, how many genuinely needed a human? Of those that stayed automated, how many should have escalated? Both directions matter, and the second is the dangerous one.

Cost per resolved ticket. Fully loaded: model spend, infrastructure, plus the human minutes still involved. This is the number that goes in the board deck.

What to expect, honestly

For a typical D2C brand with a clean product catalogue and reasonable systems:

• Month 1: WISMO live. 25–35% of total volume automated.
• Month 2: Returns and refunds. Cumulative 45–60%.
• Month 3–4: Product questions, once the content work lands. Cumulative 70–80%.
• Month 6+: Long tail, edge cases, proactive outreach. 85–92% is where well-run deployments plateau.

The last 8–15% is not a technology gap. It's genuinely novel situations, genuine judgement calls, and customers who want a person — and it should stay that way. An operation targeting 100% is optimising the wrong number.

The mistake nearly everyone makes

Starting with the hardest tickets.

There's an intuitive pull toward automating the cases the team hates most — the complex, contentious, multi-system ones. They're the most memorable and the most emotionally expensive.

They're also the lowest volume, the highest risk, and the worst possible place to build your first agent. Start with the boring high-volume category, get the infrastructure right, build the eval set, accumulate the accuracy record, earn the team's trust. Then move up.

The team that hates the hard tickets will be much happier with the hard tickets once they have time to actually do them properly — which is exactly what automating the boring ones buys.

---

We ship a first support agent in 14 days from signed spec, with the eval set and the escalation policy handed over. Book an audit call.