How do you measure the business value of a GenAI customer support assistant?

Short answer

Define outcome metrics before launch (containment/deflection rate, resolution rate, average handle time for agents, customer satisfaction, escalation quality), measure them against a control group or baseline, include full costs (models, platform, content maintenance, oversight), and monitor quality and safety incidents alongside value.

Detailed answer

Metric Description
Containment rate Conversations resolved without human agent
Resolution accuracy Correct answers (sampled review + evals)
CSAT / NPS Compared with human-handled baseline
Agent handle time For agent-assist variants
Cost per resolved contact Total cost / resolved contacts
Escalation quality Context passed to humans

Use A/B tests or phased rollout by segment to attribute impact; report value net of costs and risks.

What the interviewer expects

Outcome orientation, rigorous attribution, full cost accounting.

Common mistakes to avoid

Reporting usage (messages sent) as value.