The business wants an AI agent that can issue refunds to customers automatically. How do you design it safely?
Short answer
Constrain the agent's capabilities: a narrow refund tool with server-side policy checks (amount limits, eligibility rules, fraud signals, per-customer caps), human approval above thresholds, delegated identity and full audit, guardrails against prompt injection, budgets on actions, evaluation on realistic scenarios, and a staged rollout with monitoring and a kill switch.
Detailed answer
- Tool design:
issue_refund(orderId, amount, reasonCode); the tool (not the model) validates order ownership, eligibility window, amount ≤ paid, policy limits, and fraud score. - Approval tiers: auto-approve small refunds within policy; route larger or unusual ones to a human queue with the agent's summary.
- Security: customer-scoped tokens; the agent cannot access other customers' orders; prompt injection defences on customer messages; no free-form database access.
- Limits: max refunds per customer per period, global daily budget, anomaly alerts.
- Audit: log conversation, tool calls, decisions, approvals.
- Evaluation: scenario suite (legitimate, borderline, adversarial), measure correct decisions and policy violations; shadow mode before enabling actions.
- Rollout: small percentage, monitoring refund rates and complaints, kill switch.
What the interviewer expects
Safety by architecture (capabilities and policy outside the model), human oversight proportional to risk, measurability.
Common mistakes to avoid
Relying on prompt instructions for policy enforcement; giving the agent broad API credentials.