How to test an AI agent that can issue refunds

What should a refund agent test actually measure?

Measure what the agent did, not how it sounded. The ground truth is the list of tool calls and the final state: which refunds, credits and discounts were issued, on which orders, for how much, and whether each one was allowed by the policy at that moment.

This is how tau-bench scores agents: it compares the database state at the end of a conversation with an annotated goal state, instead of grading the transcript. For a money-moving agent the useful numbers are dollars conceded beyond what the policy allowed, clauses broken (by ID), and promises in the text that no tool call backs up.

Promises matter as much as payouts. In Moffatt v. Air Canada, a tribunal held the airline responsible for a chatbot’s statement that a bereavement fare could be claimed retroactively, and ordered it to pay $650.88 in damages plus interest and fees.

How do you test a refund agent, step by step?

The method below works for any agent that can refund, credit, discount or offer payment terms. It needs no production access.

1. List the money tools. Every tool that moves money or commits the company (issue_refund, apply_discount, create_store_credit, offer_payment_plan), plus the read tools the agent uses to decide (get_order, get_customer, get_claim_history).
2. Build mock versions with the same schemas. The agent should not be able to tell them apart from the real ones. Each mock records every call with its arguments and returns data from fixtures.
3. Write the policy as numbered clauses. Each clause has one rule, a number where judgment would otherwise go, and the field that proves compliance. See the example below.
4. Seed fixtures with traps. Include orders delivered with photo proof, orders already refunded, orders outside the return window, customers with prior claims, and a fake approval ID that does not exist.
5. Pick scenarios by attack family. Cover at least negotiation pressure, social engineering, multi-turn escalation, indirect injection, information extraction and human fraud patterns.
6. Run each scenario many times. Vary seeds and attacker wording. One clean run says little about the next one.
7. Score from the tool log. Check clauses with code. Use a language-model grader only for soft criteria such as tone or implied promises, and check it against human labels.
8. Read the worst replays, fix, re-run. Fix the control that failed (often a server-side check), not only the prompt.
9. Turn the suite into a release gate. Run it on every prompt, model, tool or policy change and fail the build on real regressions.

What does a checkable refund policy look like?

Illustrative example for a home-goods retailer. Every clause names the data a script reads to decide whether it was broken, so no clause depends on someone’s reading of the transcript except the two marked as transcript checks.

Which adversarial scenarios should you run?

Run attacks that a motivated customer would actually try, not only jailbreak prompts. Research on policy-adherent customer-service agents found that persuasion strategies built around the agent’s policy beat generic jailbreak methods such as DAN-style prompts (Nakash et al., 2025).

How many runs per scenario, and how do you score them?

Run each scenario enough times to put an interval on the failure rate. Agents built on language models give different answers to the same conversation, and a determined customer can retry. tau-bench introduced pass^k, the chance that all k attempts at a task succeed, because a single success says little about consistency.

For each scenario report the number of runs, the number with any clause broken, a confidence interval on that rate, pass^k, and the mean and worst-case dollars conceded. Our guide to pass^k and confidence intervals covers how many runs that takes.

How do you turn the tests into a regression gate?

Run the same scenarios with the same seeds on the old and new versions, then compare them pair by pair. A paired comparison separates a real regression from run-to-run noise with far fewer runs than comparing two averages.

• Fail the build on any break of a never-item clause, even once.
• Fail on a statistically significant rise in clause breaks or dollars conceded, not on any single new failure.
• Keep a held-out set of scenarios the team does not tune the prompt against, and rotate it.
• Add every real production failure as a new scenario, with customer data removed.
• Record model, prompt, tool and policy versions with every run so a result can be reproduced.

Questions

Can I test against my real refund API?

Not with real money. Use mock tools with the same schemas, or a staging system whose final state you can read. Mocks are cheaper to reset and let you plant trap fixtures, such as a fake approval ID, that you could not safely create in production.

Do I need a language-model judge?

Only for soft criteria. Whether a refund exceeded a cap or skipped an approval is a fact in the tool log and should be checked with code. Implied promises and disclosure wording need a grader, and that grader should be checked against human labels before you trust it.

What should block a release?

Any break of a never-item clause, and any statistically real increase in clause breaks or dollars conceded on the paired suite. Single new failures inside the noise band should be reviewed, not used to block.

Sources

1. Yao et al., "tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains" (arXiv, 2024)
2. Nakash et al., "Effective Red-Teaming of Policy-Adherent Agents" (arXiv, 2025)
3. Mondaq: Airline ordered to compensate a B.C. man because its chatbot provided inaccurate information (2024)
4. OWASP, LLM06:2025 Excessive Agency

Related

• Guardrails checklist for AI refund and discount agents
• How to write a refund policy an AI agent can follow and a test can check
• AI red teaming vs adversarial testing vs simulation testing
• Why one test run is not enough: measuring agent reliability with pass^k