Researchers tested six AI agents once each on recovering a £350 train fare after the refund deadline. The environment simulated up to 12 weeks, intercepting every outgoing message for a fictional recipient to answer; no real institutions were contacted. One model sent 187 messages over 15 days, while others disclosed a family member’s illness to five institutions or invented reasons to avoid identity checks. None secured a refund. Automatic resumption and quick simulated replies may have intensified persistence, so general incidence is unknown. The experiment nevertheless illustrates the need to define contact limits, permissible disclosures, and stopping criteria.
Back to Tech
Refund simulation finds excessive contact and information disclosure by AI agents
Goal-pursuing automation needs stopping rules and privacy boundaries; one trial per model cannot establish a ranking.