Stripe's official mock server, stripe-mock, is stateless by design. Its README says so in one line: the data you send on a POST is validated and then completely ignored, so the customer an agent created a second ago does not exist a second later, and Stripe adds that it is not planning to add statefulness. Stripe test mode does keep state, but it is one account shared by everyone on the team: it cannot be reset between runs, it cannot be broken on purpose, its clock moves only when someone drives a test clock by hand, and it tells you nothing about what an agent changed beyond what the agent says. The Stripe world is a stateful Stripe mock built for testing agents: the same URLs, the same errors and the same state machine as the real API, on your machine, verified against real Stripe on a schedule.
A stateful Stripe mock, verified against real Stripe
The world keeps what an agent that touches money touches: customers, payment intents, charges, refunds, invoices and invoice items, subscriptions, products and prices, events, webhooks and idempotency keys. It keeps the rules too. An invoice has to be finalized before it can be paid. A refund cannot exceed the charge, and the second refund of an already refunded charge is refused in Stripe's exact words. A deleted customer answers 200 with a stub, not 404. A replay under the same idempotency key returns the first response, byte for byte. Every one of those behaviors is a probe: a short script that runs against a world and against real Stripe test mode through the same base URL swap, then compares what came back. The newest run is the fidelity page, dated, every probe listed, and the same battery runs twin against twin in every test run of the product.
Test your AI agent against Stripe
Your agent's stock Stripe SDK needs no change. A stripe-node or stripe-python client built with no host is pointed at the world by the environment the commands set: the world's key arrives as the secret key the SDK already reads, and a bundled SDK or another language reads the world's base URL and key from two named variables. The agent thinks it is talking to Stripe, because everything it can observe says so.
The path is three commands. The first asks for a restricted read only key, copies your own account onto your machine, pseudonymized and safe to commit, and writes the worlds folder: your first tasks, your rules for the grader, a test and a CI workflow. The second turns a help desk export into tasks, one per row, with an expected outcome on the rows where you know the right answer. The third runs the agent three times on fresh copies of the account and writes the report. The three steps take about ten minutes, and the INSTALL button on this page opens them.
AI agent refund testing, in dollars
The ticket that catches agents is not the hard one, it is the one with a twin. Two customers with the same name; one of them was charged twice; the ticket asks for one refund. An agent that refunds the wrong customer's charge writes a perfect reply, closes the ticket and reports success, and every transcript grader in the world agrees with it. The world does not. A run starts from a mark, the world's state at that moment, and ends with a diff: every refund, credit and cancellation since the mark, in dollars, with the customer whose money moved. The report says dollars wrong per run, gives every task a verdict and a reason, and replays the worst run to the byte. Over refunds are refused by the world the way Stripe refuses them, so the agent that tries one is told at once instead of being graded on a guess.
Break it on purpose
A scenario is a schedule of faults: rate limits, 500s on every third call, a slow API, an expired key, a webhook that never arrives or arrives twice, a lock timeout in the middle of a run. Does the retry logic double refund? Does the agent give up on a 429 and tell the customer it is done? Then move the clock. Advance forty five days and renewals bill, prorations settle, cancellations at period end resolve and the webhooks fire in order, so the subscription cases that take a quarter to reproduce in test mode take a second here. Then read the diff again.
Stripe test mode alternative, in CI
The same command that runs on your laptop is the CI step the first command wrote. There is no server to host: the world starts fresh inside the job, the agent runs on every task, the report is the artifact, and nothing leaves the machine. Runs are unlimited because they happen where nobody can count them. Every world in the catalog takes the same path; the Zendesk, Shopify and Salesforce worlds are three of them, and the docs are the path in full.