Google's Gmail API documentation describes usage limits, not a test mode: calls are metered in quota units per minute, per user and per project, with a daily threshold for the project (Gmail API usage limits). The Workspace sending limits stop a user who reaches them from sending for up to 24 hours, and the API quickstart's first prerequisite is a Google account with Gmail enabled. So an agent that reads, labels, drafts and sends mail is tested against a real mailbox, with real recipients on the other end and real quota to burn, or it is not tested at all. The Gmail world is a stateful Gmail mock built for testing agents: the same URLs, the same errors and the same state machine as the real API, on your machine, verified against the real system.
Gmail API test environment
A mailbox cannot be reset, and a send cannot be taken back. That is what makes an agent on email hard to test: the cases that matter are the ones where sending is the mistake, the reply to the whole thread that should have gone to one person, the draft that went out unfinished, the third follow up to someone who answered the first. Nobody wants to rehearse those on a real account. A world is a pseudonymized copy of a mailbox, taken when you ask for it: the same threads, the same labels, the same drafts and the same contacts, under new names and new addresses that reach nobody. Every run starts from those bytes, in milliseconds, and ends by throwing the world away.
A stateful Gmail mock: threads, labels, drafts, send limits
The world keeps messages and the threads they belong to, the system labels and the user's own, drafts, and the sending limits an account is held to. It keeps Gmail's rules with them. A send creates a message in its thread and moves the draft out of drafts. A message removed from the inbox is still in the thread. A message in the trash is gone after its time. A reply carries the thread, and a reply to all carries every recipient. A send past the limit is refused the way Gmail refuses it. Everything the agent creates or changes stays in the world, so the second request sees what the first one did.
Test your AI agent against Gmail
Point the agent's Gmail client at the world. Nothing in the agent changes. The path is three commands: the first copies the mailbox and writes the worlds folder, the second turns your own queue export into tasks, one per row, with the expected outcome on the rows where you know it, and the third runs the agent three times on fresh copies and writes the report. The three steps take about ten minutes; the GET ACCESS form on this page is where the Gmail world starts.
Grade what the agent did, not what it said
A run starts from a mark and ends with a diff: every message sent, to whom, in which thread, with which body; every draft created or discarded; every label applied or removed; every message archived or trashed. The transcript says the customer was answered; the diff shows a draft that was never sent. The transcript says the invoice went to the customer; the diff shows it went to the whole thread, the customer's accountant included. The transcript says the thread was archived; the diff shows it starred instead. Where a message promises money, the payments world beside this one shows whether any moved, in dollars. The report gives every task a verdict and a reason, and replays the worst run to the byte.
Break it on purpose
Quota runs out, sends bounce and the API answers slowly, and an agent working an inbox will meet all three at the worst moment. In the world that moment is on your schedule: a quota refusal on every third call, a send limit reached mid run, an outage, a slow API, an expired token. Does the retry logic send the same reply twice? Does the agent tell the customer it wrote back when the send was refused? Then move the clock, so follow ups come due and trashed messages expire, and read the diff again.
Gmail test environment for agents, in CI
The same command that runs on a laptop is the CI step the first command wrote. No real mailbox, no real recipients, no quota to spend: the world starts fresh inside the job, the agent runs every task, the report is the artifact, and nothing leaves the machine. Every world in the catalog takes the same path; Twilio, Intercom and Stripe are three more of them, and the docs are the path in full.