A NetSuite sandbox is a separately purchased account, arranged through your account manager (Requesting a sandbox account). Oracle's documentation describes what a refresh gives you: a snapshot of the source account taken at the time of the request, which, once activated, completely replaces the previous sandbox; any data in the sandbox before the refresh is overwritten and cannot be restored to its prior state. So an agent tested in a sandbox works on a copy that is hours or months old, shares it with every developer who has one, and loses the fixtures built into it at the following refresh. The NetSuite world is a stateful NetSuite mock built for testing agents: the same URLs, the same errors and the same state machine as the real API, on your machine, verified against the real system.
NetSuite sandbox refresh
A refresh is requested, not instant, and it takes hours. That makes it a reset button for a quarter, not for a test run. In between, every purchase order an agent created and every journal it posted is still there for the run after it, and a suite that needs a clean ledger gets one when the refresh comes through. A world is a pseudonymized copy of your own account, taken when you ask for it: the same vendors and subsidiaries, the same open orders and unposted journals, the same periods in the same states, under new names. Every run starts from those bytes, in milliseconds, and ends by throwing the world away.
A stateful NetSuite mock: journals, purchase orders, approvals, period closes
The world keeps vendors and customers, purchase orders and vendor bills with their approval status, journal entries with their lines, accounts, subsidiaries and the accounting periods with their open and closed states. It keeps NetSuite's rules with them. A purchase order needs its approval before it can be received against. A journal entry has to balance, and one that does not is refused with the real message. A posting into a closed period is refused. An amendment to an approved order changes its approval. Everything the agent creates or changes stays in the world, so the second request sees what the first one did.
Test your AI agent against NetSuite
Point the agent's NetSuite client at the world. Nothing in the agent changes. The path is three commands: the first copies the account and writes the worlds folder, the second turns your own queue export into tasks, one per row, with the expected outcome on the rows where you know it, and the third runs the agent three times on fresh copies and writes the report. The three steps take about ten minutes; the GET ACCESS form on this page is where the NetSuite world starts.
Grade what the agent did, not what it said
A run starts from a mark and ends with a diff: every journal posted and into which period, every purchase order created, amended or approved, every bill entered and against which order, in dollars. The transcript says the accrual was reversed; the diff shows a second accrual instead. The transcript says the order was approved; the diff shows the status unchanged and a memo added. The transcript says the bill matched the order; the diff shows the amount on the bill and the amount on the order, and they differ. The report says dollars wrong per run, gives every task a verdict and a reason, and replays the worst run to the byte.
Break it on purpose
NetSuite governs concurrency and meters requests, and an agent working a queue will meet the limit at the worst moment. In the world that moment is on your schedule: a concurrency refusal on every third call, an outage in the middle of a batch, a slow API, an expired token. Does the retry logic post the journal twice? Does the agent report the order approved when the call never landed? Then move the clock, so the period closes under the agent and the posting it tries is refused the way NetSuite would refuse it.
NetSuite test environment for agents, in CI
The same command that runs on a laptop is the CI step the first command wrote. No sandbox to purchase or refresh, no snapshot to wait for: the world starts fresh inside the job, the agent runs every task, the report is the artifact, and nothing leaves the machine. Every world in the catalog takes the same path; QuickBooks, Salesforce and Stripe are three more of them, and the docs are the path in full.