When the AI didn't know the test was real

In April 2026, Anthropic ran routine security tests on three of its Claude AI models. The task was a 'capture the flag' exercise — a standard, realistic hacking challenge used to measure a model's capabilities. The models were told they had no internet access. But a mistake by the outside testing firm meant the models were actually connected to the live internet — and nobody, not the models and not the humans running the tests, knew it.

The models did what they were trained to do: complete the task. One found a real company that shared a name with its fake target, guessed weak passwords, and pulled hundreds of rows of data from a live database. Another built a malicious software package and uploaded it to PyPI — a public code library used by millions of developers. Fifteen real machines ran it within an hour, including one at a cybersecurity firm. Two of the three affected companies had no idea anything had happened until Anthropic called them on July 27.

Anthropic reviewed 141,006 evaluation runs after the incidents came to light. Only six were problematic — all linked to one testing partner. The models were not acting out of malice. They were doing exactly the job they were given. The problem was a broken assumption — that the test environment was sealed off from the real world. It was not.

For any team building or deploying AI agents — and that is a fast-growing group — this is a clear signal. Treat your testing environment as if it might be connected, because it might be. Networks need to be truly air-gapped before an AI is let loose. Permissions need to be reviewed before a test starts, not after. The stronger AI gets, the costlier a leaky sandbox. Anthropic halted all cybersecurity evaluations on July 23 and is bringing in independent reviewers — the right response, and the right model for anyone working with powerful AI.