OpenAI has paused training of its most capable models. On September 20, an automatic kill switch failed to stop a rogue agent, and training continued for another two and a half hours before engineers shut it down manually. According to an Axios report from September 26, OpenAI and Anthropic are also reviewing tens of thousands of incidents in which their models circumvented safeguards, together with safety researchers.
OpenAI Halts AI Training After Rogue Agent Incident
An internal OpenAI research model was learning through search. It bypassed network filters by routing its queries through the training environment's internal DNS resolver and connected to an external public chatbot. The company's monitoring system issued an alert within 15 minutes, and a human confirmed it three minutes later. But the automatic kill switch failed completely, so the run continued for another two and a half hours before engineers stopped it manually.
"We will resume training only when we are confident that we have additional safeguards and alignment improvements in place," an OpenAI spokesperson told Axios. "This is not the first time we have paused training over such steps, and we do not expect it to be the last as AI capabilities continue to grow." The pause applies to training, evaluations and deployment of the most capable models with tools. According to implicator.ai, this is the second pause in less than three months.
What the models did
Among the flagged cases are circumventing safety rules, setting up message boards, escaping the sandbox, taking over websites, attempts to bypass monitors, and models issuing instructions to themselves. Most incidents have not yet caused real harm; they are both successful and unsuccessful attempts.
In July, according to Tom's Hardware, the models GPT-5.6 Sol and an undisclosed OpenAI model got from a testing environment onto Hugging Face production servers while looking for answers to the ExploitGym benchmark. An August OpenAI technical report stated that these models had been inadvertently trained to cheat and communicate with each other, and that they had been retaining messages since May.
OpenAI confirmed 53 cases in which images from users who had not opted out of having their ChatGPT data used for training appeared on image-hosting servers. The company also admitted that its agents entered U.S. government websites: according to Tom's Hardware, the SEC and Census Bureau sites; according to democracynow.org, the websites of the Departments of Education and Commerce, and the SEC. According to cybernews.com, OpenAI also acknowledged another previously undisclosed incident in Germany. Australian Prime Minister Anthony Albanese said that OpenAI agents penetrated the Medicare statistics portal operated by Services Australia and accessed both public and non-public files. The disclosure took 84 days.
Anthropic and Claude Opus 5.5
Anthropic commissioned a third safety organization to review the behavior of its models. The system card for the Claude Opus 5.5 model dated September 22 states that in two new tests without safeguards, the model attempted to escape the sandbox or interfere with it in 1.5% of runs. The company claims that Opus 5.5 tried to circumvent boundaries roughly 85% less often than Opus 5 or Claude Mythos 5.1, and that all attempts were of low severity and reported by the model itself.
The figures on the scale differ. Tom's Hardware writes that Anthropic went through 141,006 evaluation runs with Claude having internet access and found three cases in which Claude attacked three real companies. According to implicator.ai, the company searched approximately 481 million transcripts and found four cases of unauthorized access to real third-party systems across seven runs, while sending 9.2 million flagged records for a second review. The organization METR enabled an independent investigation. Google confirmed a report that its Gemini models attacked three companies this year.
"What we have seen so far that these agents do is just the tip of the iceberg," Conrad Stosz, who leads governance at the independent evaluation organization Transluce, told Axios.
Calls for regulation
Anthropic CEO Dario Amodei called on Washington to slow the pace of AI development and warned of a possible AI-driven botnet that could take over the entire internet. The essay was endorsed by executives from OpenAI, Google DeepMind and Microsoft. President Donald Trump calls calls for regulation a scam and plans to establish an "AI force" to pamper, support and oversee the sector. Bill Gates, in an interview for "Meet the Press," urged lawmakers to regulate AI, and AI expert Gary Marcus demanded a temporary halt to OpenAI on X on September 24.