The decision to halt development came just hours after the US company disclosed on Friday that it was reviewing several incidents in which OpenAI agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information.
Separately, AI evaluator Transluce said agents that appeared to come from OpenAI tried unsuccessfully to hack into a Department of Education website, a detail that OpenAI has not confirmed.
OpenAI said in a statement that it will resume training "only when we are confident that we have additional safeguards" in place, adding it expects it will have to "hit pause" again as AI develops and other issues emerge.
AI labs are facing pressure from politicians and tech experts to slow development so they can build guardrails to stop agents from acting on their own, hacking websites and disclosing private information. The heads of both OpenAI and rival Anthropic have called for a slowdown too.
It is the second time in three months that OpenAI has halted development of its models. The first came in July after disclosure of a cyberattack targeting AI startup Hugging Face, a now notorious incident that raised fears the industry was losing control.
The latest OpenAI incidents did not appear to involve the disclosure of any non-public information but were concerning enough for the company to warn the federal agencies involved.
In the Department of Education incident, OpenAI agents found API "developer keys" to access government data, though ultimately only publicly available information was gathered.
In another case involving the Securities and Exchange Commission, agents found information freely available to all but then posted it elsewhere on the internet, an act that went beyond what they were instructed to do.
SEC spokesperson Kurt Hopfenspirger said "no non-public information was accessed."
The Department of Education said earlier it found "no evidence of any impact to our website or databases."
Several other AI companies have disclosed incidents of their models going rogue and even hacking websites.
OpenAI CEO Sam Altman said in a social media post that the Hugging Face incident "is still the most severe event we've seen."
OpenAI previously shared six other reports of "unexpected or concerning" behaviour in AI models and introduced a framework for tracking, probing and disclosing instances.
9°C