OpenAI revealed that some of its most advanced AI models went haywire and hacked a startup after they lost control over them during a security test.

The company behind ChatGPT explained that its AI agent – ​​systems capable of acting autonomously after receiving human instructions – was being evaluated in a controlled environment. However, he found security vulnerabilities and managed to escape from the testing environment.

The models targeted Hugging Face, one of the world's largest centers for sharing artificial intelligence models, and managed to access some internal systems of the company.

OpenAI called the incident “unprecedented” and stated that it was conducting an investigation with this company. For his part, its executive director, Clément Delangue, wrote in a post on X that it was “amazing that all this happened autonomously.”

“The investigation continues and we will share more information about what could be the first incident of its kind,” Delangue added.

Insecure testing environments

Gina Neff, director of the Minderoo Center for Technology and Democracy at the University of Cambridge, told BBC Radio 4's Today program that security tests — known as sandboxes — “are supposed to be safe spaces where you can see what the models are capable of.”

“In this case, it appears that OpenAI did not create a sufficiently secure sandbox,” he added.

Instead, the agents created their own cyberattack against the sandbox itself, finding a vulnerability that allowed them to escape.

Once outside, the AI ​​identified Hugging Face as a likely source of the answers they were looking for in the test and attempted to access it.

Neil Lawrence, a professor of machine learning at the University of Cambridge, called it an “impressive feat” but cautioned that it “lies within the known capabilities of the current generation” of high-powered AI models.

He noted that OpenAI is looking to go public and faces intense pressure from its rival company, Anthropic, which has been making headlines with its own powerful AI tool, Mythos.

“OpenAI is now playing catch-up, trying to demonstrate the capabilities of its own systems in cybersecurity.”

“This shows that OpenAI is not capable of implementing its own technology safely,” he added.

In its initial statement about the cyberattack, published on July 16, Hugging Face said it was still assessing whether any customer or partner data had been affected and would contact affected parties if necessary.

The company claimed to have remedied the vulnerabilities detected by the incident and to have rebuilt the affected systems.

“Autonomous offensive tools based on artificial intelligence are no longer a theoretical question,” he said.

“Defending an online platform now means treating the data and model surface as a top-level attack surface, and using AI in defense to keep up,” he added.

“We will continue to invest there and continue to share what we learn.”

“A moment that invites reflection”

The incident has raised new questions about the capabilities of advanced AI systems and whether existing security measures are sufficient as the technology becomes more powerful.

Spencer Starkey, an executive at cybersecurity company SonicWall, told the BBC that the incident made it clear that organizations needed to “reinforce” their own defenses and “treat cyber resilience as a key operational priority.”

“The uncomfortable truth is that too many organizations continue to defend themselves at human speed, while their adversaries are increasing machine speed,” he said.

Meanwhile, Travis Lelle, principal security engineer at cybersecurity consultancy Guidepoint Security, said the update marked a “sobering moment for cybersecurity.”

“This reveals an already known asymmetry,” he said.

“Offensive agents are unrestricted, while the best defensive tools are locked behind barriers that cannot understand context.”

But Jake Moore, global cybersecurity advisor at ESET, said the announcement could also have a competitive dimension.

He argued that OpenAI could be trying to highlight its own AI capabilities, while rival Anthropic attracts increasing attention for its Claude Mythos model.

“This raises the possibility that OpenAI is chasing the marketing success that Anthropic has been achieving lately,” he said.

This comes a week after Chinese AI startup Moonshot unveiled Kimi K3, a huge new AI model that it claims could rival major US companies.

This article was originally written in English and we used an artificial intelligence tool to translate it. A BBC journalist reviewed the text before publication. Learn more about how we use AI.

This news has been tken from authentic news syndicates and agencies and only the wordings has been changed keeping the menaing intact. We have not done personal research yet and do not guarantee the complete genuinity and request you to verify from other sources too.