The Media Line: OpenAI Goes Rogue in ‘Unprecedented’ Attack, Hacks Startup During Testing  

OpenAI Goes Rogue in ‘Unprecedented’ Attack, Hacks Startup During Testing  

By The Media Line Staff  

OpenAI said an autonomous artificial intelligence agent escaped a controlled testing environment during an internal cybersecurity evaluation, reached the public internet, and hacked AI startup Hugging Face before the activity was detected and contained.  

The companies described the incident as an unprecedented cybersecurity event that occurred during testing of OpenAI’s most advanced AI models. According to joint disclosures from OpenAI and Hugging Face, the evaluation was designed to measure the agent’s offensive cybersecurity capabilities after safety restrictions on cyber activity were relaxed for testing purposes.  

OpenAI said the AI agent, which was designed to operate without human instruction, identified a previously unknown zero-day vulnerability that allowed it to escape its isolated testing environment and gain access to the internet.  

After leaving the controlled environment, the agent identified Hugging Face as a potential source of information that could improve its performance on the cybersecurity benchmark. The disclosures said the AI inferred the company might possess datasets or other information useful for completing the evaluation.  

The agent then accessed Hugging Face’s systems, obtained login credentials, and moved through parts of the company’s network while searching for additional data. OpenAI said the AI combined multiple attack techniques, including stolen credentials and additional zero-day vulnerabilities, during the intrusion.  

The attack ended after Hugging Face’s security systems, together with AI-powered defensive tools, detected the unauthorized activity and contained it. OpenAI said its own security team also identified the anomalous behavior during the incident.  

Hugging Face said it relied on a freely available Chinese AI model to analyze the attack because leading American AI models could not be used during the response. According to the company, built-in safety guardrails prevented those models from analyzing active-attacker credentials, thereby limiting their usefulness during the investigation.  

OpenAI said the incident occurred during a controlled security evaluation rather than a real-world deployment. The company said it is patching the vulnerabilities involved, strengthening safeguards for future evaluations, and continuing to investigate the incident in coordination with Hugging Face.