A blogpost from Hugging Face, a company that hosts open source AI models and leaderboards, has stirred up the AI world for two reasons.
First, the company said it had come under a cyber attack from a fully autonomous AI agent that swarmed its system with “tens of thousands of automated actions.” Experts have been warning that AI agents are quickly becoming capable enough to carry these sorts of autonomous attacks—but the Hugging Face hack appears to be among the first real world examples.
That disclosure would normally be news-worthy in and of itself. But what Hugging Face said it did next has received even more attention: the company fought AI with AI, using a Chinese-built open-source model to detect the attack and understand its scope.
Hugging Face said it turned to the Chinese model—Z.ai’s GLM 5.2—after its security team initially tried to use an unnamed frontier AI model from one of the leading U.S. AI companies but found it was unable to do so because of the model’s guardrails. The company said in its blog post that these models “cannot distinguish an incident responder from an attacker.”
That claim was bound to generate a lot of buzz at a time when many in Silicon Valley and Washington, D.C., are deeply worried about the speed of Chinese AI advances. These concerns have been heightened by last week’s debut of Kimi K3, an advanced open-source AI model from Chinese AI startup Moonshot. Some venture capitalists and AI policy analysts who want to see the U.S. do everything possible to accelerate American AI progress in order to stay ahead of China worry that too much emphasis on AI safety, both within the leading U.S. labs and in policy circles, is holding back U.S. progress.
In June, the Trump administration used export controls to block the distribution of Anthropic’s Fable 5 and Mythos 5 models after it received reports of a jailbreak in Fable’s guardrails around cyber tasks. It also initially asked OpenAI to restrict the release of its GPT-5.6 Sol model until OpenAI could offer assurances its guardrails around cyber capabilities were also robust.
David Sacks, the former Trump administration AI and crypto czar, posted the Hugging Face example on social media platform X.com and said, “There’s no reason to limit American models on tasks that Chinese models handle without issue. We’re only making ourselves less competitive.” Referring to the Hugging Face incident specifically he said, “The guardrails actually impaired defensive security.”
Hugging Face CEO Clem Delangue, whose business is built around open source AI and who has previously spoken out against any U.S. policy that would restrict such models for security and safety reasons, told Fortune that the proprietary models from leading U.S. AI companies are actually dangerous to use to defend against a cyber attack. “When you’re in the middle of an active incident, you can’t have your tools refusing to examine malicious payloads or getting your account flagged,” he said. “Open models let us do that work without asking anyone’s permission.”
Although the lack of guardrails on some AI systems may seem risky, Delangue argues it’s necessary to meet attackers on their level.
“Attackers are already using agents, and they obviously don’t respect any guardrails,” he said. “Defenders need the same capabilities, and open-source is the fastest way to put them in everyone’s hands, not just the biggest companies.”
Hugging Face said that from its analysis, the AI agent attacking its systems seems to have acted entirely on its own, without any human initiating the attack or directing its progress.
“We believe we caught the attack before the initiating humans were put in the loop, which helped us win that cybersecurity battle more easily,” Delangue said. “[This ]shows that speed will be key in cybersecurity defense in the age of agents.”
Cybersecurity officials have been warning for the past year that increasingly powerful AI agents would soon be able to carry out autonomous cyber attacks at speeds and scale that could overwhelm conventional cybersecurity methods. But the Hugging Face incident appears to be among the first of a small number of real world autonomous AI cyber attacks that have been documented.
Earlier this month, cybersecurity company Sysdig said it had documented the first completely autonomous ransomware attack in the real world. It dubbed the AI agent that carried out the attack and the method it used “Jadepuffer.” This week Sysdig said it had discovered a new version of Jadepuffer ransomware that specifically targeted trained AI models sitting on corporate networks. These models are considered valuable ransomware targets as they are expensive to train and may not have back-up copies.
To combat the attack it was experiencing, Hugging Face said it used GLM 5.2 running on its own infrastructure to analyze more than 17,000 logs, or footprints, that the attackers left behind.
The company said the attacking AI agent entered its systems through Hugging Face’s data-processing pipeline, a “uniquely exposed” part of AI platforms. It then set up a series of temporary sandboxes, or disposable coding environments in the cloud, where it executed its plan.
The company then fixed the vulnerability, kicked out the attacker, and improved its detection and security guardrails.
“Cybersecurity is always a race between finding and patching exploits,” Delangue said. “AI systems change how this race is run with a different attack surface. Hopefully this will be an example for other organizations to follow to boost up their own defenses.”
Hugging Face said it is still investigating the impact of the attack, and does not know which large language model powered it. The attacker broke into a limited set of internal datasets and credentials, but Hugging Face is still working on assessing the full scope of the attack. The company said it plans to contact any affected parties directly. So far, it has not found any evidence of tampering with public, user-facing models, it said.
GLM 5.2 was released in mid-June by Beijing-based Z.ai, and is the company’s new flagship model. It made waves in Silicon Valley for being on-par with Anthropic’s Claude Opus 4.8 and OpenAI’s GPT-5.5, Business Insider reports. Chinese AI companies have continuously kept the American industry on its toes, beginning with DeepSeek R1 in 2025, and most recently with this month’s release of Kimi K3. Both are open source.
Credit: Source link
