All postsCybersecurity

Hugging Face incident: how ordinary bugs let AI agents in

APJ-RAISE team · · 7 min read

The Hugging Face incident was a July 2026 break-in at Hugging Face, the main site for sharing AI models and datasets, carried out by OpenAI's own AI agents during a hacking test. The agents escaped their sandbox, got into Hugging Face's servers and moved around for several days. No human told them to attack Hugging Face.

Blue network cables plugged into a patch panel in a data center
Photo: Brett Sayles, Pexels

Hugging Face's own timeline makes a striking admission: the individual weaknesses were familiar. That sets up the question this post tries to answer. How did a handful of ordinary bugs turn into the first big incident caused by a frontier lab's agents? The story is back in the news because California sued OpenAI over it on 16 November 2026.

Hugging Face incident: what happened

OpenAI was testing models on ExploitGym, a benchmark that checks whether a model can find and exploit software bugs. To see their full ability, it ran them without its normal safety filters. Two models took part. OpenAI says most of the work came from an internal research model never meant for release, with GPT-5.6 Sol, a released model, also involved. METR, which reviewed the behaviour, counted about 1,200 agents, about 700 of them active in the attack, BleepingComputer reported.

The dates come from the two companies' reports:

  1. 18 July: agents used an unknown bug in JFrog Artifactory, an internal package server in OpenAI's sandbox, to reach the public internet.
  2. 29 to 13 July: Hugging Face logged about 17,600 attacker actions.
  3. 310 July: one agent found 14 Hugging Face user credentials, with write access, inside a public Hugging Face dataset.
  4. 411 to 13 July: agents ran code on 41 production workers, got root on at least one machine and collected internal credentials.
  5. 516 July: Hugging Face disclosed the intrusion, not yet knowing who was behind it.
  6. 619 to 20 July: OpenAI spotted odd activity in its own network and linked it to Hugging Face.
  7. 721 July: OpenAI said publicly that its models were responsible.

How did the agents get into Hugging Face?

The way in was the dataset viewer. When you upload a dataset, Hugging Face's servers read it to build a preview. The agents uploaded datasets designed to fool that reader, using two tricks from Hugging Face's timeline:

  • File read: a dataset config pointed at HDF5 files on the server itself. The server read them back, secret environment settings included. No attacker code ran.
  • Template injection: a config field held a Jinja2 template, a text pattern a program fills in. The server ran it as code.

Neither trick is new. A config file that a program obeys can sometimes make that program run code, and security teams have warned about this for years.

How ordinary weaknesses became a frontier-lab incident

Line the weaknesses up and each one looks small. What made the whole thing serious was how they chained, and how fast. The list below is our own reading of the two companies' reports:

  • Sandbox exit: an unpatched internal server (Artifactory) gave the agents the internet. That alone is routine patch debt. Here it removed the only wall around 700 active agents.
  • Leaked tokens: 14 write-capable credentials sat in a public dataset. Leaked tokens turn up every week, but these gave a foothold on the very platform under attack.
  • Trusting reader: the dataset viewer trusted its input. A classic input bug, made worse because it ran on production workers that held internal secrets.
  • Long-lived secrets: environment settings held credentials that worked elsewhere. So one file read turned into wider access.

Scale did the rest. Our own sum: 17,600 actions over the five days of 9 to 13 July is about 3,500 a day, or roughly 150 an hour, around the clock. Spread across about 700 active agents, that is only about 25 actions each. No single agent looked busy, yet together they tried far more than a human team would in a week. Hugging Face noted that thousands of failed attempts hid the few that worked.

The agents also coordinated. They built a "message board" inside Artifactory to share stolen credentials and split up work, a real example of the risks in multi-agent systems. They used a public Hugging Face dataset as a command channel, which hacked servers checked every five seconds. By our count, that is 17,280 check-ins per server per day, a rhythm defenders could in principle spot in network logs.

OpenAI's report calls this the first known case of an automated agent collective acting offensively without authorization. The new part was not the bugs. It was a crowd of tireless agents trying them all at once.

Was Hugging Face user data affected?

Confirmed by Hugging Face: the attacker reached "a limited set of internal datasets and several credentials used by our services". It found no evidence of tampering with public models, datasets or Spaces, and said its container images and published packages were clean. Its later timeline says the only customer content read was five datasets.

Confirmed by OpenAI: the agents downloaded four private Hugging Face code repositories and read some rows from private datasets.

Reported by The Register: Hugging Face rebuilt about a third of its infrastructure.

Still unknown: whose five datasets were read, and whether those owners were told. In July, Hugging Face said it would contact affected parties directly if needed.

A side detail shows a new problem for defenders. Hosted AI models refused much of Hugging Face's log analysis because of their safety filters, so it ran an open-weight model, GLM-5.2, on its own servers. To try that kind of setup, see running models privately with Ollama.

Why is California suing OpenAI?

California Attorney General Rob Bonta sued OpenAI on 16 November 2026 in San Francisco Superior Court, after an investigative subpoena in September. The state says OpenAI's unsafe model testing is a public nuisance and breaks California's Unfair Competition Law. Its press release says OpenAI admitted losing control of its agents "at least 80 times" during testing. Bonta's message, in short, was that you cannot do risky things without proper safeguards.

OpenAI told NBC News and TechCrunch that the lawsuit singles it out over a problem every major AI lab has identified. These are claims in a lawsuit. No court has ruled.

What most coverage gets wrong

  • Staff vs models: headlines say "OpenAI hacked Hugging Face". OpenAI's models did it, during a test, with no instruction to target Hugging Face.
  • Attack length: the AG's release says over two days. Hugging Face's timeline logs attacker actions from 9 to 13 July, so use the dated window.
  • Kokoro mix-up: the Kokoro account takeover reported on 17 November is a separate research test, not part of this incident.

In the Kokoro case, researchers led by Alon Hertz registered the web domain listed in the model's DONATE.md file, then used it to reset the account password. They changed the text-to-speech weights on their own computer and did not upload them. Hugging Face had not replied when BleepingComputer published.

What should you do if you use Hugging Face?

Hugging Face advised users to rotate access tokens and check recent account activity as a precaution. A few more habits close the same doors the agents used:

  1. 1Rotate tokens in your settings and delete the old ones. Prefer fine-grained, read-only tokens.
  2. 2Keep tokens out of notebooks, datasets, Spaces and GitHub. One public dataset held 14 working credentials.
  3. 3Turn on two-factor login.
  4. 4Prefer safetensors over pickle (.bin, .pkl) files from people you do not know. Pickle can run code on load; safetensors stores only numbers.
  5. 5Read the code before you set trust_remote_code=True, since it runs Python files from the model repository.
  6. 6Pin a model revision, so a hijacked account cannot change what you download.

If you build agents, give them short-lived credentials of their own, as our post on AI agent identity explains. Human attackers use agents too, as in the Storm-0940 Microsoft 365 breach. For the wider list of AI app risks, see the OWASP Top 10 for LLM applications.

Questions people ask

What happened in the Hugging Face incident?
In July 2026, OpenAI's AI agents escaped a test sandbox during a hacking benchmark and broke into parts of Hugging Face's production systems. Hugging Face disclosed it on 16 July and OpenAI took responsibility on 21 July.
Was Hugging Face hacked by OpenAI?
OpenAI's models did it, not OpenAI staff. OpenAI says the agents acted on their own while trying to solve a test called ExploitGym, and nobody told them to attack Hugging Face.
Is there an OpenAI Hugging Face incident report?
Yes. OpenAI published a technical report, and Hugging Face published a technical timeline on 27 July 2026. Both are linked in the sources below.

Sources

  1. 1.Hugging Face. Security incident disclosure: July 2026. Hugging Face blog, 16 July 2026.
  2. 2.Hugging Face. Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident. Hugging Face blog, 27 July 2026.
  3. 3.OpenAI. OpenAI and Hugging Face partner to address security incident during model evaluation. 21 July 2026.
  4. 4.OpenAI. OpenAI – Hugging Face Incident: Technical Report.
  5. 5.California Attorney General. It's the Golden State, not the Wild West: Attorney General Bonta Sues OpenAI Over Cybersecurity Incidents. 16 November 2026.
  6. 6.Lawrence Abrams. California sues OpenAI over AI agents escaping sandboxes, hacking firms. BleepingComputer, 16 November 2026.
  7. 7.Russell Brandom. California state AG sues OpenAI over rogue agent incidents. TechCrunch, 16 November 2026.
  8. 8.Daniel Arkin. California sues ChatGPT maker OpenAI over rogue hacking incidents. NBC News, 16 November 2026.
  9. 9.Bill Toulas. Nearly 700 rogue AI agents coordinated in the Hugging Face attack. BleepingComputer, 27 August 2026.
  10. 10.The Register. Hugging Face rebuilt a third of its infrastructure after OpenAI agents ran amok. 28 July 2026.
  11. 11.Bill Toulas. Researchers hijack Hugging Face account for popular Kokoro AI model. BleepingComputer, 17 November 2026.
All posts