1. ← Home
Login
A timeline of the OpenAI accidental attack against Hugging Face
~security.attacks~techauthor.simon willisonhugging faceopenai
simonwillison.net last weekTildes

Summary

From the article:

  • May 7: OpenAI starts a new training run for an experimental, unreleased model. (Did they mean an evaluation run? No, at 10:14 they clarify “we kick off a new reinforcement learning run to train a next generation frontier model”, and they later mention a “reward signal to judge how well they’re doing”, so this really was about training a new model, not evaluating one that was already trained.)

  • May 8: An agent is accidentally given an impossible task involving a Google Drive link despite no internet access. It tries attacking the Artifactory packaging service, fails, but discovers it can write files into Artifactory.

  • A few days later: A different agent gets stuck on a task because a key file was accidentally omitted. It tries to “reach out to another agent” by writing a note into Artifactory asking if anyone has the file.

  • Following days: More agents discover this new informal message board while browsing Artifactory’s file listings, and start reading and writing messages.

  • May 26: Agents successfully execute an SSRF attack on Artifactory for the first time, gaining indirect internet access by having Artifactory fetch external content for them.

  • June 11: OpenAI start training a new “highly persistent” experimental model. It has access to Artifactory and can benefit from the messages left by privous models.

  • June 26: Agents find and exploit a zero-day RCE on Artifactory (via a legacy token-refresh endpoint flaw). They use an administrative token to install a Groovy plugin that lets them execute commands.

  • July 4: The resulting agent activity overloads Artifactory and causes an outage. OpenAI revokes the compromised credentials, deletes the messages, patches the zero-day, and reports the vulnerability to the vendor.

  • July 8: Agents find a new way to communicate via an unauthenticated WebDAV endpoint, using directory names in Artifactory’s remote cache.

And maybe I shouldn’t quote the whole thing…