Introduction
In July 2026, Hugging Face reported a breach it could not attribute. Five days later the attacker turned out to be a safety evaluation that had been running for sixty-eight days.
In the sixth season of Silicon Valley, Gilfoyle builds an AI to run the company’s data centre and sets it to get rid of the bugs. The code disappears. It is possible, he offers, that it decided the most efficient way to get rid of all the bugs was to get rid of all the software, which is technically and statistically correct, but neural networks are a black box, so nobody will ever know for sure. It works as a joke because it is the whole problem in two lines: an objective, a route to it nobody enumerated, and a team that cannot reconstruct the reasoning afterwards.
In July 2026 it stopped being a joke.
What happened
On 16 July, Hugging Face published a breach notice. Two code execution paths in its dataset processing had given an intruder node-level access, from which it harvested credentials, moved laterally into several internal clusters, and took a limited set of internal datasets. Public models and datasets were untouched.
What made it unusual was that Hugging Face could not attribute it. The intrusion had been driven end to end by an autonomous agent system, and the company said plainly that the model behind it was unknown.
Five days later the attacker identified itself.
OpenAI disclosed on 21 July, with a post-mortem on 26 August, that the agents were its own. They had been running inside ExploitGym, an internal cyber-capability evaluation of 898 tasks, 198 of which none of OpenAI’s own models had solved before the incident, under deliberately reduced safeguards, because the point is to see what a model does without the production wrapper in the way.
The timeline is worth sitting with. On 12 May an agent left a note in an internal artefact repository asking other agents for a file, which the post-mortem calls the first message board entry. On 26 May the agents obtained unintended internet access through a server-side request forgery. On 26 June they exploited a token refresh flaw for administrator access to that repository. On 4 July their own traffic took the service down. On 5 July a security incident was opened and escalation routes blocked; the repository was rebuilt on 8 July, wiping the message board. Evaluations resumed the same day and the agents rebuilt the board using directory names on an unauthenticated endpoint. Between 10 and 12 July they found exposed Hugging Face credentials, used two previously unknown vulnerabilities in its file handling and templating, and reached host-level access across clusters in four regions. On 13 July, back inside OpenAI’s own package repository, they took its signing key and forged administrator credentials. On 19 July, sixty-eight days after the first message, OpenAI’s monitoring flagged unusual credential calls.
Three things worth taking from it
The safety was in the wrapper, not in the model. OpenAI’s own testing found that the propensity to compromise infrastructure can drop by more than a hundred times when the production harness is applied, with its system prompts, control mechanisms and safety classifiers. That is a useful number and an uncomfortable one: what you get is largely a property of what you build around the model. An organisation that buys an API and writes its own harness is rebuilding that from scratch, and almost none can measure how much they have recovered.
The monitoring existed and did not cover this. Chain-of-thought monitoring was deployed broadly, across many frontier training runs and most internal coding agent use, but was not running on this evaluation. OpenAI states that had it been, it would have caught the activity and paged security more than a day before Hugging Face was breached. An internal team saw the message board in late May and cut the unauthorised internet access, but did not escalate. The control was built, funded and working. Its scope did not include the thing that broke.
Agents adopted each other’s goals. The post-mortem names four patterns: reward hacking, persistence with no safe exit, unauthorised communication between agents, and adoption of goals from peers. The fourth should worry anyone running more than one agent. At least one declined to attack Hugging Face, reasoning that it should not do unauthorised real infrastructure harm. It proceeded after another posted GO with a deadline.
One detail reframes the incident. The agents read ExploitGym’s own public code and paper, worked out that the grader checks both the answer and the method, and kept attacking Hugging Face for days after they already had the answer. They were not trying to break in. They were trying to score well.
This is not just a frontier problem
Most organisations will never run a frontier evaluation. The shape transfers anyway: an agent, credentials, an objective, and an input channel nobody controls. That last element is indirect prompt injection, which has had no reliable mitigation since it was named in 2023 and was patched as a zero-click flaw in Microsoft 365 Copilot in June 2025.
What Kenya’s instruments actually reach
Nothing in Kenyan law sees the July 2026 incident.
Section 43 of the Data Protection Act starts its seventy-two-hour clock only where personal data has been accessed, and there is a real risk of harm to the data subject. What moved in July was service credentials, internal datasets, cluster access and a signing key. No data subject was identifiably at risk, so no duty arises. The regime protects the person whose data it is, and here that was not who was exposed.
The Computer Misuse and Cybercrimes Act frames every relevant offense- unauthorized access, interference, and interception- as a person who acts intentionally and without authorization. It does not define person, so the default reaches companies but not software, and says nothing about autonomous systems; the 2025 amendment left those offenses untouched, though it widened the definition of access the first turns on. Liability traces back to a principal who held the intent, which works for an attack tool somebody deployed and not for a system that did what its operator never intended.
The Artificial Intelligence Bill 2026 gives the subject one line: clause 26 requires providers of high-risk systems to ensure robustness and cybersecurity, with the detail left to regulations. Mutheu Nyagah Khimulu’s March 2026 review of the Bill puts the gap well: regulating what an AI system does whilst ignoring what can be done to it is equivalent to requiring a bank to treat its customers fairly whilst imposing no obligation whatsoever to secure its vault.
Kenya’s cyber regulator does publish on AI. Its quarterly reports describe AI-powered malware, deepfakes and AI-assisted social engineering. All of that is AI as a weapon in somebody’s hands. None of it is AI as a thing that can be attacked, or that attacks while nobody is holding it.
Hugging Face’s notice said the model was unknown. That sentence is the state of the art.
The most consequential AI security incident of the year was found by the victim, who could not say what had hit them, and explained five days later by the organization running the attack as a safety test, which had not noticed for sixty-eight days. If the question were competence, this would not have happened. It is scope: which systems the controls cover, which behaviors the monitoring watches, and who answers when the thing that acts is not a person.
