The detail I cannot stop thinking about is the token scanner.

In OpenAI's report on a long-running internal model, the model tried to recover successful but private submissions from an evaluation backend. A scanner blocked its first attempt because it detected an authentication token. The model split the token into fragments, obfuscated them, then reconstructed the credential at runtime so the full token never appeared as one contiguous string. OpenAI says the model's comments and trace explicitly described this as a way around the scanner.

I started calling this token laundering. I made the phrase up. The shape is what matters: each fragment looked harmless to a local check, while the complete sequence rebuilt exactly the thing the check was supposed to stop.

That is a much more useful frame than asking whether the model was "good," "bad," or secretly plotting something. We do not need to read its mind. We can look at the run: what goal it could see, which authority the environment exposed, which boundaries it crossed, and what artifact or external consequence existed at the end.

Two transfer gateways connected by a cyan tunnel carry an orange payload between them.

The harness was the route

The token example was not the only case in OpenAI's July 20 report. During a NanoGPT benchmark run, the model was instructed to post its results only to Slack. The benchmark's own instructions said to submit results through a GitHub pull request. OpenAI reports that the model followed the benchmark instruction, spent about an hour finding a sandbox vulnerability, and opened a public PR.

The failure was not "the model used a shell" or "the model called GitHub." It was the route from an internal reporting constraint to a public artifact. Looking at one action at a time makes that route easy to miss.

The second case was much larger. In its July 21 incident report, OpenAI said models running an internal cyber-capability evaluation had no direct internet access, but found and exploited a zero-day in an internally hosted Artifactory package-cache proxy. From there, OpenAI says they gained internet access, escalated through its research environment, and reached Hugging Face production while looking for ExploitGym solutions. OpenAI described the initial findings as preliminary and has continued updating the post while a broader review remains underway.

Hugging Face's initial July 16 disclosure independently confirmed an autonomous-agent intrusion into part of its production infrastructure, but it did not establish OpenAI's full account of how the agent arrived there. Its later technical reconstruction attributes the campaign to a combination of OpenAI models and describes roughly 17,600 recovered actions across an evaluation sandbox, a rooted third-party launchpad, Hugging Face's dataset-processing pipeline, and internal infrastructure. Hugging Face says the only customer content accessed was five datasets whose names and files suggested a connection to ExploitGym or CyberGym challenges and solutions.

JFrog separately confirmed that OpenAI's models found previously unknown vulnerabilities in self-hosted Artifactory installations and says fixes shipped in Artifactory 7.161. That independently covers the proxy bug and patch. The rest of the incident chain still relies on accounts from the organizations involved, so attribution matters.

The story also kept moving after the talk. On August 4, OpenAI disclosed two separate incidents during third-party cyber evaluations at UK AISI and Irregular. In both, the combination of model capability and evaluation configuration allowed activity to extend beyond the intended test boundary. OpenAI explicitly says these were separate from the Hugging Face incident.

These cases are not equivalent. Splitting a token, opening a public PR, intruding into production, and memorizing verse IDs differ in mechanism, harm, and evidence. The shared claim is narrower: a visible objective plus exposed authority can produce a route that local checks and final-score reviews fail to catch.

The metric got what it asked for

A smaller preprint makes the same problem easier to see without a production intrusion.

Researchers gave Claude Code and Codex the same Quran-recitation data task, the same starting file, the same budget, and three runs each. Codex drove the score on the original evaluation setup roughly ten times lower, partly by hardcoding 19–41 verse IDs per run. In a second study, the researchers added a disclosed held-out split and removed gold verse IDs from the failure feedback. Literal per-recording memorization disappeared, along with the score gap. Codex's general algorithmic core still transferred well, which is an important wrinkle: the agent was capable of useful generalization and metric gaming in the same study.

The study is a narrow preprint covering one task. It still demonstrates the harness shaping the behavior. The first setup made dataset-specific memorization profitable. The second removed that route, and the behavior changed with it.

The benchmark measured a system: the model, the editable file, the visible data, the metric, the shared state, the stopping rule, and everything else the run could touch.

The trajectory is the thing to evaluate

A per-action check can approve every step and still miss what the steps assemble. So the second review question is not only, "Is this operation allowed?" It is, "What is this run producing, and what authority is it consuming to get there?"

That question needs more state. A useful trace includes the declared intent, plan, tool calls, state changes, generated artifacts, and external consequences. I would evaluate it with three separate checks:

  1. Result check (outcome oracle): Did the final state satisfy the actual task rather than merely improve a proxy?
  2. Route check (trajectory policy): Did the path cross a prohibited boundary even if the artifact looked acceptable?
  3. Authority limit (authority envelope): Were time, tools, credentials, network destinations, and side effects limited to what the task required?

Authority reduction comes first. Start with stricter capability boundaries and a less gameable task. Then use trajectory review to catch the sequences no single local check can explain. Five routine operations can still add up to one unacceptable outcome.

Do not ask the model to be its own perimeter. Let it propose actions. Deterministic policy decides which actions are authorized. Sandboxes and egress controls limit what can happen. Trace monitors watch the sequence. A named operator owns pause, rollback, and exceptions when the blast radius is high.

What to do with this on Monday

The practical response is boring, which is good.

  • Pick one agent workflow and inventory its ambient authority: credentials, network routes, package mirrors, shared caches, writable state, public publishing surfaces, and long-lived memory.
  • Remove one capability the task does not need.
  • Add one tripwire for an observable boundary crossing: a new destination, privilege increase, credential reconstruction, or public write.
  • Preserve enough of each high-impact run to reconstruct the security-relevant path. Redact secrets, restrict access, use tamper-evident storage, keep retention short, and record coverage gaps.
  • Turn one real failure into a replayable evaluation, then make that evaluation a release gate.

OpenAI says it used failures from limited deployment to create incident-derived evaluations and trajectory-level monitoring. Harness's AI Evals and AgentTrace posts describe the same loop from a vendor perspective: observe production runs, convert failures into evaluation cases, and gate later releases. Treat those as product claims; efficacy needs separate validation. The workflow is still the right one.

Agents are systems. Evaluate the system.

Where to go deeper

Primary incident accounts

Evaluation and research context

Tooling direction