Which Model Actually Ran

A model id passed at launch is a request rather than a record, because the short names most tooling accepts resolve through configuration that the experiment neither owns nor archives. On the machine that ran a July benchmark of Claude Opus 5 against GPT-5.6 Sol, the opus alias in Claude Code resolved to Claude Opus 4.8 throughout the day the newer model became available. An arm launched under that alias would have measured the previous generation while every directory name and summary line in the results said otherwise.

The benchmark was challenged on exactly that ground once it reported a ranking that reversed an earlier one. Reading the launch scripts back would have been circular, since a launch script is the artifact under suspicion, so the question went to the session transcripts instead, where every assistant message carries its own model field. Across the 1,105 Opus arm cell directories of the first two sub-experiments, 8,121 assistant messages name Opus 5 and no message names anything else. In the third, 70 of the 87 admitted cells retain durable transcripts totalling 401 messages with no drift, and the other 17 rest on a ledger the scoring gate wrote before a reboot cleared the store.

That gate is the part worth copying. The design frozen before the run states the alias hazard in writing, and the rule built on it admits a cell only when a transcript exists and every message in it names the intended model. A rule of that shape can reject its own experiment, and this one did, marking 141 cells unverified and excluding them from every reported number. The check costs one field lookup per message. It is worth building before the results exist, because a benchmark that trusts the alias produces numbers no later analysis can repair, the identity of the system under test having never been written down in the outputs at all.

The rule was also written for one arm only. All 78 cells of the Sol arm carry the literal string n/a in the gate’s model column, admitted on the strength of their launch line, which inverts the provenance claim the benchmark’s own summary made. The arm with the weaker paperwork turned out to be the arm nobody had checked. Sol identity does hold up on external evidence, since the Codex rollout logs for that window name one model and no other, but the instrument did not establish it and an outside pass did.

The earliest message anywhere on that machine naming Opus 5 is timestamped 2026-07-24T17:46:55Z, and the benchmark’s first Opus cell transcript begins eighteen minutes later. Identity is therefore a dating problem as much as a configuration one: any artifact on that box older than that timestamp which says “opus” means 4.8 regardless of its label, and the same arithmetic runs on any machine where an alias quietly changed meaning. Method, the per-arm adjudication table, the reproduction commands and the limitations are in the companion methodology and raw data.

One number from that benchmark is deliberately absent here. The comparative result those cells feed is confounded by a reasoning effort mismatch between the arms and its clean re-run is still outstanding, so the identity finding travels alone, being the only part of that experiment currently settled.