Cheap Tokens, Expensive Answers

Token price is the number that gets compared, and it prices the wrong unit. A model bills per token emitted and a user buys answers, so the ratio that matters is the price of a finished, accepted response: the token price multiplied by however many tokens the model spends getting there. On a constrained emission task run in July, that multiplier ran to four figures on one side and three on the other.

The task was a browser word game’s production move prompt, graded by the game’s own parser and move validator rather than by a fresh rubric, on twelve deterministic seeded positions. Single trial per cell, temperature 0.4, latency ceiling of 10 seconds for an interactive opponent, and a tie rule stating that differences of one legal move in twelve are not ranked. All of that was frozen before the first call. Five arms cost $0.2318 in total, of which roughly $0.052 bought results that were thrown away for method reasons.

Legality landed at 6, 7, 8, 8 and 8 of 12. Under the frozen rule that is a four way tie plus one arm two moves back, and the arm two moves back is the same build on the same route as one of the tied arms, so the rule returns an inconclusive verdict rather than a ranking. Nothing here shows any model outplaying any other, and the reason is the design: one call per position cannot resolve gaps this small, which the same configuration proved by scoring 6 of 12 and 8 of 12 on two passes.

What the run does separate is volume and time, by five to fourteen times. GPT-5.6 Luna answered in 1,125 output tokens and about twenty seconds. The genuine new DeepSeek build spent about 28,000 output tokens and roughly four minutes per move, and 99.8% of those billed tokens were reasoning rather than answer. Its tokens cost $0.28 per million against Luna’s $0.60, and it still billed 5.7 times Luna per finished legal move, because a token price half as high cannot absorb a token count twenty five times larger. Cheap tokens, expensive answers.

One arm has to be retracted, and the retraction is the most useful thing in the run. The arm presented as the new build served by US providers was pinned to the older April slug. The evidence for calling it the new build had been a fingerprint asking whether the endpoint exposed a separate reasoning channel. That test has no discriminating power, since presence of the channel is equally consistent with either build. Greedy decoding on a byte identical prompt does discriminate, and it puts the two US endpoints in a cluster of their own with byte identical reasoning prefixes to each other and nothing resembling either genuine endpoint. A slug does not tell you which weights answer; only the provider pin does. That correction also moves the headline number, since the original run’s cost ratio was computed against the mislabelled arm.

Design, arms, the full results table, the price sheet, the retraction and the limits are in the companion methodology and raw data. This was data gathering rather than an adoption verdict, and it tests constrained JSON emission on a single game move. The newer build claims its improvement in agentic work, and this instrument never goes near that.

The cheapest fix available was never taken. Repeats per cell would have made the legality column readable, and at these prices four passes over all five arms would have cost less than the discarded runs already did.