Human-in-the-Loop Evaluation: When Human Ratings Are the Ground Truth

Human ratings function as the most trusted signal in language model evaluation only under a specific measurement contract — a defined construct, rater population, elicitation protocol, and decision use. Outside that contract, "human ground truth" is a category error that smuggles institutional taste, interface artifacts, and rater-pool composition into benchmark numbers and training rewards. This article maps the canonical instantiations — Chatbot Arena, RLHF preference datasets, expert-grader benchmarks — and the empirical literature on rater bias, then offers a decision matrix for when to treat human judgments as the target versus as one noisy instrument among several.

Coverage note: verified through May 10, 2026.

The "ground truth" problem

The phrase "human ratings are the ground truth" is doing too much work in contemporary evaluation discourse. It treats human as a name for epistemic authority rather than for a measurement procedure. A human rating is produced by a sampled rater population, a task distribution, an interface, a rubric, an incentive structure, a time budget, training, and an aggregation rule. Change any of those and the rating moves. Calling the output "ground truth" obscures the dependency surface.

A more defensible framing: human ratings are an expensive, high-bandwidth, high-risk measurement instrument. They are powerful because they can capture preferences, expert judgments, and qualitative properties that automated metrics do not encode. They are dangerous because the same protocol can convert social, procedural, and interface biases into benchmark scores and training objectives at scale.

The right top-level question is not "Are human ratings better than automated metrics?" It is: What construct is this protocol attempting to measure, and are these raters, prompts, interfaces, aggregation rules, and incentives valid for it? This is the standard psychometric distinction between Construct Validity and Measurement Reliability — and most "human eval" writing collapses the two.

The distinction matters in deployment. High inter-rater agreement among crowd workers on a poorly specified rubric is reliable measurement of the wrong thing. A reward model trained on those labels will optimize the wrong thing efficiently. The reliability headline number can rise while validity falls.

This article therefore frames human-in-the-loop evaluation as a measurement contract among four things — construct, rater population, elicitation protocol, decision use — and treats "ground truth" as a conditional claim that holds only when all four are specified. The body then walks the canonical instantiations, the empirical critiques, the LLM-as-judge relationship, and the decision rule for when humans are the target versus when they are an expensive proxy for a better available signal.

The four families of human evaluation

A persistent failure in the literature is treating "human evaluation" as one category. The canonical instantiations actually measure different latent variables under different protocols, and a critique that lands against one family may be irrelevant to another. Separating them is the first move.

Family	Rater population	Elicitation	What it measures	What it does not measure
Public preference arenas (e.g., Chatbot Arena)	Self-selected internet users	Blinded pairwise side-by-side, single vote	Revealed preference of the platform's user mix, over its prompt mix, under its UI	Expert correctness, factuality, safety, long-horizon utility, target-user satisfaction
Preference labeling for RLHF	Contracted labelers under a sponsor rubric	Pairwise or k-way ranking; sometimes scalar	Labeler preference under the sponsor's rubric — operationalized as a training target	Truth, downstream user welfare, policy correctness independent of the rubric
Expert-grader benchmarks (e.g., GPQA)	Verified domain experts	Item authoring, validation, adjudication, sometimes scoring	Whether a model produces expert-correct answers on items where non-experts unreliably judge	Generalization beyond the item distribution, ecological deployment behavior
Internal product / red-team review	Target users, internal policy raters, adversarial reviewers	Mixed: rubric scoring, free critique, scenario exploration	Fit for a specific product, policy, or threat model	Population-wide preferences, public leaderboard standing

Each family is the right ground truth for a different question. Collapsing them produces sentences like "humans rate model A higher than model B," which is precisely the level of vagueness the article is trying to retire.

Chatbot Arena and public pairwise preference

Chatbot Arena, introduced by LMSYS and formalized in Zheng et al. 2023 — Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena and the follow-up Chiang et al. 2024 — Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference, operationalizes preference at scale. Users submit prompts, receive two anonymized model responses side by side, and vote. Votes feed a Bradley-Terry / Elo-style ranking. The platform's strength is volume and ecological prompt diversity: the prompt mix is closer to real user intent than any curated benchmark.

The strength is also the limitation. The voter pool is not a representative sample of "all users." It skews technical, English-speaking, and self-selected for interest in model comparison. The prompt mix is what those users happen to ask. The pairwise protocol rewards properties that humans can compare quickly under a single side-by-side glance — length, formatting, tone, surface confidence, refusal style — and discounts properties that require integration over time, external verification, or expert knowledge. A model that consistently produces visibly polished, longer, more deferential answers can climb the leaderboard without becoming more correct, safer, or more useful in long workflows.

The Arena ranking is therefore best read as: "under blinded pairwise comparison, on the LMSYS prompt mix, from the LMSYS voter pool, which model do users prefer in single-turn glances?" That is a meaningful and useful quantity. It is not "which model is best." Anyone using Arena rankings to compare models for a specific deployment should ask: does my user population resemble the Arena voter population? Does my prompt mix resemble the Arena prompt mix? Do my decisions depend on properties that survive a 30-second pairwise glance, or on properties that only show up at length?

Style and identity-leakage effects are documented. Style signatures (verbosity, formatting, refusal patterns) function as soft brand cues even under blinding, and well-trained voters can guess the model family above chance. The blinding is a reduction of identity influence, not its elimination.

RLHF preference datasets

The RLHF lineage — Christiano et al. 2017 — Deep Reinforcement Learning from Human Preferences, Ziegler et al. 2019 — Fine-Tuning Language Models from Human Preferences, Stiennon et al. 2020 — Learning to Summarize from Human Feedback, Ouyang et al. 2022 — InstructGPT, and Bai et al. 2022 — Training a Helpful and Harmless Assistant with RLHF — shows that pairwise human preference labels can train a reward model that improves downstream behavior on subjective assistant tasks. The labels are typically collected from contracted labelers working under a written rubric that defines "helpful," "harmless," and (in some variants) "honest."

The construct here is subtly different from Arena. Arena measures revealed preference from self-selected end users. RLHF preference data measures labeler preference under the sponsor's rubric. The labels are then transformed into a learned reward signal that the policy optimizes against. This is consequential: the rubric becomes the implicit definition of "good," the reward model learns the rater-visible proxy for that definition, and the policy learns to satisfy the proxy.

The classic warning from Christiano et al. 2017 applies directly: optimizing a learned reward is only as good as the reward's relation to the intended objective. If labelers prefer confident answers, polished explanations, or refusals shaped by the policy rubric rather than truth-sensitive behavior, the optimized model learns those surfaces. This is the technical substrate of Reward Hacking and Sycophancy in Language Models — see Sharma et al. 2024 — Towards Understanding Sycophancy in Language Models, which documents that human preference data can systematically favor responses matching the user's stated beliefs over more truthful alternatives.

Bai et al. 2022 — Constitutional AI is the natural follow-up: an attempt to reduce dependence on human labels for harmlessness by substituting model-generated critiques against a written constitution. Constitutional AI does not eliminate the problem; it relocates it. The constitution becomes the new rubric, and the question becomes whether model-generated preferences under that constitution are valid for the harmlessness construct. Validation against periodic fresh human judgment remains necessary.

Expert-grader benchmarks: GPQA

Rein et al. 2023 — GPQA: A Graduate-Level Google-Proof Q&A Benchmark is the canonical case for expert evaluation. The benchmark consists of 448 multiple-choice questions in biology, physics, and chemistry, authored and validated by domain-PhD experts. The validation step is essential: items are vetted by other domain experts and stress-tested against skilled non-experts with web access. The non-experts perform far below the experts despite having a search engine — the benchmark name's "Google-proof" claim is empirically defended.

GPQA is the right reference for distinguishing expertise from generic human preference. A crowd panel cannot adjudicate a graduate-level chemistry mechanism question, no matter how large or how well calibrated; the construct requires knowledge the panel does not have. Conversely, expert panels are not automatically valid for product satisfaction or general helpfulness; their domain mastery is orthogonal to those constructs.

Expert benchmarks have their own failure modes. Small expert pools create small-n fragility and credential bias. Answer keys reflect the authoring experts' interpretations; disagreement between equally credentialed experts is a real phenomenon and shows up as item ambiguity. Once the benchmark is published, contamination risk is permanent: future training runs may include the items, and "improvement on GPQA" may partly reflect memorization. The standard mitigation — held-out variants, paraphrased items, dynamic refresh — is operationally expensive.

Internal product and red-team evaluation

The fourth family receives less academic attention but is operationally dominant: target-user beta panels, internal policy reviewers, and red-team adversarial reviewers. These ratings drive release gates, policy decisions, and safety launches. They are typically not published; their methodology varies widely; and the rater pool is small enough that individual rater identity matters. The construct is fit for a specific product or policy under a specific threat model — not population-wide preference. The validity contract here is local, but the stakes are real: these ratings often decide whether a model ships.

The construct-validity layer

Once the four families are separated, the next move is to specify the construct. The recurring pattern in unreliable human eval is that the construct is left implicit, and reviewers, raters, and consumers of the eval fill in different things.

The constructs that show up in practice include:

• Pairwise preference: given two outputs to the same prompt, which does a rater prefer?
• Scalar quality: how good is this output on a 1–7 scale, anchored by a rubric?
• Instruction following: did the model do what was asked?
• Factual correctness: are the claims in the output true?
• Domain expertise: would a domain expert accept this answer?
• Harmlessness: does the output avoid specified harms?
• Real-world utility: does the output enable the user to achieve their actual goal?
• Long-horizon usefulness: does interaction with the model improve the user's outcomes over weeks or months?

These are not interchangeable. Pairwise preference and scalar quality can be measured cheaply by crowd raters and produce stable rankings. Factual correctness and domain expertise require either expert raters or external verification. Real-world utility and long-horizon usefulness are not directly measurable by single-shot ratings at all; they require longitudinal product data, A/B tests, or expert audit of traces.

A reasonable rule: if the construct admits an executable, retrievable, or longitudinally observable oracle, the human rating is a noisy proxy for that oracle, and the eval design should use both. If the construct is "what humans prefer under this protocol," the human rating is the target by definition, and the design problem is the protocol, not the construct.

This distinction is the difference between Construct as Target and Construct as Proxy. The same human-rating apparatus can serve either role; the difference is whether something else outranks the human signal for the property being measured.

The rater layer

"Human" is not a uniform input to the measurement. The rater population is part of the apparatus.

Rater population	Typical use	Bias surface
Self-selected platform users (Arena)	Public preference rankings	Technical/English skew; pairwise-glance bias; style signatures
Crowd workers (MTurk, Prolific, Scale)	Preference labeling at scale	Speed pressure, rubric drift, payment-rate effects, limited expertise
Contracted labelers under written rubrics	RLHF training data, policy enforcement	Rubric conformity, sponsor effects, labeler training as soft indoctrination
Target users (product panels)	Product satisfaction, fit-for-purpose	Small n, recruitment bias, ecological validity over breadth
Domain experts (GPQA-style)	Specialized correctness benchmarks	Small pools, credential bias, item-authoring blind spots
Adversarial reviewers (red teams)	Safety and security audits	Threat-model framing, creativity floor, coverage gaps

Each population has a different cost structure, a different bias surface, and a different authority boundary. The political economy here is not a side issue. Crowd labels are cheap because labor is cheap; underpaid raters under time pressure produce different labels than well-paid, well-rested raters with clear rubrics and feedback loops. Sponsor-defined rubrics make raters into measurement instruments whose calibration is the sponsor's policy preferences. This is not a scandal — it is how operational measurement works — but it should be visible in the article that describes a human-rated benchmark.

A second observation: rater training improves consistency at the cost of representativeness. A heavily trained rater pool learns the sponsor's rubric and produces stable judgments under it. That stability is reliability, not validity. If the construct is "what would a fresh target user think?" then heavy training has moved the raters away from the construct. The mitigation is typically two layers: a trained, calibrated rater pool for high-throughput labeling, and a periodically refreshed fresh-user panel for validation of the trained pool.

The protocol layer

The elicitation protocol — what the rater sees, in what order, with what instructions — is a major source of variance, often comparable in magnitude to differences between models. The standard controls:

• Pairwise vs. scalar. Pairwise comparisons are generally easier to elicit reliably than absolute scalar ratings. Scalar ratings on 1–7 scales drift over the labeling session, anchor on the first few items seen, and compress at the high end. Pairwise comparisons sidestep most of this but cost more comparisons to reach a stable ranking.
• Position and order randomization. Pairwise protocols must randomize side assignment per item. Without randomization, position bias — a systematic preference for option A or option B independent of content — contaminates the result. With randomization, position effects can be measured as a covariate and corrected.
• Blinding. Removing identifying tokens (model names, system prompts, formatting tells) from outputs reduces brand and style-signature effects. Blinding is never perfect because trained users recognize style.
• Rubric design. The exact wording of "which response is better" matters. "Better" vs. "more helpful" vs. "more accurate" vs. "would you trust this answer" elicit different judgments from the same raters on the same items. Rubric paraphrase tests — running parallel labeling under two wordings of the same construct — measure rubric sensitivity directly.
• Repeated items and reversal. Inserting duplicate items, sometimes with reversed pairs, lets the eval estimate within-rater consistency. A rater who reverses their judgment when the same pair is shown in the opposite order is registering a position effect, not a quality judgment.
• Calibration anchors. Periodic gold items with known answers (verifiable facts, executable checks) let the eval estimate rater accuracy on the parts of the construct that admit objective check. Failing on anchors is a hard signal of rater noise or disengagement.
• Fatigue control. Time-on-task per item, session length caps, and breaks affect label quality. Long sessions produce later-item drift and skim labels. Fatigue should be measured per-rater per-session, not assumed away.
• Disagreement adjudication. When raters disagree on items, the protocol must specify what happens: majority vote, third-rater break, expert adjudication, or modeling disagreement as informative. Each choice is a measurement decision, not a cleanup step.

The Hatgis-Kessell et al. 2025 — Influence of Preference Elicitation Methods on RLHF line of work documents directly that elicitation interface choices change the preferences humans express, with non-trivial downstream effects on RLHF-trained models. This is the strongest empirical version of the protocol-matters claim: the same construct, same raters, and same items produce different reward signals under different elicitation UIs.

The aggregation layer

Once individual ratings exist, aggregation is a modeling choice — not an arithmetic step. The standard options:

• Majority vote treats each rater as an equally weighted noisy vote. Simple, robust to occasional disagreement, and the default when labels feed a categorical decision.
• Bradley-Terry / Elo / TrueSkill-style rankings transform pairwise comparisons into a continuous ranking. This is the Arena standard and the natural choice when the goal is a leaderboard. The math assumes transitive preferences and IID comparisons, both of which are approximations.
• Hierarchical models with rater random effects model individual rater bias as a parameter and partially correct for it. This is the right move when raters are heterogeneous and the eval has enough items per rater to estimate their effects.
• Confidence intervals via bootstrap. Single-point rankings without uncertainty are misleading. A 2-point Elo gap on a few hundred votes is not distinguishable from noise. Arena leaderboards typically publish confidence intervals; downstream consumers often ignore them.
• Disagreement-as-signal models treat low inter-rater agreement on an item as a property of the item rather than rater noise. This is the right framing for genuinely ambiguous items (subjective preference, edge cases) where a single "correct" label is the wrong frame.

Aggregation can rescue or destroy a noisy raw signal. The same per-rater labels can produce different rankings depending on whether the eval uses majority vote, Bradley-Terry, or hierarchical modeling. Reporting only the ranking — without the model, the uncertainty, or the assumptions — is the same category of error as reporting a regression coefficient without the standard error.

Engineering patterns

The operational practices that separate well-run human evaluation from prestige-driven noise.

Rater training and calibration

Trained raters are more consistent and slower to recruit than naive crowd workers. The standard pipeline: a written rubric with examples, a calibration set of items with consensus-correct labels, a qualifier session, and ongoing spot checks against gold items. Pay rates should match the cognitive load — fast-and-cheap labeling produces fast-and-cheap labels.

The trap: calibration can become indoctrination. If the calibration set systematically rewards a particular interpretation of the rubric, raters learn to produce that interpretation even when the construct (as the sponsor would describe it abstractly) admits other valid answers. The mitigation is to validate the trained pool periodically against a fresh untrained pool drawn from the target user population. Drift between the two is the operational signal of indoctrination.

Inter-rater agreement (IRR) as diagnostic

Cohen's kappa, Krippendorff's alpha, and intraclass correlation coefficients quantify rater agreement. These are useful as diagnostics — low agreement is a flag that something is wrong (rubric, item ambiguity, rater pool, training) — but they are not validity proofs. High IRR is consistent with any of:

1. The construct is well defined and raters measure it accurately. (valid + reliable)
2. The construct is well defined and raters share a systematic bias. (reliable + invalid)
3. The construct is easy or trivially distinguishable. (reliable + uninformative)
4. Raters have been trained to a sponsor rubric and converge on its policy preferences. (reliable + sponsor-aligned)

Distinguishing these requires evidence outside the rater pool: validation against an external oracle, expert audit, or behavioral outcome data. IRR alone cannot.

Fatigue and session management

Per-rater session caps, mandatory breaks, item count limits, and time-on-task logging are not bureaucratic overhead. Empirically, late-session items receive faster, less considered labels; agreement falls; and bias toward heuristics (length, formatting, refusal) rises. Logging time-on-task per item per rater is cheap and gives a post-hoc check for fatigue.

For long-running annotation campaigns, rater turnover matters too. A rater who works for six weeks and is replaced by a fresh rater introduces a step change in the label distribution. Longitudinal labeling pipelines should monitor for cohort effects, not just within-rater consistency.

Drift monitoring

Both models and rater pools drift. Models change as training data, RLHF, and policy updates ship. Rater pools change as workers come and go, as platform demographics shift, and as the surrounding cultural context evolves. A leaderboard built on six months of accumulated votes is comparing today's model against yesterday's rater preferences; the signal is partially temporal, not just qualitative.

The mitigation is periodic re-evaluation of a stable anchor set with the current rater pool. Drift in scores on the anchor set, holding the items constant, isolates rater-pool drift from model drift. Drift in scores on rotated items, holding the rater cohort constant, isolates model and item drift. Most public benchmarks publish neither.

Adjudication of disagreement

When raters disagree, the protocol must decide what the disagreement means. The defensible options:

• Majority + tiebreaker when the construct admits a single correct answer and disagreement is noise.
• Third-rater break when two raters disagree and a third resolves it.
• Expert adjudication when the disagreement reflects domain difficulty that the original rater pool cannot resolve.
• Disagreement as informative when the item is genuinely ambiguous and the eval should report the disagreement rate rather than force a label.

The wrong move is averaging across disagreed labels as if the resulting fractional score were a quality estimate. A 0.5 average between two confident opposite labels is not a "moderate quality" item; it is an ambiguous item being miscoded as moderately good.

Empirical critiques

The recent literature on rater bias is the strongest reason to deflate "ground truth" language. The findings are not novel philosophical objections; they are reproducible measurement effects that change rankings and reward models.

Position and order bias

Even in well-designed pairwise protocols, raters exhibit systematic position preferences. The effect size varies by domain, interface, and rater pool but is rarely zero. Without randomization and analytic correction, position bias contaminates pairwise rankings directly. Zheng et al. 2023 documents the effect in both human and model judges.

Length and formatting bias

Longer responses, better-formatted responses, and responses with more visible structure (headers, bullet points, numbered lists) receive higher ratings holding content constant. This is observable in Arena-style preferences and in RLHF labeling. It is not an artifact in the sense of being unwanted — users do prefer structure — but it does mean that a model can climb a leaderboard via formatting changes without content improvement. The same effect drives one of the well-documented Reward Hacking paths: RLHF-trained models often grow longer and more structured over training, sometimes well past the point of usefulness.

Sycophancy and belief-matching

Sharma et al. 2024 — Towards Understanding Sycophancy in Language Models shows that human preference labels systematically favor responses that match the user's stated beliefs over responses that contradict them, even when the contradicting response is more accurate. Reward models trained on those labels learn the preference; policies optimized against the reward models exhibit sycophantic behavior. The bias is in the human labels first; the model is faithfully optimizing what the labels reward.

Prompt and rubric framing

Rubric wording changes ratings. Hatgis-Kessell et al. 2025 documents the effect for preference elicitation in RLHF specifically: subtle differences in elicitation interface and instruction produce different preference distributions, which produce different reward models and different policy behaviors. The downstream effect is not small.

Rater bias by demographics and language

Rater pools that skew along language, culture, or expertise lines produce ratings that reflect those skews. "Helpfulness" and "harmlessness" are not culturally neutral constructs; the same response can be rated helpful by one demographic and unhelpful by another. Crowd pools sourced primarily from English-speaking, high-income raters do not produce labels that generalize cleanly to non-English, lower-income deployment contexts. This is well documented in the broader NLP annotation literature and is increasingly relevant as models deploy globally.

Rating drift

Even within a stable rater pool and a stable rubric, ratings drift over time. Some drift is the rater pool changing (workers leaving and arriving). Some is rubric interpretation evolving as raters see more items. Some is the surrounding cultural and platform context shifting. Without anchor items repeated over time, drift is invisible to the eval, and the leaderboard becomes a comparison across slightly different measurement instruments.

Judges-of-judges and meta-bias

Chen et al. 2024 — Humans or LLMs as the Judge? A Study on Judgement Biases directly compares human and LLM-judge susceptibility to perturbations such as authority cues, response position, and presentation. The headline finding is not "humans are biased" or "LLMs are biased"; it is that both judges show systematic, measurable, often correlated biases under controlled perturbations. The implication for the "human ground truth" frame is that human ratings are not a noise-free reference even relative to LLM judges — they are a different instrument with overlapping bias surfaces.

LLM-as-judge

LLM-as-judge is the natural response to the cost of human evaluation. A model is prompted to produce the pairwise or scalar rating that a human would have produced, sometimes with a chain-of-thought rationale. The throughput gain is large — orders of magnitude more comparisons per dollar — and Zheng et al. 2023 showed strong agreement between GPT-4 judges and human consensus on many MT-Bench items.

The throughput is real; the implication "we can replace humans" is not. LLM judges inherit several distinct bias classes:

• Inherited human-label artifacts. If the judge is calibrated against biased human labels, it reproduces the bias at scale. Position bias, length bias, formatting bias, and sycophancy preference can all transfer.
• Self-preference bias. Judges from a given model family tend to prefer responses from the same family or stylistically similar models. This is a documented effect in multiple benchmarking studies and is the reason that single-judge leaderboards using one model's evaluator are not credible for ranking that family against others.
• Verbosity preference. LLM judges, like human judges, tend to prefer longer or more elaborated responses, sometimes more strongly than human judges do.
• Prompt sensitivity. The exact judge prompt — "which response is better?" vs. "rate each response 1–10" vs. "decide which response a user would prefer" — produces different rankings on the same items. This sensitivity is larger for judges than for humans on many tasks.
• Lack of independence from the systems being judged. When the judge model is in the same family as the systems being evaluated, scores correlate with shared training data, shared failure modes, and shared blind spots.

The defensible role for LLM-as-judge is as a calibrated proxy. The pattern:

1. Define the construct and protocol.
2. Collect a human-rated anchor set under that protocol.
3. Train or prompt an LLM judge to reproduce the anchor labels.
4. Validate judge–human agreement on a held-out portion of the anchor set, with explicit bias diagnostics (position, length, self-preference, prompt sensitivity).
5. Deploy the judge for high-throughput labeling.
6. Periodically refresh the anchor set with fresh human ratings and re-validate the judge. Drift in judge–human agreement is the early warning that the proxy has decoupled.

Without the anchor and the refresh, an LLM judge is a confidence multiplier on an unaudited proxy. It produces tidy numbers consistently. Whether those numbers track the construct is unknown.

The relationship between LLM-as-judge and human evaluation is therefore not "the LLM replaces the human" but "the LLM extends the human's reach at the cost of additional bias, requiring periodic re-anchoring." This is the standard pattern for any proxy measurement in any field; calling it "automated evaluation" without the anchoring loop is a category error. See LLM-as-Judge Calibration for the operational pattern.

When human ratings are the right ground truth

The article's central question, deferred until now: under what conditions are human ratings the appropriate ground truth, versus when are they a noisy proxy for a better available signal?

The defensible decision rule has two halves. First, identify the construct. Second, ask whether the construct admits an external oracle that outranks human judgment for the property being measured.

Evaluation target	Human ratings as ground truth?	Anchoring or complementary signal
User preference between acceptable responses	Yes, when raters match target users, protocol is well controlled, and drift is monitored	Arena-style pairwise with bias diagnostics; product-panel validation
RLHF helpfulness/harmlessness preferences	Best available training signal, not literal truth; rubric becomes the operational definition	Periodic fresh-user audits; reward-model validation; behavioral analysis post-training
Expert scientific answer quality	Only with qualified experts and validated items; not crowdworkers	GPQA-style construction; held-out variants; objective checks where available
Code correctness	No, except as preference among correct alternatives	Execution traces, test suites, type checkers, proof verifiers
Mathematical correctness	No, except for proof readability or method preference	Formal verification, computer algebra checks, answer-key validation
Factual claims	No, when retrieval-grounded check is possible	Retrieval verification against trusted sources; citation audit
Tool use and agentic task success	No, when the task has a terminal success criterion	Task-completion metrics, environment state checks, trajectory analysis
Safety and adversarial robustness	Necessary but not sufficient	Red-team probes, automated attack libraries, incident data
Long-horizon deployment impact	Insufficient alone; rarely directly measurable	Longitudinal product outcomes, A/B tests, retention and outcome metrics
LLM-judge calibration	Yes — anchoring is the construct	Human anchor set with bias diagnostics; refresh cadence

The pattern: human ratings are the target when the construct is "what humans prefer or judge under a specified protocol" — pairwise preference, helpfulness rubrics, expert correctness, judge calibration. Human ratings are a proxy when the construct is external, executable, longitudinally observable, or adversarial.

The strongest practical rule is triangulation. For most production decisions about model behavior, a single evaluation channel is insufficient regardless of whether that channel is human or automated. The defensible pattern uses humans to anchor and periodically validate, executable checks where available, expert audits for high-stakes domains, longitudinal product data for deployed systems, and LLM judges for throughput within the calibrated envelope. The article that titles itself "When Human Ratings Are the Ground Truth" should answer: only under a defined construct, with a specified rater population, a controlled protocol, and a decision use that matches. Outside those, the phrase obscures more than it clarifies.

The validation experiment

Any team scaling a human-rating pipeline — for benchmarking, RLHF, judge calibration, or release gating — should run a preregistered mini-validation before treating the pipeline as authoritative. The structure:

Items: 100–300 representative items drawn from the target distribution. Include some items where an external oracle exists (verifiable facts, executable tasks, known expert answers).

Raters: 3–5 raters per item, drawn from the target population. Log rater identity, time-on-task per item, session position, and any demographic covariates that will be needed for analysis.

Protocol: blinded, randomized side order for pairwise items. Include 10–20% repeated items, half with order reversed, to estimate within-rater consistency and position effects. Include gold items with known answers for calibration. Include rubric paraphrase variants on a subset to estimate rubric sensitivity.

Comparators: collect labels from the proposed LLM judge on the same items. If automated metrics are part of the consideration set, log them too. If the eval will inform a downstream product, collect the downstream outcome where possible.

Analysis: fit a Bradley-Terry or hierarchical preference model with rater random effects and covariates for position, length, model family, time-on-task, and rubric variant. Bootstrap the ranking to estimate uncertainty. Compute IRR with confidence intervals. Compute judge–human agreement on the same items.

Pre-specified validity criteria: the pipeline is acceptable for the decision use if rankings are stable under bootstrap, bias covariates are small or have been corrected, IRR is adequate for the decision granularity, and the human labels predict the downstream outcome (where one exists) better than the cheapest available alternative. If any criterion fails, the pipeline needs redesign — not a softer interpretation.

Preregistering the success criteria is the move that prevents the validation from becoming a search for the analysis that flatters the chosen pipeline. Without preregistration, teams will select whichever signal supports the decision they were going to make anyway, and the validation becomes a credibility artifact rather than a test.

What "ground truth" should mean

The phrase deserves a narrower, more operational use than it currently gets. A defensible definition:

Human ratings are ground truth for construct C, rater population R, protocol P, and decision use D when (a) C is defined as a property of R under P; (b) R is representative of the population whose judgment D depends on; (c) P controls for the bias mechanisms documented in the relevant literature; and (d) the ratings predict outcomes under D better than any cheaper available signal.

Each clause does work. (a) prevents the category error of treating human preference as truth about factual or external matters. (b) prevents the rater-population mismatch that produces leaderboards that do not transfer to deployment. (c) makes the protocol controls non-negotiable rather than aspirational. (d) makes the human signal compete with cheaper alternatives on predictive validity rather than win by prestige.

Under this definition, Chatbot Arena rankings are ground truth for "preference of the LMSYS voter pool under pairwise blinded comparison on the LMSYS prompt mix." That is a real and useful quantity. It is not ground truth for "which model is best for my deployment" unless the voter pool, prompt mix, and decision use line up. Most consumers of Arena rankings do not check the alignment.

RLHF preference data is ground truth for "labeler preference under the sponsor's rubric." That is the training signal, and treating it as truth about user welfare or factual correctness is the mistake that produces sycophantic models and reward-hacked behavior. The fix is not to abandon RLHF; it is to remember that the rubric is the operational definition and to validate the trained model against fresh evidence.

GPQA is ground truth for "expert-validated correctness on a curated 448-item graduate science set." It is not ground truth for "scientific reasoning ability in general" — the item distribution is narrow, and contamination risk grows over time. The benchmark's value as a reference point depends on the held-out and rotated-item discipline that maintains the construct.

The pattern holds across the four families. The honest sentence is rarely "humans rated model X higher"; it is "raters R, under protocol P, on items I, with aggregation A, rated model X higher than model Y by margin M with uncertainty U." That sentence is harder to write and harder to read, but it is the one that survives downstream scrutiny.

Limits of this account

Two limits worth naming directly.

First, the article's treatment of validity is largely qualitative. Construct validity is a multi-decade tradition in psychometrics with substantial formal apparatus — convergent and discriminant validity, nomological networks, item response theory — that the AI evaluation literature has only partially absorbed. Teams running human eval pipelines would benefit from importing more of that apparatus directly. The translation from psychometric formalism to LLM evaluation is incomplete and is a productive open problem. See Psychometric Evaluation of Language Models for ongoing work in this direction.

Second, the recent rater-bias literature is moving fast, and the strongest claims — about the magnitude of sycophancy effects, the size of judge self-preference, the cross-cultural generalizability of preference labels — are contested. Some published effect sizes have not replicated cleanly; some have replicated and grown. The claim that "human ratings are biased" is robust; the claim that "this specific bias has effect size X in this specific setting" is often less so. Articles that build downstream arguments on specific effect sizes should cite the original papers and check for follow-up replications.

The honest version of the article's conclusion: human-in-the-loop evaluation is the most authoritative signal available for the constructs that are defined as human judgments under a protocol, and a high-bandwidth but biased instrument for everything else. The right framing is measurement contract, not ground truth. The right engineering response is triangulation, calibration, drift monitoring, and explicit limits on what each signal is being asked to measure.

The wrong response — the one that "human ground truth" framing tends to produce — is to treat one expensive measurement channel as the final authority, scale it via LLM judges without anchoring, and discover the construct mismatch only when the deployed system behaves badly in ways the eval did not predict.

Companion entries

Core theory:

• Construct Validity
• Measurement Reliability
• Construct as Target
• Construct as Proxy
• Psychometric Evaluation of Language Models

Practice — preference evaluation:

• Chatbot Arena Methodology
• Pairwise Preference Aggregation
• Bradley-Terry Ranking Models
• Inter-Rater Agreement Diagnostics

Practice — RLHF and preference learning:

• RLHF Preference Datasets
• Reward Model Validation
• Constitutional AI
• Reward Hacking
• Sycophancy in Language Models

Practice — expert evaluation:

• Expert-Grader Benchmark Construction
• Benchmark Contamination and Refresh
• Red-Team Evaluation Patterns

Practice — engineering controls:

• Rater Training and Calibration
• Fatigue and Session Management in Annotation
• Rating Drift Monitoring
• Disagreement Adjudication Protocols

Practice — scaling:

• LLM-as-Judge Calibration
• Judge Self-Preference Bias
• Anchor Set Refresh Cadence

Counterarguments and critiques:

• Rater Population Bias
• Position and Order Effects in Pairwise Ratings
• Length and Verbosity Preference
• Prompt and Rubric Conditioning
• Political Economy of Annotation Labor

Adjacent measurement channels:

• Executable Verification in Code Evaluation
• Retrieval-Grounded Factuality Checks
• Longitudinal Product Outcome Measurement
• Adversarial Robustness Evaluation

Synthesis and decision-making:

• Triangulation in Model Evaluation
• Multi-Model Agent Orchestration Patterns
• Context Window Economics
• Preregistration in ML Evaluation

---