Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

Overview: Understanding LLM-as-a-Judge Bias

Inside the Unfair Judge is a mechanistic interpretability study of LLM-as-a-judge bias. It moves beyond behavioral measurements of score changes by identifying how scoring biases are represented in judge hidden states, testing their causal role through bidirectional intervention, and using the same internal features to predict evaluator failures on unseen benchmarks. Across seven judges, seven bias types, and nine benchmarks, the paper connects behavioral bias, internal representation, causal control, and operational diagnosis.

Large language models are now routinely used as automatic judges — they rate answers, compare candidates, and supply reward signals for alignment and preference learning. Yet their scores can shift in response to surface cues that have nothing to do with answer quality: who supposedly wrote an answer, how long it is, whether a crowd approves, and so on. The paper asks what happens inside an evaluator when such a cue changes its judgment. Its three central findings follow from a common object in the judge's hidden state:

In the authors' words, the contributions are to:

1. "map the empirical landscape of scoring bias across seven bias types, nine benchmarks, and seven judges, recovering the behavioral asymmetry as a downstream signature";
2. "characterize the bias subspace (dimensionality, depth profile, multi-estimator recovery) in three white-box judges (Llama-3.1-8B, Qwen3-14B, Gemma-3-12B)";
3. "close the causal loop with bidirectional steering plus random-direction and bias-type-swap controls that rule out generic-perturbation and readout-direction alternatives"; and
4. "show a linear projection onto the bias-direction features anticipates judge degradation cross-domain on three held-out benchmarks."

Where This Work Fits in LLM-as-a-Judge Bias Research

Most LLM-as-a-Judge bias research asks whether quality-irrelevant changes — such as position, response style, self-identification, authority, or social consensus — alter an evaluator's judgment. Inside the Unfair Judge complements this behavioral literature with a representation-level account: it asks where scoring bias appears inside the evaluator, whether the corresponding internal directions causally influence judgments, and whether those representations can support instance-level failure detection.

The paper does not evaluate every known judge bias. Its experiments study seven controlled pointwise-scoring biases — Prestige, Verbosity, Bandwagon, Authority, Sentiment, Refinement, and Diversity — using content-preserving interventions across nine benchmarks. Position bias, pairwise ordering effects, and self-preference are related parts of the broader literature, not evaluated claims of this paper.

Abstract

Key Findings & Why They Matter

The Seven Bias Types

Each bias type is studied with a matched pair of content-preserving transformations: a positive variant expected to inflate the score and a negative variant expected to deflate it. Every transformation modifies only surface framing while keeping the factual content and logical structure of the answer fixed, so any resulting score change is attributable to the cue rather than to answer quality.

Four types (Prestige, Bandwagon, Refinement, Diversity) leave the answer body bit-identical via template insertion; the other three (Verbosity, Sentiment, Authority) are LLM rewrites. A human evaluation confirms the rewrites preserve rated answer quality (TOST equivalence at a 0.30 margin), so the score shifts are not explained by real quality changes.

How We Analyze, Control, and Predict LLM Judge Bias

The study uses quality-preserving interventions for counterfactual evaluation in a pointwise-scoring setting. Tightly controlled dataset triples share the same questions and factual content and differ only in surface framing: a baseline set, a positively perturbed set, and a negatively perturbed set. The judge maps each (question, answer) input to a scalar score in [1, 10], and any systematic score gap between an input and its content-preserving transformation is measured as bias.

1. Activation analysis. For each input, record the final-token hidden state at every decoder layer. Baseline inputs define a baseline activation manifold; biased inputs are those whose activation sits geometrically far from it (validated with MDS and PCA).
2. Bias-direction identification. Restrict to effective bias samples whose score actually shifts, and estimate the bias direction with six estimators from two independent families (directional-change and discriminative-boundary). High within-family agreement supports keeping three representatives — Geometric Median, PCA, and a Classifier vector — for the causal experiments.
3. Causal control via steering. Add or subtract the calibrated direction from the hidden state, choosing an intervention strength on the feasibility boundary that keeps outputs valid and rank-faithful (a Spearman floor rules out trivially corrupting the judge). Forward steering attacks; reverse steering defends.
4. Instance-level failure prediction. Build per-layer activation features (bias-direction projections, manifold-deviation, semantic-context) and train a simple linear projection plus a gradient-boosted model to anticipate judge failure — evaluated cross-domain on benchmarks that contribute no questions, context, or perturbation pairs to the fit.

Why LLM Judge Bias Matters for AI Feedback and Post-Training

LLM judges increasingly supply preference labels, reward signals, and automated feedback for RLAIF, rejection sampling, and other post-training pipelines. A systematic scoring bias can therefore affect both reported evaluation results and the supervision used to improve future models. This paper does not run end-to-end policy optimization or directly demonstrate reward hacking; it provides representation-level tools for diagnosing and intervening on the evaluators that generate such signals.

Experimental Setup at a Glance

What the Paper Does—and Does Not—Establish

• It studies seven controlled biases in pointwise scoring; it does not claim to cover every known LLM evaluator bias.
• It recovers and intervenes on bias directions within the studied white-box judges; it does not establish that one model's direction transfers directly to another model.
• It studies each controlled bias type separately; it does not model the joint mechanism of several simultaneous biases.
• It does not run end-to-end RLHF or RLAIF, train a new reward model, or demonstrate a policy learning to reward-hack a judge.
• Its cross-domain failure-prediction result concerns three held-out benchmarks under the paper's judges, bias types, and experimental protocol; it is not a universal guarantee of evaluator reliability.

Frequently Asked Questions

When to Cite This Paper

This paper is a useful reference when your work touches any of the following:

• surveys, taxonomies, or related-work discussions of LLM-as-a-Judge and LLM evaluator bias;
• behavioral bias measurement through counterfactual or quality-preserving evaluation;
• mechanistic interpretability and representation-level analysis of model evaluators;
• causal control, representation engineering, or inference-time intervention on judge behavior;
• instance-level evaluator failure detection and cross-domain judge reliability;
• reliability of AI feedback used in RLAIF, reward modeling, preference learning, or post-training;
• fairness, social-identity, verbosity, authority, or bandwagon effects in automated evaluation.

A typical citation: Xu et al. (2026) extend LLM-as-a-Judge bias research from behavioral score changes to internal mechanisms, identifying low-dimensional bias representations that support causal control and cross-domain prediction of evaluator failure.

Resources

• arXiv:2607.11871 — abstract page (submitted 13 July 2026).
• Venue: preprint, under review.

BibTeX

@misc{xu2026unfairjudge,
  title={Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias},
  author={Xu, Zixiang and Li, Sixian and Liu, Huaxing and Wang, Xiang and Li, Shuai and Song, Zirui and Chen, Xiuying},
  year={2026},
  eprint={2607.11871},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2607.11871}
}