What is LLM-as-a-judge?

LLM-as-a-judge is the practice of using a large language model to grade or compare the outputs of an AI system, in place of a human evaluator. You give the judge model an output (and often a rubric, a reference answer, or two candidates to compare) and ask it to score or pick a winner. It is popular because it scales cheaply and can assess open-ended work that has no single correct string, which exact-match metrics cannot. The catch is that the judge is itself a fallible AI system, so it has to be validated before you trust it.

Is LLM-as-a-judge reliable?

It can be, for the right task and after validation, but it is not reliable by default. LLM judges have well-documented biases: they favour the first or last option in a comparison (position bias), prefer longer answers (verbosity bias), tend to over-score, are sensitive to how the prompt is worded, and cannot verify claims against the real world. Research finds they are weakest exactly where it matters most, such as verifying evidence and judging multi-step reasoning. Used carefully, on coarse judgements, and checked against human labels, a judge is useful. Used blind, it is just an unaccountable AI grading another AI.

What biases affect LLM judges?

The main ones are: position bias (favouring the first or last candidate in a pairwise comparison), verbosity bias (rewarding length over quality), self-preference or self-enhancement bias (a model rating its own outputs, or outputs in its own style, more highly), prompt sensitivity (small wording or example-order changes shifting the verdict), leniency or sycophancy (over-agreeable, over-generous scoring), and poor score calibration (unreliable fine-grained numeric scores). Most of these can be mitigated, but only if you know they are there and design around them.

How do you validate an LLM judge?

You measure the judge against human judgement. Have people label a representative sample of outputs, run the judge on the same sample, and measure agreement (for example with a metric like Cohen's kappa, or simple accuracy against the human labels). If the judge agrees with humans often enough for your risk level, you can use it, and you keep re-checking as your system, prompts and models change. A judge you have never checked against humans is not an evaluation method; it is an assumption.

Should an LLM judge give scores or pass/fail?

Prefer coarse over fine-grained. LLM judges are reasonably reliable at binary or few-category decisions (is this factually supported, yes or no) and unreliable at fine-grained scales (rating something 7 versus 8 out of 10), where they tend to produce arbitrary numbers. If you need a score, decompose it into several specific yes/no criteria and add them up, rather than asking for one holistic 1 to 10. Coarse, criterion-based judging is both more reliable and easier to audit.

Can an LLM judge evaluate its own outputs?

Be very careful. Models exhibit self-preference bias: they tend to rate their own outputs, and outputs written in their own style, more favourably. Using the same model to generate and grade bakes that bias into your results. Where you can, use a different (and ideally stronger) model as the judge than the one under test, and validate its verdicts against human labels regardless. If you must use the same family, treat the scores with extra suspicion and lean harder on human spot-checks.

LLM-as-a-judge: when you can trust it (and when you can't)

There is a shortcut in AI evaluation that is almost too good to refuse: instead of paying humans to grade your system’s output, use a smart model to grade it. It scales to thousands of examples, costs pennies, and it can judge the open-ended work that no exact-match metric can touch. This is LLM-as-a-judge, and it has quietly become the backbone of how teams evaluate AI.

It is genuinely useful. It is also a trap if you use it the way most people first do: blindly. Because the judge is not a ruler or a test suite. It is another AI system, with its own biases and blind spots, now quietly making the decisions you use to decide whether your AI is good enough to ship. This post is how LLM judges fail, and how to build one you can actually trust. It is the companion to the metrics that actually matter and to evaluating AI agents, both of which lean on judges and both of which said this topic deserved its own treatment. Here it is.

What LLM-as-a-judge is, and why it is everywhere

The idea is simple. You give a model an output to assess, usually with some context: a rubric, a reference answer, the source material it was supposed to use, or a second candidate to compare against. Then you ask it to score the output or pick the better one. That verdict becomes your evaluation signal.

It spread for good reasons. Human evaluation is accurate but slow and expensive, and it does not scale to the thousands of runs a real evaluation harness needs. Exact-match scoring is cheap but useless for anything open-ended: summaries, drafts, answers, agent behaviour. LLM-as-a-judge sits in the gap, scalable and able to handle judgement-heavy work. For a lot of evaluation, it is the only practical option.

The catch: your judge is an AI system too

Here is the thing everyone forgets in the rush to automate. A test suite is deterministic and auditable. A human grader is fallible but accountable. An LLM judge is a probabilistic model that can be confidently wrong, and if you have not checked it, you have no idea how often it is. You have not removed the reliability problem from your evaluation; you have moved it into the judge and stopped looking at it. That is worse, because now the unreliability is invisible.

So the question is never “should I use an LLM judge?” It is “is this judge, on this task, good enough to trust, and how do I know?”

How LLM judges fail

The failure modes are well documented, and they are consistent enough to design around once you know them:

Position bias. In a head-to-head comparison, judges systematically favour the answer shown first (or last), regardless of quality. Swap the order and the winner can change.
Verbosity bias. Longer, more elaborate answers get rated higher, even when the extra length adds nothing. Judges mistake fluency and length for quality.
Self-preference bias. A model tends to rate its own outputs, and outputs written in its own style, more highly. Using the same model to generate and grade quietly rigs the result.
Prompt sensitivity. Small changes to the wording of the grading prompt, or the order of examples in it, can shift the verdicts. The judge is not measuring a fixed standard; it is reacting to your phrasing.
Leniency and over-confidence. Judges skew generous and state their wrong verdicts as confidently as their right ones, so the score gives you no signal about its own reliability.
Bad at fine detail. Judges are reasonable at coarse calls (supported or not) and unreliable at fine-grained scores (a 7 versus an 8), where they produce more or less arbitrary numbers.
They cannot verify against reality. A judge assesses linguistic plausibility, not truth. It cannot check a claim against the world, which is why judges are weakest exactly where it matters: verifying evidence and grading multi-step reasoning.

None of this makes LLM judges useless. It makes an unvalidated judge useless, which is a different and more fixable problem.

The one rule: validate the judge against humans

If you take one thing from this post, take this. Before you trust a judge, measure it against human judgement.

The process is not exotic. Have people carefully label a representative sample of outputs. Run your judge on the same sample. Measure how often they agree, with a simple accuracy figure or an agreement statistic like Cohen’s kappa. If the judge agrees with humans often enough for the stakes involved, you can use it, within those limits. If it does not, you fix the rubric, change the model, or fall back to humans for that call. Then you re-validate periodically, because your prompts, models and data drift, and a judge that was reliable last quarter may not be now.

A judge you have never checked against humans is not an evaluation method. It is an assumption wearing the costume of a measurement.

Building a judge you can trust

Beyond validation, a handful of design choices do most of the work:

Keep the task coarse. Ask for binary or few-category verdicts (is this grounded in the source, yes or no), not holistic 1-to-10 scores. If you need a score, decompose it into specific yes/no criteria and add them up.
Give the judge the right inputs. To judge faithfulness, it needs the source material; to judge relevance, the original question. A judge grading blind on the output alone is guessing.
Use a clear rubric. Spell out what good and bad look like with examples. Vague instructions produce vague, unstable judgements.
Control for position bias. In comparisons, run each pair both ways and average, or randomise order, so first-place bias cancels out.
Do not let a model grade itself. Use a different, ideally stronger, model as the judge than the one under test.
Distrust a perfect score. A 100% pass rate almost never means the system is perfect; it means the eval is too easy or the judge too lenient. Make the hard cases part of the set.

Where judges are weakest

Know the edges. LLM judges are at their least reliable on precisely the hardest evaluation problems: verifying whether a claim is actually true, and judging long, multi-step reasoning or agent trajectories. Studies of judges on agent-style failures have found even strong models scoring not far above a coin flip on some of these. So the more consequential and complex the thing you are judging, the more you should lean on validated rubrics, multiple judges, and human review, and the less you should trust a single model’s single-pass verdict.

Judging is a governance decision, not just an engineering one

Step back and the reason this matters for governance becomes obvious. If your evals are the evidence that your AI is safe to ship, and an LLM judge produces those evals, then the judge is part of your control system. An unvalidated judge means your evidence rests on an AI whose reliability you never measured, which is exactly the kind of unaccountable automation that good governance exists to prevent. A validated judge, with its agreement-with-humans documented, is the opposite: a measurement you can stand behind, and one that human oversight can meaningfully sit on top of. Treat your judge with the same rigour you would want applied to any other AI making decisions in your business, because that is what it is.

The short version

LLM-as-a-judge is the scalable way to evaluate open-ended AI output, and often the only practical one. But a judge is not a ruler; it is another AI system, with real and consistent biases: position, verbosity, self-preference, prompt sensitivity, leniency, poor fine-grained scoring, and an inability to check claims against reality. The single rule that separates a trustworthy judge from a rigged one is validation: measure it against human labels, keep the judgements coarse, give it the right inputs, never let a model grade itself, and re-check as things drift. Do that and LLM-as-a-judge is a genuine force multiplier. Skip it and you have not automated your evaluation; you have outsourced it to an AI you forgot to evaluate.

Building evaluation you can actually defend to a customer or regulator? The AI assurance guide covers the full picture, and our AI governance consulting helps you build evaluation, judges included, that holds up under scrutiny. For a quick read on where you stand, the free AI governance check takes about ten minutes, no email.