LLM-as-a-judge explained: how one AI grades another

LLM-as-a-judge is a way to evaluate AI answers by asking a strong language model to grade them, in place of human graders. The judge model reads a question and one or two answers, then gives a score or picks the better answer, often with a short reason.

The idea became popular after a 2023 paper. It showed that GPT-4, used as a judge, agreed with expert humans about as often as experts agree with each other. In 2026, a NeurIPS workshop describes LLM judges as "embedded in training pipelines, safety evaluation, and production deployments at scale". So you should know how they work, where they fail and how to test one.

What is LLM-as-a-judge?

LLM-as-a-judge is the practice of using one large language model (LLM) to grade the outputs of another model. The judge gets a prompt with the question, the answer to grade and the rules for grading. It replies with a verdict such as "8 out of 10" or "Answer B is better".

Example: you build a chatbot that explains science in simple English, and you have 500 test questions. Reading all 500 answers after every change takes hours. So you ask a judge model: "Rate this answer from 1 to 5 for correctness and clarity. Explain your score in one sentence." The judge grades all 500 for you.

Why do researchers use an LLM as a judge?

Researchers use LLM judges because many AI answers have no single correct answer, and human graders are slow and costly. A chatbot reply, a summary or an essay can be good in many different ways.

• Old metrics fail: word-overlap scores such as BLEU and ROUGE have a fairly low correlation with human ratings, especially for creative tasks. The G-Eval paper (2023) points this out.
• Humans are costly: the 2023 MT-Bench paper calls human preferences "very expensive to obtain". To collect about 3,000 votes, it used 58 expert graders, mostly graduate students.
• Speed: a judge can grade a new batch of answers each time you change your model, so you can test ideas quickly.
• Explanations: a judge can explain its verdict, so you can read why an answer lost.

Judges now grade whole research papers too. The "AI Scientist" system, for example, uses an automated reviewer to score the papers it writes.

How does LLM-as-a-judge work?

An LLM judge works in one of three ways. The MT-Bench paper by Lianmin Zheng and colleagues describes all three:

• Pointwise scoring: the judge sees one answer and gives it a score, for example from 1 to 10. The paper calls this "single answer grading".
• Pairwise comparison: the judge sees one question and two answers. It says which answer is better, or calls it a tie.
• Reference-guided: the judge also gets a reference answer to compare against. This helps most with maths and code, where a right answer exists.

Every answer goes through the same prompt. You then compare models by their average score or by how often they win.

Pointwise scoring vs pairwise comparison: which is better?

Pairwise comparison is often a little more accurate and stable. Pointwise scoring is cheaper and easier to scale. The MT-Bench paper lists the trade-offs:

• Pairwise: the number of pairs grows fast. With 10 models, you need 45 pairs for every question.
• Pointwise: one call per answer, so it scales well. But scores can miss small differences between two answers, and they can shift more if you change the judge model.

In the paper, GPT-4 with pointwise scores still matched human preferences very well. Our view: use pairwise comparison when you compare two versions of your own model. Use pointwise scores with a clear rubric when you track one model over time.

What is a rubric in LLM-as-a-judge?

A rubric is a written scoring guide that tells the judge what to look for and what each score means. Without a rubric, the judge uses its own idea of "good", and that idea can change from one answer to the next.

Example rubric for correctness, on a 1 to 5 scale:

• 5: every fact is correct and the answer covers the whole question.
• 3: the main point is correct, but there is one small error or a missing part.
• 1: the main point is wrong.

The G-Eval method (Yang Liu and colleagues, 2023) adds more structure. It gives the judge the task and the criteria, and the model first writes out its own evaluation steps. Then it fills in a score form from 1 to 5. With GPT-4, G-Eval reached a Spearman correlation of 0.514 with human ratings on a summary task. That was better than all earlier methods the authors compared, by a large margin. Still, 0.514 is a moderate match, so even a good judge is far from a copy of human taste.

Our advice: grade one criterion per call, use a short scale such as 1 to 5, and write one line for what each score means.

What biases does an LLM judge have?

LLM judges have known biases. They can favour the first answer, longer answers and their own answers, and they can mark wrong maths as right. The MT-Bench paper measured these with 2023 models:

• Position bias: the judge prefers an answer because of where it appears. When the authors swapped the order of two similar answers, GPT-4 gave the same verdict only 65% of the time. Claude-v1 favoured the first answer in 75% of cases.
• Verbosity bias: the judge prefers longer answers. The authors padded 23 answers with a reworded copy of their own list, with no new information. GPT-3.5 and Claude-v1 preferred the padded answer 91.3% of the time. GPT-4 did so 8.7% of the time.
• Self-preference: the judge may favour answers it wrote itself. Compared with humans, GPT-4 gave its own answers a 10% higher win rate, and Claude-v1 a 25% higher one. The authors said their data was too limited to be sure this is a real bias.
• Weak maths grading: GPT-4 marked a wrong answer to a basic maths question as correct, even though it could solve the question on its own. The answers it was grading misled it.

These numbers come from 2023 models. Newer judges may do better on some tests and worse on others, so measure the judge you actually use. The G-Eval paper also warned that LLM judges may prefer text written by LLMs.

How do you reduce LLM judge bias?

You reduce LLM judge bias mostly by changing how you ask the judge. The first four steps come from the MT-Bench paper. The last two are our own advice.

1. 1Swap the order: judge each pair twice, once as A then B, and once as B then A. Count a win only if the judge picks the same answer both times. If it changes its mind, call it a tie.
2. 2Use a reference: for maths and code, have the judge solve the question first, then show that answer as a reference while it grades. On 10 maths questions, each graded in both orders, this cut GPT-4's wrong verdicts from 14 out of 20 to 3 out of 20.
3. 3Reasons first: ask the judge to work through the question before it scores. This cut wrong verdicts to 6 out of 20, but the judge sometimes made the same mistake as the answer it was grading.
4. 4Add examples: three worked examples raised GPT-4's consistency after swapping from 65% to 77.5%. But each call became 4 times more costly, and the examples may add new biases.
5. 5Watch length: tell the judge not to reward length, then check whether longer answers win more often.
6. 6Use another family: pick a judge from a different company than the models you test, and hide which model wrote each answer.

LLM-as-a-judge vs human evaluation: can you trust the judge?

You can trust an LLM judge only as far as you have checked it against humans on your own task. On general chat questions, the MT-Bench paper found that GPT-4 agreed with expert humans 85% of the time, with ties left out. Two humans agreed with each other 81% of the time.

But agreement depends on how different the two answers are. When one model was much better than the other, GPT-4 agreed with humans nearly 100% of the time. When the two models were close, agreement fell to about 70%. Close cases are often the ones researchers care about most.

The question is still open. NeurIPS 2026, one of the largest AI conferences, has a full workshop in Atlanta in December called "Can We Trust the Judge? Building Reliable Evaluation for Language Models". The problems it names include "sycophancy", "criteria drift" and "positional bias". It plans to publish a "Judge Deployment Disclosure Template", like a model card but for evaluation tools.

How do you check an LLM judge against humans?

You check an LLM judge by grading a small sample with humans, then measuring how often the judge agrees. Do this before you trust its scores on the full dataset.

1. 1Pick a sample: take 100 to 200 answers from your real task, not easy demo cases.
2. 2Get human labels: ask two or three people to grade them with the same rubric. They must not see the judge's verdicts.
3. 3Measure humans first: check how often the humans agree with each other. That is the best you can expect from the judge.
4. 4Run the judge: grade the same sample with your judge prompt. For pairs, run both orders.
5. 5Measure agreement: for A or B verdicts, report percent agreement and Cohen's kappa. For 1 to 5 scores, report Spearman correlation.
6. 6Read the misses: open every case where the judge and humans disagree. Look for patterns, such as longer answers winning or maths marked wrong.
7. 7Report it: in your paper, give the judge model and version, the full prompt and your agreement numbers.

Cohen's kappa measures agreement after removing the agreement you would get by pure chance. A kappa of 0 means chance level, and 1 means perfect agreement.

Keeping the judge's verdicts hidden from human graders is a form of blinding, which our designing experiments course explains. With only 100 to 200 items, your agreement number has real uncertainty. Our statistics for researchers course covers samples and confidence intervals.

A small LLM-as-a-judge research project for students

A good first project is to test one free judge model for the biases above, using public human labels. You can run a small open model on a free notebook service such as Google Colab, so you need no paid API.

1. 1Get the data: the MT-Bench team released about 3,300 human judgments on answers from 6 models to its 80 questions. Their GitHub page links to the data.
2. 2Choose a judge: pick a small open model you can run for free.
3. 3Run pairwise judging: take 200 answer pairs that humans voted on. Ask the judge for A, B or tie, in both orders.
4. 4Measure three things: agreement with humans, consistency after you swap the order, and how often the longer answer wins.
5. 5Try one fix: add a rubric or a reference answer, then measure again.
6. 6Write it up: your question, method, numbers and limits. Compare your results with the 2023 GPT-4 numbers.

A clear research question could be: "Does a small open judge show more position bias than GPT-4 did in 2023?" Keep the question small and the method honest. For a full map of the field, read "A Survey on LLM-as-a-Judge" by Jiawei Gu and colleagues, first posted on arXiv in November 2024.

At APJ-RAISE, students run small projects like this in groups and publish what they find. Joining is free.