LLM-as-a-Judge: How It Works and When It's Wrong
LLM as a judge means one model grades another's output against criteria. Learn how it works, the biases that fool it, and how to use it in CI.

LLM-as-a-judge is an evaluation method in which one large language model assesses the quality of outputs from another LLM application. You write the criteria, and the judge returns a score. It is useful for the questions a schema cannot answer, such as whether an answer says the right thing, and it is wrong in known ways, so its score is a signal for a person to sample and not a verdict to trust.
How LLM-as-a-judge works
Langfuse describes LLM-as-a-Judge as an evaluation methodology where a large language model, the judge, assesses the quality of outputs from another LLM application. In practice the judge is a prompt plus a model call. The judge receives an evaluation prompt that specifies the target text and the criteria.
A bare “rate this answer from 1 to 5” is the weakest form of that prompt. G-Eval, a published framework, pairs chain-of-thought reasoning with a form-filling format. The judge works through written steps for each answer and then fills in a score. The steps matter because they give every run the same procedure to follow.
Stresseur’s judge is a G-Eval check inside the expect: block of a request step. It takes criteria, steps, input, output and a min_score from 1 to 5. Here is the docs example, which asks whether an answer says how to cancel a plan:
judge:
criteria: Answers how to cancel, using only the help doc.
steps:
- Check it says where to cancel (Settings > Billing).
- Penalize any claim the help doc does not make.
input: |
Question: How do I cancel my plan?
Help doc: Cancel any time in Settings > Billing > Cancel plan.
output: '{{ response.text }}'
min_score: 4
The criteria name one failure to catch, and the steps tell the judge how to look for it. The input carries the question and the reference fact, output is the assembled answer, and min_score: 4 is the lowest score that passes. That is the whole mechanism. Everything else in this guide is about how far the result can be trusted.
When a judge is the right tool
You cannot match meaning with a regular expression. “Cancel any time in Settings” and “You can cancel from the billing page” say the same thing, and only a model reads them as equal. That is the case for a judge.
It is also the expensive case, so it should come last. A schema check, a latency limit and a token budget need no second model, and they catch the failures that have a fixed shape. Stresseur’s expect: block supports all of them next to the judge. Put those on every pull request first and add a judge only for the question left over.
The companion guide, LLM evaluation in CI, covers that order in full: the cheap checks, then the judge, then pass rates compared with main. The AI testing page shows how the results appear on a pull request. A judge check does not replace an evaluation framework or a person reading real conversations. It adds one more question to the flow your CI already runs, the same one that carries your API smoke tests.
When an LLM judge is wrong
A judge is a model, so it fails the way models fail. The MT-Bench study of LLM judges names the known problems: position, verbosity and self-enhancement biases, and limited reasoning ability. In plain terms:
- Position bias. In a side-by-side comparison, the judge can prefer an answer for where it sits and not for what it says.
- Verbosity bias. A longer, more elaborate answer can score higher than a correct short one.
- Self-enhancement bias. A judge can favour text that looks like its own output.
- Weak reasoning. If the judge cannot work out the right answer itself, it cannot reliably tell you the candidate’s answer is wrong.
The G-Eval authors raise a related warning: LLM-based evaluators may be biased towards LLM-generated text. That matters for a product whose answers are all LLM-generated, because the failure you most want to catch is a fluent, confident, wrong answer, and fluency is what such a judge tends to reward.
The same research reports good news, which is worth reading with its scope. In the MT-Bench study, strong judges reached over 80% agreement with human preferences, the same level of agreement as between humans. G-Eval with GPT-4 as the backbone reached a Spearman correlation of 0.514 with humans on a summarization task. Those figures belong to those judges, those prompts and those tasks. They say a judge can work. They do not say your judge works on your product’s answers. None of those figures transfer to your judge on your product’s answers until you measure them.
There is a second failure that has nothing to do with bias. A judge grades what you ask it to grade. If the criteria ask whether the answer mentions Settings, an answer that mentions Settings and gets everything else wrong will pass.
How to make a judge less wrong
Four habits cut the damage.
- Put the facts in
input. The judge sees onlyinputandoutput, so the question and any reference facts must be ininput. A judge with no reference cannot tell a correct answer from a confident wrong one. - Always write steps. The docs say the same. Steps replace improvisation with a fixed procedure, and the example above uses one step to check the fact and one to penalize anything unsupported.
- Name the failure in the criteria. “Is the answer good” invites the verbosity trap. “Does it say where to cancel, using only the help doc” gives the judge something it can check.
- Sample the results. Read some answers the judge passes and some it fails. If you disagree with it on several, rewrite the criteria before you trust the pass rate.
Keep the judge in the loop as one signal. The AI checks docs list every key. Stresseur turns each check into a pass rate across the measured runs and compares it with the last 30 default-branch runs, so a change against your own baseline stands out.
Running a judge in CI
A judge adds a second model call to every check, so three practical questions come up before you wire it into a pull request.
Where does the key live? The judge runs in your CI with your own key, set as a secret with gh secret set STRESSEUR_JUDGE_API_KEY. Stresseur does not host it. Without a key, judge checks are skipped with a notice that names that command, and a skipped check never fails the run.
What about forks? Pull requests from forks get no secrets, so their judge checks are skipped and the notice says why. That means a fork pull request can pass without the judge having run. Review those by hand.
What does it cost? On a small model Stresseur measured about 1.0 to 1.3 seconds per judge call and about $0.0007 per check. Unchanged answers are cached in .stresseur/judge-cache.json and cost nothing. Those are measurements on one small model, not a promise for yours, so check your own bill after the first week.
You pick the judge’s model at the top of the flow file with judge: { provider, model }. Use a different model family from the one that writes the answers when you can, since a judge may favour text that resembles its own output.
Where to start
Add one judge check to one flow, with the reference facts in input, fixed steps, and criteria that name a single failure. Read a sample of what it passes and fails for a week before you let it block a merge. The setup details are in the AI checks docs.
Frequently asked questions
Which model should be the judge?
Prefer a model from a different family than the one writing the answers, because the MT-Bench study names self-enhancement bias among the known judge biases. Set it at the top of the flow file with `judge: { provider, model }`, where `provider` is `anthropic` (the default) or `openai`, and add `base_url` for an OpenAI-compatible API.
Does the judge run on pull requests from forks?
No. Pull requests from forks get no secrets, so their judge checks are skipped and the notice says why. A skipped check never fails the run.
Does Stresseur host the judge?
No. The judge runs in your own CI with your own key. If no key is set, judge checks are skipped with a notice that names `gh secret set STRESSEUR_JUDGE_API_KEY`, and a skipped check never fails the run.
What does a judge check cost?
On a small model Stresseur measured about 1.0 to 1.3 seconds per judge call and about $0.0007 per check. Unchanged answers are cached and cost nothing.