Your files stay on your device

LLM evaluation rubric builder

Build weighted, anchored scoring rubrics with critical gates and a pass threshold, preview the deterministic score, and export the rubric plus an LLM as judge prompt.

1 Criteria and anchors

Rubric name

Pass threshold

Criterion 1
Label
Weight
Scale low
Scale high
Critical gateA score under the minimum fails the whole preview, whatever the average says
Critical minimum
Anchors, one per level (1 to 5)
Every claim contradicts the source documents Half or more of the claims are wrong One clear factual error, the rest hold All load-bearing claims hold, one vague detail Every claim holds against the source documents
Preview score (blank skips)
Criterion 2
Label
Weight
Scale low
Scale high
Critical gateA score under the minimum fails the whole preview, whatever the average says
Anchors, one per level (1 to 5)
Misses most of the asked parts Covers one part only Covers the main parts, skips one Covers every part, one thinly Covers every asked part with specifics
Preview score (blank skips)
Criterion 3
Label
Weight
Scale low
Scale high
Critical gateA score under the minimum fails the whole preview, whatever the average says
Anchors, one per level (1 to 5)
Rude or dismissive Cold, bordering on rude Neutral, serviceable Warm and professional Warm, professional, on brand
Preview score (blank skips)

Load rubric JSON

A rubric earns its keep when the model grading never does the math: it hands back one integer per criterion, and code applies the weights, the threshold, and the critical gates. Per-criterion normalization is why a 1-5 scale and a 0-1 scale can share one weighted total. Anchors are the whole game, though: a criterion without a written description for its low end grades on vibes, and low scores drift. This page builds and previews rubrics; it does not run models, store rubrics, or collect scores over time.

All AI tools

Was this tool useful?

LLM evaluation rubric builder

More AI tools