Your files stay on your device

LLM evaluation rubric builder

Build weighted, anchored scoring rubrics with critical gates and a pass threshold, preview the deterministic score, and export the rubric plus an LLM as judge prompt.
Rubric namePass threshold

- Criterion 1RemoveLabelWeightScale lowScale highCritical gateA score under the minimum fails the whole preview, whatever the average saysCritical minimumAnchors, one per level (1 to 5)Preview score (blank skips)
- Criterion 2RemoveLabelWeightScale lowScale highCritical gateA score under the minimum fails the whole preview, whatever the average saysAnchors, one per level (1 to 5)Preview score (blank skips)
- Criterion 3RemoveLabelWeightScale lowScale highCritical gateA score under the minimum fails the whole preview, whatever the average saysAnchors, one per level (1 to 5)Preview score (blank skips)
Load JSON

A rubric earns its keep when the model grading never does the math: it hands back one integer per criterion, and code applies the weights, the threshold, and the critical gates. Per-criterion normalization is why a 1-5 scale and a 0-1 scale can share one weighted total. Anchors are the whole game, though: a criterion without a written description for its low end grades on vibes, and low scores drift. This page builds and previews rubrics; it does not run models, store rubrics, or collect scores over time.
Options

More AI tools

- Eval dataset builder (/ai/ai-evaluation-dataset-builder)
- Fine-tune JSONL formatter (/ai/jsonl-formatter)
- Context fit (/ai/context-fit)
All AI tools (/ai)

Was this tool useful?
YesNo

Canonical HTML: https://nutter.tools/ai/llm-evaluation-rubric-builder
Markdown version: https://nutter.tools/ai/llm-evaluation-rubric-builder.md
Plain-text version: https://nutter.tools/ai/llm-evaluation-rubric-builder.txt
Agent index: https://nutter.tools/llms.txt