Check out the newest way to compare different models for a task/agent harness: AutoEvals

Command Reference

Rubrics

Inspect eval rubrics and their versions.

Inspect eval rubrics (the judge prompts evals run against) and their versions from the terminal.

inf rubric vs inf eval

inf rubric is the read-oriented view of rubrics: get a rubric, list versions, and pin. inf eval rubric is the full workflow — create rubrics, launch run groups, and inspect results. For anything beyond inspection, use inf eval rubric.

Commands

Run inf rubric <verb> --help for each verb's options. The generated surface covers:

VerbDescription
listList rubrics for the team or a project
getGet a rubric with its latest version
get-versionGet a rubric pinned to a specific version
versionsList every version of a rubric
create / create-versionCreate a judge rubric (with its first version) or add a version
renamearchive / unarchive
set-judge-modelSet a rubric's judge model route
set-modelsSet the candidate model routes a rubric evaluates
generate-draftGenerate a draft judge rubric from a dataset's traffic

On this page