Check out the newest way to compare different models for a task/agent harness: AutoEvals

Command Reference

Tasks

Inspect and manage a project's tasks.

Tasks are auto-discovered from spans, grouped by trace. Use inf task to inspect a task's rollup summary, cost and latency time series, model distribution, auto-train config, and status history — or to rename, archive, and read its stored config.

Commands

Run inf task <verb> --help for each verb's options. The generated surface covers reads and lightweight edits:

VerbDescription
summaryA task's rollup summary (or null for an empty window)
configGet a task's stored config (name, archived)
renameRename a task
archiveArchive or unarchive a task
status-historyList a task's auto-train status transitions
auto-train-configs / auto-train-summariesList a task's auto-train model configs / rollups
model-distributionA task's model usage distribution
requests-time-series / tokens-time-series / cost-time-seriesRequest-count / token / cost time series
error-rate-time-series / http-code-time-seriesError rate and HTTP status-class counts over time
input-size-time-series / output-size-time-series / payload-size-time-seriesPayload-size time series
duration-percentiles / duration-success-percentiles / ttft-percentilesLatency percentiles
duration-percentile-time-series / ttft-percentile-time-seriesLatency percentile time series
peak-throughputA task's peak request throughput (per-minute/per-hour)
system-prompt-versionsList a task's distinct system-prompt versions
activity-by-taskPer-task request activity + window totals for a batch of tasks
  • inf agent — trace-level grouping; tasks are the finer-grained grouping inside an agent.

On this page