Platform
Glossary
Quick-reference definitions for Inference platform concepts and terminology.
| Term | Definition |
|---|---|
| Task | A user-defined objective that groups LLM calls (e.g., "summarize document," "classify ticket"). Tasks persist even as the implementation changes. |
| Rubric | A plain English description of how to judge model output on a quality dimension. Scored numerically by an LLM judge. |
| Dataset | A curated set of inference samples used for evaluation. |
| Eval dataset | A dataset used to measure model quality. Should remain stable over time. |
| Inference | A single LLM request-response pair captured by the platform. |
| Deployment | A model running on a dedicated GPU, accessible via an OpenAI-compatible API. |
| LLM-as-a-judge | The evaluation mechanism where an LLM scores model outputs against rubric criteria. |
| TTFT | Time to first token. Measures streaming responsiveness. |
| Agent | An entity that groups runs sharing a stable agentId/agentName into one workspace in the Agents dashboard, where its metrics, sessions, trajectories, signals, and HALO analysis live. |
| Trajectory | The user-facing name for a trace in the dashboard. The URL slug and CLI still say traces. |
| Signal | A plain-language classifier you define once; an LLM judge labels each span, trace, or session into metrics you can chart, filter, and alert on. |
| HALO | "Hierarchical Agent Loop Optimization." The open-source agent-loop engine that reads traces, finds systemic failure modes across runs, and writes concrete fixes cited to specific traces. |
| AutoEval | An automated comparison that samples recent traffic, replays it against candidate models, scores outputs with LLM judges, and recommends a better fit with supporting evidence. |