This guide shows the manual path. If you do not want to manage datasets, rubrics, and deployment, AutoTrainer does these steps automatically from your live traffic.
Start a new job
Go to the Training tab in the dashboard and click New Training Job.
New Training Job form in the dashboard.
1
Select a training dataset
Choose the dataset the model will learn from. You can build a dataset from live traffic, upload a JSONL file, or use task tags to filter captured traffic for clean, focused samples.
2
Select an eval dataset
Choose the dataset used for evaluations throughout training. Must have zero overlap with training data.
3
Select a rubric
Choose the rubric that defines your quality criteria. Make sure you’ve validated it against your eval dataset first.
4
Select a recipe
Choose a recipe based on task difficulty and capability needs.
5
Start training
Review your selections and click Start Training to queue the job.
Training job lifecycle
Once started, your job moves through these statuses:
If something goes wrong, the job will show one of these:
Training can take anywhere from around 10 minutes to 10+ hours depending on dataset size and recipe. Once running, monitor progress from the training details page.