AutoTrainer

Growth plan required. AutoTrainer is available on the Growth plan and above. Contact us to enable it for your team.

AutoTrainer is a hands-off version of the training loop. You point an existing chat completions call at AutoTrainer and pick a teacher model — a large model that’s already good at the task. Requests are served by the teacher while successful responses are collected as training samples. Once enough samples accumulate, the platform trains a distilled model, deploys it, and transparently cuts traffic over. Same endpoint, same code — lower cost and latency. Use it for high-volume, single-purpose tasks with a stable prompt: extraction, classification, summarization, tagging. If you want control over datasets, rubrics, and recipes, use a manual training run instead.

How it works

Every AutoTrainer request belongs to a task. Each task moves through three stages:

Collecting

Requests route to the teacher model. Each successful teacher response is recorded as a training sample and counts toward your minSamples target.

Training

When minSamples successful samples have accumulated (and autoTrain is enabled), the platform builds training and eval datasets from them and trains a distilled model on a pre-configured recipe. Your requests keep serving from the teacher in the meantime.

Live

With autoDeploy enabled, the trained model is deployed to a dedicated GPU, smoke-tested, and your traffic cuts over automatically — no code change required.

📍 TODO:MEDIA

Lifecycle diagram: collecting → training → live, showing traffic on the teacher model until the distilled model takes over.

AutoTrainer never breaks serving. If anything fails — training, deployment, or routing — requests fall back to the teacher model and keep working.

Quickstart

The fastest way to use AutoTrainer is createAutoTrainClient from @inference/sdk, which scopes a task, teacher model, system prompt, and optional output schema into a single typed client.

Install the SDK

npm install @inference/sdk openai zod

Create an AutoTrainer client

import OpenAI from "openai";
import { z } from "zod";
import { createInferenceClient } from "@inference/sdk";

const client = createInferenceClient({
  openai: OpenAI,
  apiKey: process.env.INFERENCE_API_KEY,
});

const personExtractor = client.createAutoTrainClient({
  task: "person-extraction",
  teacherModel: "glm-5.2",
  systemPrompt: "Extract information about people from text.",
  schema: z.object({
    name: z.string(),
    age: z.number(),
  }),
  config: {
    minSamples: 100,
    autoTrain: true,
    autoDeploy: true,
  },
});

Call it like any other model

// Typed as { name: string; age: number }
const person = await personExtractor.run({
  input: "Hello, my name is John and I am 30 years old.",
});

The first minSamples successful calls are served by the teacher model. After that, the platform trains and deploys the distilled model, and this exact call starts returning its responses instead.

Omit schema for free-form text tasks — run() then resolves to the raw response string.

Use the API directly

AutoTrainer works with any OpenAI-compatible client. Set the request model to "auto-train" and pass the configuration as headers. The model value is only a marker — the teacher or the distilled model is chosen server-side. All four x-inference-* configuration headers are required on direct API calls (the SDK fills in defaults for you).

import OpenAI from "openai";
import { createInferenceClient } from "@inference/sdk";

const client = createInferenceClient({
  openai: OpenAI,
  apiKey: process.env.INFERENCE_API_KEY,
});

const response = await client.chat.completions.create(
  {
    model: "auto-train",
    messages: [
      { role: "system", content: "Extract information about people from text." },
      { role: "user", content: "John is 30 years old." },
    ],
  },
  {
    task: "person-extraction",
    autoTrain: {
      teacherModel: "glm-5.2",
      config: { minSamples: 100, autoTrain: true, autoDeploy: true },
    },
  },
);

console.log(response.choices[0].message.content);

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inference.net/v1",
    api_key=os.environ["INFERENCE_API_KEY"],
)

response = client.chat.completions.create(
    model="auto-train",
    messages=[
        {"role": "system", "content": "Extract information about people from text."},
        {"role": "user", "content": "John is 30 years old."},
    ],
    extra_headers={
        "x-inference-task-id": "person-extraction",
        "x-inference-auto-train-teacher-model": "glm-5.2",
        "x-inference-config-min-samples": "100",
        "x-inference-config-auto-train": "true",
        "x-inference-config-auto-deploy": "true",
    },
)

print(response.choices[0].message.content)

curl https://api.inference.net/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-inference-task-id: person-extraction" \
  -H "x-inference-auto-train-teacher-model: glm-5.2" \
  -H "x-inference-config-min-samples: 100" \
  -H "x-inference-config-auto-train: true" \
  -H "x-inference-config-auto-deploy: true" \
  -d '{
    "model": "auto-train",
    "messages": [
      {"role": "system", "content": "Extract information about people from text."},
      {"role": "user", "content": "John is 30 years old."}
    ]
  }'

Configuration reference

SDK option	Header	Default	Description
`task`	`x-inference-task-id`	Required	Stable identifier for the task being distilled. Samples accumulate per task, and each task serves at most one distilled model.
`teacherModel`	`x-inference-auto-train-teacher-model`	Required	The model that serves requests while samples are collected — and the quality bar the distilled model learns from. Any model the gateway can route works.
`config.minSamples`	`x-inference-config-min-samples`	Required	Number of successful teacher responses to collect before training starts. Must be at least 2.
`config.autoTrain`	`x-inference-config-auto-train`	`true`	Whether training starts automatically once `minSamples` is reached. Set to `false` to collect samples without training yet.
`config.autoDeploy`	`x-inference-config-auto-deploy`	`true`	Whether the trained model is deployed and traffic cut over automatically. Set to `false` to train without switching traffic — requests keep serving from the teacher.

A few things to know when choosing values:

teacherModel — pick the model that already does the task well; the distilled model can only be as good as the samples the teacher produces. Models hosted on Inference.net are called directly by identifier (e.g. glm-5.2) with just your Inference API key. Provider models (OpenAI, Anthropic, etc.) also work, with the same provider headers as any gateway request.
minSamples — set it high enough to cover the diversity of your real inputs. More diverse samples produce a better distilled model; a few hundred is a reasonable starting point for most tasks. Only successful teacher responses count toward the target — errors and refused requests are excluded.
autoTrain / autoDeploy — independent gates. autoTrain: true with autoDeploy: false trains a model but keeps serving from the teacher, which is useful when you want to review the trained model before switching traffic.

You can change minSamples, autoTrain, and autoDeploy at any time — the latest values on incoming requests win.

Keep the task identity stable

The combination of task, system prompt, output schema, and teacher model identifies what AutoTrainer is distilling. Keep all four stable so samples accumulate toward the same model. Changing the prompt, schema, or teacher starts a new identity, and a task serves at most one distilled model at a time:

The new identity collects samples from scratch while its requests are served by the teacher. The previous model keeps serving requests that still use the old prompt or schema.
When the replacement finishes training, the task’s deployment is swapped to the new model. The task briefly serves from the teacher during the swap (typically a few minutes), then traffic cuts over.
Each retrain is recorded as a new version of the same deployment, so the version history is the task’s retrain audit trail.
Reverting to a previously used prompt or schema does not resurrect its old model — a retired identity routes to the teacher permanently.

Structured output

Pass a Zod schema to createAutoTrainClient (as in the quickstart) and run() returns parsed, typed output. Under the hood the SDK sends an OpenAI structured-outputs response_format, and the schema becomes part of the task identity. When calling the API directly:

Use response_format with type json_schema if you want structured output. The json_object type is not supported for AutoTrainer requests.
Every AutoTrainer request must include a system (or developer) message.
Requests with and without a schema are separate identities — don’t mix them within a task.

Monitor progress

Open the task in the dashboard and select the Auto-training tab to see samples collected toward the target, the train and deploy flags, and links to the resulting training job, model, and deployment as they’re created.

📍 TODO:MEDIA

Screenshot of the task’s Auto-training tab in the dashboard.

Billing

There is no separate AutoTrainer fee — you pay for the pieces it orchestrates:

Teacher requests bill like any other inference through the gateway.
The training run bills like a manual training run.
The deployment bills like a dedicated deployment. Retrains reuse the task’s existing deployment rather than adding a new one.

Once the distilled model is live, per-request cost drops from teacher-model pricing to your dedicated deployment — that’s the payoff.

Get Started

Gateway

Datasets

Eval

Deploy

Platform

Train

How it works

Quickstart

Use the API directly

Configuration reference

Keep the task identity stable

Structured output

Monitor progress

Billing

Next steps

Tasks

Manage deployments

​How it works

​Quickstart

​Use the API directly

​Configuration reference

​Keep the task identity stable

​Structured output

​Monitor progress

​Billing

​Next steps

Tasks

Manage deployments

How it works

Quickstart

Use the API directly

Configuration reference

Keep the task identity stable

Structured output

Monitor progress

Billing

Next steps