> ## Documentation Index > Fetch the complete documentation index at: https://docs.inference.net/llms.txt > Use this file to discover all available pages before exploring further. # AutoTrainer > Distill a task onto a smaller, cheaper model automatically. AutoTrainer collects samples from your live traffic, trains a distilled model, and moves your traffic to it. You do not manage datasets, rubrics, or training runs. **Open beta.** AutoTrainer is in open beta. It is available to all users. If you have problems, [contact us](https://inference.net/meet-with-us/). **Automatic deployment is temporarily disabled.** We are opening it up soon. Until then, training still runs end to end, and the trained model is yours to keep: download the weights from the training page and deploy them anywhere, on any provider or your own hardware. You can also [deploy it on our platform](/platform/deploy/deploy-a-model) or [contact us](https://inference.net/meet-with-us/) for a dedicated deployment sized to your needs. See [After training completes](#after-training-completes). AutoTrainer is an automatic version of the [training loop](/platform/train/overview). You point an existing chat completions call at AutoTrainer and select a **teacher model**. The teacher model is a large model that already does the task well. The teacher model serves your requests, and each successful response becomes a training sample. When there are sufficient samples, the platform trains a distilled model, deploys it, and moves your traffic to it. Your endpoint and your code do not change. Your cost and latency decrease. Use AutoTrainer for high-volume, single-purpose tasks that have a stable prompt, for example extraction, classification, summarization, and tagging. If you want control of datasets, rubrics, and recipes, use a [manual training run](/platform/train/launch-a-run). Not ready to train your own model? [AutoEvals](/platform/eval/auto-evals/run-an-auto-eval) compare your task's live traffic against a set of catalog models and recommend the best off-the-shelf option. Start there to find the right model for your task, then come back to AutoTrainer when you want better cost and latency than any off-the-shelf model gives you. ## How it works AutoTrainer is tied to one [task](/platform/gateway/tasks). Each AutoTrainer request must identify its task. Samples collect for each task separately. Each task trains its own distilled model and serves a maximum of one distilled model at a time. If you have two different workloads, give each workload its own task. Each task moves through three stages: The teacher model serves your requests. The platform records each successful teacher response as a training sample. Each sample counts toward your `minSamples` target. When the task has `minSamples` successful samples, and `autoTrain` is on, the platform builds training and eval datasets from the samples. It then trains a distilled model with a pre-configured recipe. The teacher model continues to serve your requests during training. When `autoDeploy` is on, the platform deploys the trained model to a dedicated GPU and does a smoke test. Then it moves your traffic to the trained model automatically. You do not change your code. `autoDeploy` is temporarily disabled; until it opens up, deploy the trained model yourself as described in [After training completes](#after-training-completes). AutoTrainer does not stop your service. If training, deployment, or routing fails, your requests go to the teacher model and continue to operate. ## Quickstart The fastest way to use AutoTrainer is `createAutoTrainClient` from [`@inference/sdk`](https://www.npmjs.com/package/@inference/sdk). This function binds a task, a teacher model, a system prompt, and an optional output schema into one typed client. ```bash theme={"system"} npm install @inference/sdk openai zod ``` ```typescript theme={"system"} import OpenAI from "openai"; import { z } from "zod"; import { createInferenceClient } from "@inference/sdk"; const client = createInferenceClient({ openai: OpenAI, apiKey: process.env.INFERENCE_API_KEY, }); const personExtractor = client.createAutoTrainClient({ task: "person-extraction", teacherModel: "glm-5.2", systemPrompt: "Extract information about people from text.", schema: z.object({ name: z.string(), age: z.number(), }), config: { minSamples: 100, autoTrain: true, autoDeploy: true, }, }); ``` ```typescript theme={"system"} // Typed as { name: string; age: number } const person = await personExtractor.run({ input: "Hello, my name is John and I am 30 years old.", }); ``` The teacher model serves the first `minSamples` successful calls. Then the platform trains and deploys the distilled model. After that, the same call returns responses from the distilled model. For free-form text tasks, omit `schema`. Then `run()` resolves to the raw response string. `run()` also accepts these optional fields together with `input`: | Field | Description | | ------------- | --------------------------------------------------------------------------------------------------------------- | | `metadata` | Key-value tags for the request. The SDK sends them as `x-inference-metadata-*` headers for filters and metrics. | | `environment` | Environment label for the request, for example `production` or `staging`. | | `projectId` | Replaces the client's project for this call. | ## Use the API directly The SDK is a convenience, not a requirement. An AutoTrainer request is a standard OpenAI-compatible chat completions call plus four `x-inference-*` headers. You can send it from all languages that have an HTTP client, for example Rust, Go, Java, or Ruby. To make a direct call: * Set the request `model` to `"auto-train"`. This value is only a marker. The server selects the teacher model or the distilled model. * Send all four `x-inference-*` configuration headers with each request. The SDK adds these headers for you. Direct calls must include them. ```typescript TypeScript theme={"system"} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.inference.net/v1", apiKey: process.env.INFERENCE_API_KEY, }); const response = await client.chat.completions.create( { model: "auto-train", messages: [ { role: "system", content: "Extract information about people from text." }, { role: "user", content: "John is 30 years old." }, ], }, { headers: { "x-inference-task-id": "person-extraction", "x-inference-auto-train-teacher-model": "glm-5.2", "x-inference-config-min-samples": "100", "x-inference-config-auto-train": "true", "x-inference-config-auto-deploy": "true", }, }, ); console.log(response.choices[0].message.content); ``` ```python Python theme={"system"} import os from openai import OpenAI client = OpenAI( base_url="https://api.inference.net/v1", api_key=os.environ["INFERENCE_API_KEY"], ) response = client.chat.completions.create( model="auto-train", messages=[ {"role": "system", "content": "Extract information about people from text."}, {"role": "user", "content": "John is 30 years old."}, ], extra_headers={ "x-inference-task-id": "person-extraction", "x-inference-auto-train-teacher-model": "glm-5.2", "x-inference-config-min-samples": "100", "x-inference-config-auto-train": "true", "x-inference-config-auto-deploy": "true", }, ) print(response.choices[0].message.content) ``` ```bash cURL theme={"system"} curl https://api.inference.net/v1/chat/completions \ -H "Authorization: Bearer $INFERENCE_API_KEY" \ -H "Content-Type: application/json" \ -H "x-inference-task-id: person-extraction" \ -H "x-inference-auto-train-teacher-model: glm-5.2" \ -H "x-inference-config-min-samples: 100" \ -H "x-inference-config-auto-train: true" \ -H "x-inference-config-auto-deploy: true" \ -d '{ "model": "auto-train", "messages": [ {"role": "system", "content": "Extract information about people from text."}, {"role": "user", "content": "John is 30 years old."} ] }' ``` ```rust Rust theme={"system"} use reqwest::Client; use serde_json::{json, Value}; #[tokio::main] async fn main() -> Result<(), Box> { let response: Value = Client::new() .post("https://api.inference.net/v1/chat/completions") .bearer_auth(std::env::var("INFERENCE_API_KEY")?) .header("x-inference-task-id", "person-extraction") .header("x-inference-auto-train-teacher-model", "glm-5.2") .header("x-inference-config-min-samples", "100") .header("x-inference-config-auto-train", "true") .header("x-inference-config-auto-deploy", "true") .json(&json!({ "model": "auto-train", "messages": [ {"role": "system", "content": "Extract information about people from text."}, {"role": "user", "content": "John is 30 years old."} ] })) .send() .await? .json() .await?; println!("{}", response["choices"][0]["message"]["content"]); Ok(()) } ``` ```go Go theme={"system"} package main import ( "bytes" "encoding/json" "fmt" "net/http" "os" ) func main() { body, _ := json.Marshal(map[string]any{ "model": "auto-train", "messages": []map[string]string{ {"role": "system", "content": "Extract information about people from text."}, {"role": "user", "content": "John is 30 years old."}, }, }) req, _ := http.NewRequest( "POST", "https://api.inference.net/v1/chat/completions", bytes.NewReader(body), ) req.Header.Set("Authorization", "Bearer "+os.Getenv("INFERENCE_API_KEY")) req.Header.Set("Content-Type", "application/json") req.Header.Set("x-inference-task-id", "person-extraction") req.Header.Set("x-inference-auto-train-teacher-model", "glm-5.2") req.Header.Set("x-inference-config-min-samples", "100") req.Header.Set("x-inference-config-auto-train", "true") req.Header.Set("x-inference-config-auto-deploy", "true") resp, err := http.DefaultClient.Do(req) if err != nil { panic(err) } defer resp.Body.Close() var result struct { Choices []struct { Message struct { Content string `json:"content"` } `json:"message"` } `json:"choices"` } if err := json.NewDecoder(resp.Body).Decode(&result); err != nil { panic(err) } fmt.Println(result.Choices[0].Message.Content) } ``` ```ruby Ruby theme={"system"} require "json" require "net/http" require "uri" uri = URI("https://api.inference.net/v1/chat/completions") request = Net::HTTP::Post.new(uri) request["Authorization"] = "Bearer #{ENV.fetch("INFERENCE_API_KEY")}" request["Content-Type"] = "application/json" request["x-inference-task-id"] = "person-extraction" request["x-inference-auto-train-teacher-model"] = "glm-5.2" request["x-inference-config-min-samples"] = "100" request["x-inference-config-auto-train"] = "true" request["x-inference-config-auto-deploy"] = "true" request.body = JSON.generate({ model: "auto-train", messages: [ { role: "system", content: "Extract information about people from text." }, { role: "user", content: "John is 30 years old." } ] }) response = Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |http| http.request(request) end result = JSON.parse(response.body) puts result.dig("choices", 0, "message", "content") ``` ## Configuration reference | SDK option | Header | Default | Description | | ------------------- | -------------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `task` | `x-inference-task-id` | Required | Stable identifier for the task that AutoTrainer distills. Samples collect for each task separately. Each task serves a maximum of one distilled model. | | `teacherModel` | `x-inference-auto-train-teacher-model` | Required | The model that serves requests during sample collection. The distilled model learns from its responses. All models that the gateway can route are applicable. | | `config.minSamples` | `x-inference-config-min-samples` | Required | The number of successful teacher responses to collect before training starts. The minimum value is 2. | | `config.autoTrain` | `x-inference-config-auto-train` | `true` | Controls if training starts automatically when the task has `minSamples` samples. Set it to `false` to collect samples without training. | | `config.autoDeploy` | `x-inference-config-auto-deploy` | `true` | Controls if the platform deploys the trained model and moves the traffic automatically. Set it to `false` to train without a traffic change. The teacher model then continues to serve requests. Temporarily disabled; see [After training completes](#after-training-completes). | Notes on the values: * **`teacherModel`**: Select a model that already does the task well. The quality of the distilled model is limited by the quality of the teacher samples. For models on Inference.net, use the model identifier, for example `glm-5.2`, with your Inference API key. Provider models, for example OpenAI or Anthropic, also operate here. Use the same provider headers as other [gateway requests](/api/api-quickstart). * **`minSamples`**: Set a value that is sufficient for the full range of your real inputs. More diverse samples make a better distilled model. For most tasks, a few hundred samples is a good initial value. Only successful teacher responses count toward the target. Errors and refused requests do not count. * **`autoTrain` / `autoDeploy`**: These are independent gates. With `autoTrain: true` and `autoDeploy: false`, the platform trains a model, but the teacher model continues to serve. Use this configuration to examine the trained model before you move the traffic. You can change `minSamples`, `autoTrain`, and `autoDeploy` at all times. The most recent values on incoming requests apply. ## Keep the task identity stable The combination of the **task, the system prompt, the output schema, and the teacher model** identifies what AutoTrainer distills. Keep all four stable so that the samples collect toward the same model. If you change the prompt, the schema, or the teacher model, a new identity starts. A task serves a maximum of one distilled model at a time: * The new identity collects samples from zero. The teacher model serves its requests. The previous model continues to serve requests that use the old prompt or schema. * When the replacement model completes training, the platform moves the task's deployment to the new model. During the change, the teacher model serves the task. The change usually takes some minutes. Then the traffic moves to the new model. * The platform records each retrain as a new version of the same deployment. The version history is the retrain record for the task. * If you use an old prompt or schema again, its old model does not come back. A retired identity routes to the teacher model permanently. ## Structured output Give a Zod schema to `createAutoTrainClient` as shown in the [quickstart](#quickstart). Then `run()` returns parsed, typed output. The SDK sends an OpenAI structured-outputs `response_format`, and the schema becomes part of the task identity. When you call the API directly: * For structured output, use `response_format` with the type `json_schema`. AutoTrainer requests do not support the `json_object` type. * Each AutoTrainer request must include a `system` or `developer` message. * Requests with a schema and requests without a schema are different identities. Do not mix them in one task. ## Monitor progress AutoTrainer progress shows on the page of the task that you defined. To see it: 1. Open your project in the dashboard. 2. Go to **Observability**, then **Tasks**. 3. Select the task that you set in your AutoTrainer requests. 4. Select the **Auto-training** tab. The tab shows the samples collected toward the target, the `autoTrain` and `autoDeploy` flags, and links to the training job, the model, and the deployment when the platform creates them. ## After training completes The trained model is yours. You own the weights, and you can deploy them anywhere: on our platform, on another provider, or on your own hardware. * **Check the evals.** Training runs evals automatically, and the results show on the training job page. To test the model further, [run your own evals](/platform/eval/run-a-comparison) against it with your own rubrics, as often as you want. * **Download the weights.** Open the training job page and download the model weights. What you do with them is up to you. * **Deploy it.** [Deploy the model on our platform](/platform/deploy/deploy-a-model), or [contact us](https://inference.net/meet-with-us/) and we will help set up a dedicated deployment sized to your traffic and latency needs. ## Billing There is no separate AutoTrainer fee. You pay for the parts that it controls: * **Teacher requests** bill the same as other inference requests through the gateway. * **The training run** bills the same as a [manual training run](/platform/train/launch-a-run). * **The deployment** bills the same as a dedicated deployment. Retrains use the task's current deployment again. They do not add a new deployment. When the distilled model is live, the cost for each request decreases from the teacher-model price to your dedicated deployment price. ## Next steps How task tags group requests for metrics, evals, and training. Monitor, version, and scale the deployment that serves your distilled model.