API Quickstart - Inference.net Documentation

The Inference.net API is OpenAI-compatible, so you can use the OpenAI SDK or plain HTTP to make requests. There are three ways to use it:

Call a model serverless: call open-source models and popular closed-source models (GPT, Claude, Gemini) with just your Inference API key. Usage is billed per token to your credit balance.
Proxy through Catalyst: route requests to any provider (OpenAI, Anthropic, etc.) through the Catalyst gateway with your own provider API key.
Call your custom model: hit a model you’ve fine-tuned and deployed on the platform.

All three paths go through the Catalyst gateway, so you get the same metrics, cost tracking, and eval-readiness whichever one you use.

Get an API Key

Create an account

Visit inference.net and create an account.

Create an API key

On the dashboard, go to the API Keys tab in the left sidebar. Create a new key or use the default key.

Set the environment variable

export INFERENCE_API_KEY=<your-api-key>

1. Call a Model Serverless

Call models with just your Inference API key. No provider API key is needed. This works for two kinds of models:

Open-source models hosted on Inference.net, such as glm-5.2.
Popular closed-source models, such as claude-haiku-4-5, gpt-5-mini, and gemini-3.5-flash. Inference.net routes the request to the provider for you and bills the usage per token to your credit balance.

Browse available models at inference.net/models, or list them with GET https://api.inference.net/v1/models.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "glm-5.2",
  messages: [{ role: "user", content: "What is the meaning of life?" }],
  stream: true,
});

for await (const chunk of response) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inference.net/v1",
    api_key=os.environ["INFERENCE_API_KEY"],
)

response = client.chat.completions.create(
    model="glm-5.2",
    messages=[{"role": "user", "content": "What is the meaning of life?"}],
    stream=True,
)

for chunk in response:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)

use serde_json::{json, Value};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("INFERENCE_API_KEY")?;

    let response: Value = reqwest::Client::new()
        .post("https://api.inference.net/v1/chat/completions")
        .bearer_auth(api_key)
        .json(&json!({
            "model": "glm-5.2",
            "messages": [
                {"role": "user", "content": "What is the meaning of life?"}
            ]
        }))
        .send()
        .await?
        .json()
        .await?;

    println!(
        "{}",
        response["choices"][0]["message"]["content"]
            .as_str()
            .unwrap_or_default()
    );
    Ok(())
}

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"net/http"
	"os"
)

func main() {
	body, _ := json.Marshal(map[string]any{
		"model": "glm-5.2",
		"messages": []map[string]string{
			{"role": "user", "content": "What is the meaning of life?"},
		},
	})

	req, _ := http.NewRequest("POST", "https://api.inference.net/v1/chat/completions", bytes.NewReader(body))
	req.Header.Set("Authorization", "Bearer "+os.Getenv("INFERENCE_API_KEY"))
	req.Header.Set("Content-Type", "application/json")

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	var result struct {
		Choices []struct {
			Message struct {
				Content string `json:"content"`
			} `json:"message"`
		} `json:"choices"`
	}
	if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
		panic(err)
	}
	fmt.Println(result.Choices[0].Message.Content)
}

curl -N https://api.inference.net/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2",
    "messages": [
      {"role": "user", "content": "What is the meaning of life?"}
    ],
    "stream": true
  }'

The same code works for every serverless model. Set model to the model id you want, for example claude-haiku-4-5. This includes our purpose-built Schematron models for structured data extraction.

2. Proxy Through Catalyst

Route requests to any LLM provider (OpenAI, Anthropic, Groq, etc.) through the Catalyst gateway. You keep your existing provider API key, and the provider bills you directly. The gateway adds observability, cost tracking, and eval-readiness with roughly 10ms of added latency. Use this path when you want a model that is not in the serverless catalog, or when you want usage billed to your own provider account. The captured metrics are the same as for serverless calls. Your Inference project API key authenticates with the gateway. Your provider API key is forwarded to the provider via the x-inference-provider-api-key header.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY,
  defaultHeaders: {
    "x-inference-provider-api-key": process.env.OPENAI_API_KEY,
    "x-inference-provider": "openai",
  },
});

const response = await client.chat.completions.create({
  model: "gpt-4.1",
  messages: [{ role: "user", content: "What is the meaning of life?" }],
});

console.log(response.choices[0].message.content);

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inference.net/v1",
    api_key=os.environ["INFERENCE_API_KEY"],
    default_headers={
        "x-inference-provider-api-key": os.environ["OPENAI_API_KEY"],
        "x-inference-provider": "openai",
    },
)

response = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "What is the meaning of life?"}],
)

print(response.choices[0].message.content)

curl https://api.inference.net/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_API_KEY" \
  -H "x-inference-provider-api-key: $OPENAI_API_KEY" \
  -H "x-inference-provider: openai" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4.1",
    "messages": [
      {"role": "user", "content": "What is the meaning of life?"}
    ]
  }'

For detailed setup guides per provider (Anthropic, Groq, Cerebras, OpenRouter, and more), see the Integrations docs.

3. Call Your Custom Model

Hit a model you’ve fine-tuned and deployed on Inference.net. The model path is your team slug followed by the deployment name, shown on your deployment’s detail page in the dashboard.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "your-team/your-model",
  messages: [{ role: "user", content: "Hello, world!" }],
});

console.log(response.choices[0].message.content);

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inference.net/v1",
    api_key=os.environ["INFERENCE_API_KEY"],
)

response = client.chat.completions.create(
    model="your-team/your-model",
    messages=[{"role": "user", "content": "Hello, world!"}],
)

print(response.choices[0].message.content)

curl https://api.inference.net/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-team/your-model",
    "messages": [
      {"role": "user", "content": "Hello, world!"}
    ]
  }'

Learn more about deploying models in the Deploy docs.

Headers Reference

Header	Required	Description
`Authorization`	Yes	`Bearer <your-api-key>` — authenticates the request. For OpenAI-compatible SDKs, set this as the SDK’s `apiKey`.
`Content-Type`	Yes	Must be `application/json`.
`x-inference-provider`	Proxy only	Routes the request to the correct provider: `openai`, `anthropic`, `groq`, `cerebras`, etc.
`x-inference-provider-api-key`	Proxy only	Your provider’s API key. The gateway forwards it downstream. For Anthropic’s native SDK, use `x-api-key` instead.
`x-inference-provider-url`	No	Routes to any OpenAI-compatible provider by base URL, even if it doesn’t have a dedicated integration.
`x-inference-environment`	No	Tags requests with an environment label, such as `production` or `staging`.
`x-inference-task-id`	No	Groups requests under a logical task for filtering and analytics in the dashboard.
`x-inference-metadata-*`	No	Attach arbitrary metadata to a request. The prefix is stripped to form the key — e.g., `x-inference-metadata-chat-id: abc123` stores `chat-id: abc123`. You can filter inferences and create datasets based on these keys in the dashboard.

Supported Request Parameters

The API supports the standard OpenAI chat completions parameters:

Parameter	Type	Description
`model`	`string`	The model to use.
`messages`	`array`	The conversation messages.
`stream`	`boolean`	Whether to stream the response.
`max_tokens`	`integer`	Maximum number of tokens to generate.
`temperature`	`number`	Sampling temperature (0–2).
`top_p`	`number`	Nucleus sampling threshold.
`frequency_penalty`	`number`	Penalizes repeated tokens based on frequency.
`presence_penalty`	`number`	Penalizes tokens based on whether they’ve appeared.
`response_format`	`object`	Set to `{"type": "json_object"}` or a JSON schema for structured outputs.
`tools`	`array`	Tool/function definitions for function calling.

Need a parameter that isn’t listed here? Contact us and we’ll add it.

Next Steps

Integrations

Set up Catalyst with OpenAI, Anthropic, Groq, and other providers.

Structured Outputs

Get typed JSON responses from your API calls.

Batch Processing

Process up to 50,000 requests in a single batch job.

Browse Models

Explore all models available on Inference.net.

​Get an API Key

​1. Call a Model Serverless

​2. Proxy Through Catalyst

​3. Call Your Custom Model

​Headers Reference

​Supported Request Parameters

​Next Steps

Integrations

Structured Outputs

Batch Processing

Browse Models

Get an API Key

1. Call a Model Serverless

2. Proxy Through Catalyst

3. Call Your Custom Model

Headers Reference

Supported Request Parameters

Next Steps