Skip to main content
Use this guide when your application talks to models through the Vercel AI SDK (the ai package) with generateText, streamText, generateObject, streamObject, or ToolLoopAgent, instead of the direct provider SDKs. Inference.net uses the same gateway pattern here as everywhere else: requests go through the Inference.net gateway, the gateway authenticates with INFERENCE_API_KEY, and the gateway forwards requests to the downstream provider using the provider key you supply in x-inference-provider-api-key.
Use the OpenAI-compatible provider (@ai-sdk/openai-compatible), not the AI SDK Gateway provider (@ai-sdk/gateway / createGateway()). createGateway() speaks Vercel’s proprietary AI Gateway protocol and cannot be pointed at an OpenAI-compatible base URL, so it cannot route through Inference.net. createOpenAICompatible({ baseURL }) sends standard OpenAI-compatible requests that the gateway proxies as-is.

​
Install

npm install ai @ai-sdk/openai-compatible

​
Set environment variables

You need two keys:
  • Inference Catalyst project API key — from your dashboard under API Keys
  • Provider API key (in this example, OpenAI) — from your OpenAI account
export INFERENCE_API_KEY=<your-project-api-key>
export OPENAI_API_KEY=<your-openai-api-key>

​
Configure the provider

Point createOpenAICompatible at the gateway. Your project API key goes in apiKey to authenticate the gateway, and your provider key goes in x-inference-provider-api-key so the gateway can forward it downstream. 
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

const inference = createOpenAICompatible({
  name: "inference",
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY!,
  includeUsage: true,
  headers: {
    "x-inference-provider": "openai",
    "x-inference-provider-api-key": process.env.OPENAI_API_KEY!,
    "x-inference-environment": "production",
  },
});

const model = inference("gpt-4.1");
includeUsage: true is useful because usage metadata is what populates token columns in the dashboard. Some providers only return token counts for non-streaming calls or after a stream finishes.

​
Generate text

import { generateText } from "ai";

const { text } = await generateText({
  model,
  prompt: "Hello, world!",
});

console.log(text);

​
Stream text

import { streamText } from "ai";

const result = streamText({
  model,
  prompt: "Stream a short sentence about observability.",
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

​
Custom OpenAI-compatible providers

To route to Gemini, Together AI, Groq, Fireworks, Mistral, OpenRouter, or another OpenAI-compatible provider, keep the provider pointed at the Inference.net gateway and move the original provider URL into x-inference-provider-url. 
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

const gemini = createOpenAICompatible({
  name: "inference-gemini",
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY!,
  includeUsage: true,
  headers: {
    "x-inference-provider-url": "https://generativelanguage.googleapis.com/v1beta/openai",
    "x-inference-provider-api-key": process.env.GEMINI_API_KEY!,
    "x-inference-environment": "production",
  },
});

const model = gemini("gemini-2.5-flash");
When x-inference-provider-url is present, you usually do not need x-inference-provider. The gateway can infer the provider protocol from the overridden URL.

​
Task IDs and per-request metadata

You can attach routing metadata at two levels:
  1. Provider-level headers apply to every call made through that provider client. This is the simplest option when a framework owns the invocation loop and you have no per-call site.
  2. Per-request headers use the headers option on generateText, streamText, generateObject, and streamObject. These are forwarded to the gateway on that request, so you can add or override routing headers per call while sharing one provider client.

​
Provider-level task ID

Add x-inference-task-id to the provider headers so every request is grouped under the same task. 
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

const inference = createOpenAICompatible({
  name: "inference",
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY!,
  headers: {
    "x-inference-provider": "openai",
    "x-inference-provider-api-key": process.env.OPENAI_API_KEY!,
    "x-inference-task-id": "research-agent",
  },
});

​
Per-request task ID

import { generateText } from "ai";

const { text } = await generateText({
  model,
  prompt: "Summarize the incident report.",
  headers: { "x-inference-task-id": "summarize-incident" },
});
The same headers option accepts any routing header, including x-inference-environment and x-inference-metadata-*, so you can tag environment and arbitrary metadata per call.

​
Add tracing

Gateway captures one record per LLM request through the proxy. To also capture the full agent hierarchy (generateText / streamText / tool spans) from inside your code, add Vercel AI SDK tracing. Gateway and tracing are independent and can be used together.

​
Next steps

Gateway overview

Routing headers, supported providers, and the full set of OpenAI-compatible base URLs.

Vercel AI SDK tracing

Capture native AI SDK spans for the full agent and tool-call hierarchy.

Organize with tasks

Group LLM calls by feature or objective to track metrics separately.

Build a dataset

Turn captured traffic into datasets for evals and training.