LangChain

Use this guide when your application talks to models through LangChain wrappers instead of the direct provider SDKs. Inference.net uses the same gateway pattern here as everywhere else: requests go through the Inference.net gateway, the gateway authenticates with INFERENCE_API_KEY, and the gateway forwards requests to the downstream provider using the provider key you supply in x-inference-provider-api-key. The important detail is that LangChain’s wrapper surfaces are not identical to the direct OpenAI and Anthropic SDK constructors. This page documents the wrapper-specific patterns that we validated for Python and TypeScript.

Supported wrappers in this guide

Language	Supported now	Notes
Python	`ChatOpenAI`, `ChatAnthropic`, `init_chat_model(..., provider="openai" / "anthropic")`	Validated against the local gateway and observability stack
TypeScript	`ChatOpenAI`, `ChatAnthropic`	Validated against the local gateway and observability stack

Deferred and unsupported wrappers

These cases are not covered in this pass:

ChatGoogleGenerativeAI
native Google SDKs such as genai.Client()
ChatNVIDIA
config-only model strings in TOML or YAML when there is no concrete wrapper instantiation site to rewrite

The LangChain-specific differences

Compared with the direct SDK patterns, LangChain changes a few important things:

Anthropic wrapper auth is pragmatic, not pure. The LangChain Anthropic wrappers still expect an Anthropic API key field, so the examples below keep anthropic_api_key / apiKey set to the real Anthropic key for wrapper compatibility. Gateway auth still goes through Authorization: Bearer <INFERENCE_API_KEY>, and downstream provider auth still goes through x-inference-provider-api-key.
OpenAI-compatible routing still uses the OpenAI wrappers. For custom OpenAI-compatible providers, keep the LangChain OpenAI wrapper pointed at the Inference.net gateway and move the original provider URL into x-inference-provider-url.
Task IDs are still supported. LangChain can pass request-level task IDs through the validated wrappers, but the exact call shape differs by wrapper. The precise task-ID call patterns are documented below.

Python

Python `ChatOpenAI`

Use ChatOpenAI for direct OpenAI routing by pointing the wrapper at the gateway, authenticating the gateway with INFERENCE_API_KEY, and forwarding the downstream OpenAI key in x-inference-provider-api-key.

import os

from langchain_openai import ChatOpenAI

model = ChatOpenAI(
model="gpt-4o-mini",
temperature=0,
base_url="https://api.inference.net/v1",
api_key=os.environ["INFERENCE_API_KEY"],
default_headers={
"x-inference-provider": "openai",
"x-inference-provider-api-key": os.environ["OPENAI_API_KEY"],
"x-inference-environment": "production",
},
)

Python `ChatAnthropic`

Use the pragmatic LangChain pattern for Anthropic wrappers:

keep anthropic_api_key set to the real Anthropic key
point anthropic_api_url at the Inference.net gateway
send gateway auth through Authorization
send downstream provider auth through x-inference-provider-api-key

import os

from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(
    model="claude-sonnet-4-5-20250929",
    temperature=0,
    anthropic_api_url="https://api.inference.net",
    anthropic_api_key=os.environ["ANTHROPIC_API_KEY"],
    default_headers={
        "Authorization": f"Bearer {os.environ['INFERENCE_API_KEY']}",
        "x-inference-provider": "anthropic",
        "x-inference-provider-api-key": os.environ["ANTHROPIC_API_KEY"],
        "x-inference-environment": "production",
    },
)

Python `init_chat_model`

For the validated OpenAI and Anthropic paths, init_chat_model passes the provider-specific kwargs through to the underlying LangChain wrapper.

OpenAI via `init_chat_model`

import os

from langchain.chat_models import init_chat_model

model = init_chat_model(
model="openai:gpt-4o-mini",
temperature=0,
base_url="https://api.inference.net/v1",
api_key=os.environ["INFERENCE_API_KEY"],
default_headers={
"x-inference-provider": "openai",
"x-inference-provider-api-key": os.environ["OPENAI_API_KEY"],
"x-inference-environment": "production",
},
)

Anthropic via `init_chat_model`

import os

from langchain.chat_models import init_chat_model

model = init_chat_model(
    model="anthropic:claude-sonnet-4-5-20250929",
    temperature=0,
    anthropic_api_url="https://api.inference.net",
    anthropic_api_key=os.environ["ANTHROPIC_API_KEY"],
    default_headers={
        "Authorization": f"Bearer {os.environ['INFERENCE_API_KEY']}",
        "x-inference-provider": "anthropic",
        "x-inference-provider-api-key": os.environ["ANTHROPIC_API_KEY"],
        "x-inference-environment": "production",
    },
)

TypeScript

TypeScript `ChatOpenAI`

For LangChain JS/TS, ChatOpenAI uses configuration.baseURL and configuration.defaultHeaders for gateway routing.

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
model: "gpt-4o-mini",
temperature: 0,
apiKey: process.env.INFERENCE_API_KEY!,
configuration: {
baseURL: "https://api.inference.net/v1",
defaultHeaders: {
"x-inference-provider": "openai",
"x-inference-provider-api-key": process.env.OPENAI_API_KEY!,
"x-inference-environment": "production",
},
},
});

TypeScript `ChatAnthropic`

For LangChain JS/TS, ChatAnthropic uses anthropicApiUrl and clientOptions.defaultHeaders. Use the pragmatic wrapper-compatible pattern:

keep apiKey set to the real Anthropic key
point anthropicApiUrl at the Inference.net gateway
send gateway auth through Authorization
send downstream provider auth through x-inference-provider-api-key

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-sonnet-4-5-20250929",
  temperature: 0,
  apiKey: process.env.ANTHROPIC_API_KEY!,
  anthropicApiUrl: "https://api.inference.net",
  clientOptions: {
    defaultHeaders: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY!}`,
      "x-inference-provider": "anthropic",
      "x-inference-provider-api-key": process.env.ANTHROPIC_API_KEY!,
      "x-inference-environment": "production",
    },
  },
});

Custom OpenAI-compatible providers

When you use a LangChain OpenAI wrapper against Gemini, Together AI, Groq, Fireworks, Mistral, OpenRouter, or another OpenAI-compatible provider, keep the wrapper pointed at the Inference.net gateway and move the original provider URL into x-inference-provider-url.

Python custom provider via `ChatOpenAI`

import os

from langchain_openai import ChatOpenAI

model = ChatOpenAI(
model="gemini-2.5-flash",
temperature=0,
base_url="https://api.inference.net/v1",
api_key=os.environ["INFERENCE_API_KEY"],
default_headers={
"x-inference-provider-url": "https://generativelanguage.googleapis.com/v1beta/openai",
"x-inference-provider-api-key": os.environ["GEMINI_API_KEY"],
"x-inference-environment": "production",
},
)

TypeScript custom provider via `ChatOpenAI`

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gemini-2.5-flash",
  temperature: 0,
  apiKey: process.env.INFERENCE_API_KEY!,
  configuration: {
    baseURL: "https://api.inference.net/v1",
    defaultHeaders: {
      "x-inference-provider-url": "https://generativelanguage.googleapis.com/v1beta/openai",
      "x-inference-provider-api-key": process.env.GEMINI_API_KEY!,
      "x-inference-environment": "production",
    },
  },
});

When x-inference-provider-url is present, you usually do not need x-inference-provider. The gateway can infer the provider protocol from the overridden URL.

Task IDs

The wrappers in this guide support request-level task IDs on a shared client. You do not need to create separate clients per task for the validated wrappers below. The important detail is that the call shape differs by wrapper.

Python task IDs

For the validated Python wrappers in this guide, pass task IDs through extra_headers.

Python `ChatOpenAI` and `ChatAnthropic`

def invoke_with_task(model, question: str, task_id: str):
    return model.invoke(
        question,
        extra_headers={"x-inference-task-id": task_id},
    )

This same extra_headers pattern also works for the validated init_chat_model(..., provider="openai") and init_chat_model(..., provider="anthropic") paths.

TypeScript task IDs

For TypeScript, the exact call shape depends on the wrapper.

TypeScript `ChatAnthropic`

Use headers directly in the second argument to invoke().

import { ChatAnthropic } from "@langchain/anthropic";

async function invokeAnthropicWithTask(
model: ChatAnthropic,
question: string,
taskId: string,
) {
return model.invoke(question, {
headers: { "x-inference-task-id": taskId },
});
}

TypeScript `ChatOpenAI`

Use options.headers in the second argument to invoke().

import { ChatOpenAI } from "@langchain/openai";

async function invokeOpenAIWithTask(
  model: ChatOpenAI,
  question: string,
  taskId: string,
) {
  return model.invoke(question, {
    options: {
      headers: { "x-inference-task-id": taskId },
    },
  });
}

Why this matters

LangChain does not normalize transport-level request options across wrappers. That is why the TypeScript OpenAI and Anthropic wrappers use different task-ID call shapes, even though both ultimately set the same x-inference-task-id header on the proxied request.

Framework-owned invocation loops

Some frameworks accept a LangChain model object and then own the actual .invoke() / .ainvoke() calls internally. In those cases you may not have a wrapper call site where you can attach per-request task IDs yourself. Use a client-level fallback instead:

Python fallback

import os

from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(
model="claude-sonnet-4-5-20250929",
temperature=0,
anthropic_api_url="https://api.inference.net",
anthropic_api_key=os.environ["ANTHROPIC_API_KEY"],
default_headers={
"Authorization": f"Bearer {os.environ['INFERENCE_API_KEY']}",
"x-inference-provider": "anthropic",
"x-inference-provider-api-key": os.environ["ANTHROPIC_API_KEY"],
"x-inference-task-id": "research-agent",
},
)

TypeScript fallback

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gpt-4o-mini",
  temperature: 0,
  apiKey: process.env.INFERENCE_API_KEY!,
  configuration: {
    baseURL: "https://api.inference.net/v1",
    defaultHeaders: {
      "x-inference-provider": "openai",
      "x-inference-provider-api-key": process.env.OPENAI_API_KEY!,
      "x-inference-task-id": "research-agent",
    },
  },
});

This is less granular than a per-request task ID, but it still gives you task grouping when the framework owns the invocation loop.

Integrations

Model providers

Frameworks

Supported wrappers in this guide

Deferred and unsupported wrappers

The LangChain-specific differences

Python

Python `ChatOpenAI`

Python `ChatAnthropic`

Python `init_chat_model`

OpenAI via `init_chat_model`

Anthropic via `init_chat_model`

TypeScript

TypeScript `ChatOpenAI`

TypeScript `ChatAnthropic`

Custom OpenAI-compatible providers

Python custom provider via `ChatOpenAI`

TypeScript custom provider via `ChatOpenAI`

Task IDs

Python task IDs

Python `ChatOpenAI` and `ChatAnthropic`

TypeScript task IDs

TypeScript `ChatAnthropic`

TypeScript `ChatOpenAI`

Why this matters

Framework-owned invocation loops

Python fallback

TypeScript fallback

Integrations

Model providers

Frameworks

​Supported wrappers in this guide

​Deferred and unsupported wrappers

​The LangChain-specific differences

​Python

​Python ChatOpenAI

​Python ChatAnthropic

​Python init_chat_model

​OpenAI via init_chat_model

​Anthropic via init_chat_model

​TypeScript

​TypeScript ChatOpenAI

​TypeScript ChatAnthropic

​Custom OpenAI-compatible providers

​Python custom provider via ChatOpenAI

​TypeScript custom provider via ChatOpenAI

​Task IDs

​Python task IDs

​Python ChatOpenAI and ChatAnthropic

​TypeScript task IDs

​TypeScript ChatAnthropic

​TypeScript ChatOpenAI

​Why this matters

​Framework-owned invocation loops

​Python fallback

​TypeScript fallback

Supported wrappers in this guide

Deferred and unsupported wrappers

The LangChain-specific differences

Python

Python `ChatOpenAI`

Python `ChatAnthropic`

Python `init_chat_model`

OpenAI via `init_chat_model`

Anthropic via `init_chat_model`

TypeScript

TypeScript `ChatOpenAI`

TypeScript `ChatAnthropic`

Custom OpenAI-compatible providers

Python custom provider via `ChatOpenAI`

TypeScript custom provider via `ChatOpenAI`

Task IDs

Python task IDs

Python `ChatOpenAI` and `ChatAnthropic`

TypeScript task IDs

TypeScript `ChatAnthropic`

TypeScript `ChatOpenAI`

Why this matters

Framework-owned invocation loops

Python fallback

TypeScript fallback