Skip to main content
Catalyst Gateway is a transparent proxy that sits between your app and your LLM provider. Point your SDK at https://api.inference.net/v1, add a couple of headers, and every request is captured with cost, latency, and full request/response payloads. You keep your existing provider API keys. Gateway adds roughly 10ms of overhead and forwards requests as-is. Use Gateway for inference observability and training. Once traffic is flowing, you can:
Gateway and Catalyst Tracing are independent. Gateway captures one record per LLM request through a proxy. Tracing captures the full agent hierarchy from inside your code. Use them on their own or together. For traces, see the Tracing overview.

​
Getting Started

The fastest path is the Inference CLI:
gateway mode points your existing LLM clients at the Catalyst Gateway. both mode does that plus installs the tracing SDK in one pass. The command launches your choice of coding agent (Claude Code, OpenCode, or Codex) to make the edits. See Install with AI for the full flow. Prefer to wire it up by hand? Start with the Gateway quickstart or pick your provider below.

​
Officially Supported Providers

These providers have dedicated guides with copy-paste setup. Any other OpenAI-compatible provider works too via the x-inference-provider-url header. See supported OpenAI-compatible providers below.
  • OpenAI: Chat Completions and Responses API, including tool calls and structured outputs.
  • Anthropic: Messages API with tool use, prompt caching, and streaming.
  • Vertex AI: Google Cloud Vertex AI for Gemini and Anthropic models on GCP.
  • Amazon Bedrock: Bedrock OpenAI-compatible Chat Completions through AWS-hosted models.
  • Google Gemini: Native Gemini API and the OpenAI-compatible Gemini endpoint.
  • OpenRouter: Route across many models through OpenRouter’s OpenAI-compatible API.
  • Cerebras: Cerebras inference with full request capture.
  • Groq: Groq’s low-latency inference endpoint.
  • LangChain: Use Gateway from LangChain by setting the base URL on the chat model.
  • Vercel AI SDK: Route the ai package through Gateway with the OpenAI-compatible provider.
  • ElevenLabs: Route ElevenLabs Agents’ LLM calls through Catalyst.

​
Routing Headers

​
Supported OpenAI-compatible Provider URLs

Any OpenAI-compatible provider can be used via the x-inference-provider-url header, even when it does not have a dedicated guide in the catalog yet. For Bedrock Runtime’s OpenAI-compatible endpoint, use https://bedrock-runtime.{region}.amazonaws.com/v1 instead of the Bedrock Mantle host. For regional Vertex AI endpoints, use https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}/endpoints/openapi instead of the global host.

​
What Gets Captured

Once traffic is flowing through Gateway, Catalyst records:
  • The full request and response payloads
  • Cost per call and aggregate spend
  • Latency, including time to first token (TTFT) and tokens per second
  • Token counts (input and output)
  • Error rates and status codes
  • Model and provider
These metrics are captured automatically without any changes to your request logic. TTFT and tokens-per-second are specifically the metrics that are invisible to application code; only a proxy can measure them.

​
Next Steps

Quickstart

Install with AI or wire it up by hand and capture your first LLM call.

Record your first LLM call

The higher-level Get Started flow with the same setup paths.

Metrics Explorer

Watch cost, latency, errors, and token usage across all your calls.

Tasks

Group LLM calls by feature or objective with x-inference-task-id.

Build a dataset

Turn captured traffic into datasets for evals and training.

Run an eval

Compare models side by side on your captured traffic.