Skip to main content
The Inference CLI is the fastest way to connect your app to the Inference platform. It scans your codebase, finds your LLM clients, and either routes them through the gateway, adds trace collection, or both — guided by an AI coding agent (Claude Code, OpenCode, or Codex). Want your AI coding assistant to query Inference platform resources directly? Configure the MCP server.
Install with AI works with OpenAI, Anthropic, Amazon Bedrock, Gemini, Vertex AI, Groq, Cerebras, OpenRouter, LangChain and more.

​
What it sets up

When you run inf instrument, you’ll be asked which Inference platform product to set up: Pass --mode gateway, --mode tracing, or --mode both to skip the prompt (useful in CI or scripted runs).
1

Install the CLI

Install the Inference CLI globally.
2

Sign in

Sign in with your Inference account. Your browser will open to authenticate.
Running in CI or another headless environment? Use inf auth set-key instead of browser login.
3

Run instrumentation in your project

Navigate to your project root and run instrumentation.
The command guides you through the following workflow:
  • Pick the project to instrument into. If you don’t have one, choose Create a new project and name it — the CLI creates it on the spot.
  • Pick what to instrument: gateway, tracing, or both.
  • Choose whether to create a new API key for the project.
    • Yes: the CLI mints a project-scoped key and writes it to .env as INFERENCE_API_KEY.
    • No: the CLI writes INFERENCE_API_KEY=PASTE_YOUR_API_KEY_HERE to .env and prints the dashboard link where you can create a key yourself. Replace the placeholder with the real key before running your app. If .env already has a real key, the CLI leaves it unchanged.
  • Select a coding agent to use: Claude Code, OpenCode, or Codex.
  • Scan your codebase for LLM clients such as OpenAI, Anthropic, LangChain, etc.
  • For gateway: redirect base URLs to the proxy, add routing and task ID headers.
  • For tracing: install the Inference Tracing SDK, initialize it at the app entrypoint, and add spans around agents, tools, and provider calls.
  • Review the generated changes before applying them.
Run inf instrument --dry-run to preview changes without modifying any files.
4

Run your app

Run your application how you normally would to produce inference requests. Requests from your application are now routed through the gateway and will appear in the dashboard.
5

Verify it worked

Open the dashboard to see request details, traces, and analytics. You can also verify from the CLI:
The CLI writes INFERENCE_API_KEY to .env for you during inf instrument. If you skipped key creation, create a key in the dashboard under Settings → API Keys and replace PASTE_YOUR_API_KEY_HERE in .env. Keep .env out of git — add it to .gitignore.

​
Supported AI coding agents

​
Supported providers

Built-in: OpenAI, Anthropic OpenAI-compatible via x-inference-provider-url: Amazon Bedrock, Google Gemini, Vertex AI, Together AI, Groq, Fireworks AI, Mistral AI, Cerebras, Perplexity, DeepSeek, OpenRouter, Azure OpenAI, and any OpenAI-compatible endpoint. Native provider APIs: Vertex AI native Gemini and Anthropic-on-Vertex are supported through the manual gateway headers documented in the Vertex AI guide.