Bifrost

Bifrost is an open-source LLM gateway. If your app already sends its LLM traffic through Bifrost, add the Inference Gateway as a Bifrost provider. Your app keeps calling Bifrost. Bifrost forwards the requests to https://api.inference.net, and the gateway captures each one with cost, latency, and the full request and response.

You can use this setup in two ways:

• Call models with your Inference.net credits. Only your Inference API key is needed. This works for open-source models such as glm-5.2 and for popular closed-source models such as gpt-5-mini. See inference.net/models.
• Bring your own provider key. Keep paying your provider directly and use the gateway for capture only.

Call models with your credits

export INFERENCE_API_KEY=<your-project-api-key>

{
  "$schema": "https://www.getbifrost.ai/schema",
  "providers": {
    "inference": {
      "keys": [
        {
          "name": "inference-key",
          "value": "env.INFERENCE_API_KEY",
          "models": ["*"],
          "weight": 1.0
        }
      ],
      "network_config": {
        "base_url": "https://api.inference.net"
      },
      "custom_provider_config": {
        "base_provider_type": "openai"
      }
    }
  }
}

docker run -p 8080:8080 \
  -v "$(pwd):/app/data" \
  -e INFERENCE_API_KEY \
  maximhq/bifrost

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080/v1",
  apiKey: "bifrost",
});

const response = await client.chat.completions.create({
  model: "inference/glm-5.2",
  messages: [{ role: "user", content: "Hello, world!" }],
});

console.log(response.choices[0].message.content);

Tag requests

Bifrost forwards any request header that starts with x-bf-eh- and removes the prefix. Use this to send the gateway headers, such as x-inference-task-id and x-inference-environment, from your app.

const response = await client.chat.completions.create(
  {
    model: "inference/glm-5.2",
    messages: [{ role: "user", content: "Summarize this ticket." }],
  },
  {
    headers: {
      "x-bf-eh-x-inference-task-id": "summarize-ticket",
      "x-bf-eh-x-inference-environment": "production",
    },
  },
);

To set the same tags on every request, add them to network_config.extra_headers in config.json. A header sent with the request replaces the extra_headers value.

"network_config": {
  "base_url": "https://api.inference.net",
  "extra_headers": {
    "x-inference-environment": "production"
  }
}

Bring your own provider key

To keep paying your provider directly, send your provider API key in the x-inference-provider-api-key header. The gateway forwards the request to the provider with that key. The example below uses OpenAI with the same inference provider from config.json.

const response = await client.chat.completions.create(
  {
    model: "inference/gpt-4.1-mini",
    messages: [{ role: "user", content: "Hello, world!" }],
  },
  {
    headers: {
      "x-bf-eh-x-inference-provider-api-key": process.env.OPENAI_API_KEY,
      "x-bf-eh-x-inference-provider": "openai",
    },
  },
);

For a provider other than OpenAI, use its OpenAI-compatible base URL in x-bf-eh-x-inference-provider-url instead of x-bf-eh-x-inference-provider. See the supported provider URLs.

Next steps