Deploy
Deploy
Dedicated GPU infrastructure for serving trained models via an OpenAI-compatible API.
Deploy gives you a dedicated GPU serving your fine-tuned model. The API is OpenAI-compatible, so switching from an off-the-shelf model to your custom model is a one-line code change. This is the last step in the loop and the beginning of the next one.
Key concepts
| Concept | Description |
|---|---|
| Dedicated GPU | Your model runs on its own GPU. No shared infrastructure, no noisy neighbors. Instance type and count are chosen at deploy time from the model's allowed instance types; the default is 1×H100. |
| OpenAI-compatible API | Same base URL, same API key, just swap the model parameter. Structured outputs, function calling, and all standard API features work the same way. |
| Project-scoped | Deployments are created within a project, and access is scoped to that project. Sibling projects in the same team cannot call them. The model path is your team/org slug followed by the deployment name you choose (e.g. acme-corp/my-model). |
| The improvement loop | Deploy → observe production performance → run evals to catch regressions → train the next version. The loop continues. |
What you can deploy
- Models trained on the Inference platform
- Open source catalog models with weights available
- Hugging Face models you've registered as custom models
- Served via an OpenAI-compatible API (chat completions endpoint)
- Same base URL and API key as the rest of the Inference API