Deploy
Deploy
Dedicated GPU infrastructure for serving models via an OpenAI-compatible API.
Deploy gives you a dedicated GPU serving an open-source catalog model or a Hugging Face model you registered. The API is OpenAI-compatible, so switching from a serverless model to your deployment is a one-line code change.
Key concepts
| Concept | Description |
|---|---|
| Dedicated GPU | Your model runs on its own GPU. No shared infrastructure, no noisy neighbors. Instance type and count are chosen at deploy time from the model's allowed instance types; the default is 1×H100. |
| OpenAI-compatible API | Same base URL, same API key, just swap the model parameter. Structured outputs, function calling, and all standard API features work the same way. |
| Project-scoped | Deployments are created within a project, and access is scoped to that project. Sibling projects in the same team cannot call them. The model path is your team/org slug followed by the deployment name you choose (e.g. acme-corp/my-model). |
| Monitoring | Route deployment traffic through Gateway to observe production performance, and run evals to catch regressions. |
What you can deploy
- Open source catalog models with weights available
- Hugging Face models you've registered as custom models
- Served via an OpenAI-compatible API (chat completions endpoint)
- Same base URL and API key as the rest of the Inference API