Deployments

Model deployments serve a fine-tuned or catalog model within a project, backed by allocated GPU instances. Use inf deployment to create a deployment, inspect everything about it now or at a past moment, and manage its lifecycle — all without opening the dashboard.

Alias: inf deployments

Quickstart

# See what a model may be deployed on (GPU families and counts)
inf deployment allowed-instance-types --model-id <model-id>

# Create a deployment and watch the rollout converge
inf deployment create \
  --name my-deployment \
  --model-id <model-id> \
  --public-model-identifier <team>/<model-slug>
inf deployment watch <deployment-id>

# Inspect the deployment and the engine config it serves with
inf deployment get <deployment-id>
inf deployment config <deployment-id>

# Stop serving (reversible) or archive to free a slot
inf deployment stop <deployment-id> --yes
inf deployment start <deployment-id>

Endpoint settings

A deployment serves an endpoint: the public model identifier callers use. The endpoint owns the model, the serverless pricing, the request-routing timeouts, the aliases, and the allowlist of extra teams. Every deployment serving it shares them, so they are changed on the endpoint, not the deployment. A deployment's endpointId names its endpoint.

inf endpoint get --id <endpoint-id>
inf endpoint update --id <endpoint-id> --model-id <model-id>

Only --model-id is open to every team. The other endpoint fields require super-admin access. Changing the model rolls every deployment on the endpoint. See Endpoints for every flag.

Super-admin pricing rules

Super-admins can set scheduled effective prices for a public serverless endpoint. Prices are entered in USD per 1M tokens and stored as fixed per-token rates. Omitting --customer-team creates an all-team rule; a matching team rule takes precedence over an all-team rule.

inf admin endpoints pricing-rules list <endpoint-id>

inf admin endpoints pricing-rules create <endpoint-id> \
  --customer-team <team-id> \
  --input 1.05 \
  --cached-input 0.115 \
  --output 5.475 \
  --starts-at 2026-09-20T00:00:00Z \
  --ends-at 2026-09-27T00:00:00Z \
  --reason "Partner launch pricing"

inf admin endpoints pricing-rules remove <endpoint-id> \
  --rule <rule-id> \
  --revert-to-base-price \
  --yes

To change a rule's price, create the replacement rule rather than removing the current one: the new rule ends the old one at the moment it starts, so there is no gap. Removing the only rule in effect for its scope bills that scope at the endpoint's standard price from that moment, so it requires --revert-to-base-price. A team rule is covered by an all-teams rule in effect: removing it bills the team at the all-teams rule, with no flag needed.

--cached-input defaults to the input price. Disabled, future, and expired rules do not apply. The endpoint's standard prices remain the fallback.

Each scope (one team, or all teams) has at most one enabled rule at a time. create applies these steps:

1. Rules for the same scope that started earlier end where the new rule starts.
2. With --ends-at, the rule that was in effect resumes after the new rule ends, as a new rule with the same prices. It stops at its own end or where the next scheduled rule starts.
3. If a rule for the same scope is scheduled to start inside the new rule's window, the save is refused. Remove that rule first, or end the new rule before it starts.
4. Saving the same price again changes nothing.

The command prints the rules it ended and resumed, or that nothing changed.

Super-admin pricing history

Every change to an endpoint's standard prices and to its pricing rules is recorded with who made it. endpoints lists standard-price changes; rules lists pricing-rule creates, edits, ends, and deletes. Both are newest first, paginated with --limit/--offset, and filtered with --search (model identifier; rules also matches team name or slug).

inf admin pricing-history endpoints --search kimi

inf admin pricing-history rules \
  --search acme \
  --sort '{"key":"changedAt","direction":"asc"}'

Rates are returned in nanocredits per 1M tokens. The same history is on the dashboard at Super Admin → Pricing History.

Super-admin model configs

inf admin model-configs (alias engine-configs) manages the engine configs (vLLM, SGLang, and so on) that deployments serve with. Every update creates a new immutable version, and history lists them all.

inf admin model-configs list --model <model-id>
inf admin model-configs create --model <model-id> --engine vllm \
  --card h100=8 --parallelism 8 --flag max-model-len=32768
inf admin model-configs update <config-id> --flag max-model-len=65536
inf admin model-configs get <config-id> --version 3
inf admin model-configs history <config-id>

Config notes

Notes are markdown documents attached to a whole config, or to one version with --version. Leave out --version to work on the config-wide note. Use notes to record how a config performed, with links to benchmarks or dashboards. Notes are insert-only: every note add or note edit saves a new revision, and older revisions stay readable in note list, note show --revision, and the dashboard's Config History panel. Each revision records its author's email and whether it was written with a session or an API key.

# Config-wide note, inline
inf admin model-configs note add <config-id> "Stable at 64 concurrent requests."

# Note on version 3, read from a file (--body-file - reads stdin)
inf admin model-configs note add <config-id> --version 3 --body-file notes.md

# Pipe benchmark output in, appended to the current revision after a blank line
./bench.sh | inf admin model-configs note add <config-id> --version 3 --append

# Edit the current revision in $VISUAL / $EDITOR (default vi); saves only if changed
inf admin model-configs note edit <config-id> --version 3

# Print the current (or a given) revision's raw markdown; author and time go to stderr
inf admin model-configs note show <config-id> --version 3 > notes.md
inf admin model-configs note show <config-id> --version 3 --revision 1

# Revisions with author, auth path, and a preview (all notes, one version, or config-wide only)
inf admin model-configs note list <config-id>
inf admin model-configs note list <config-id> --version 3
inf admin model-configs note list <config-id> --config-wide

note add takes the body from exactly one of: the argument, --body, --body-file, or piped stdin when none of those is given. Empty or whitespace-only bodies are rejected. note show and note list print the note objects as JSON with the global --json flag. The same operations are available to agents through the modelAdmin.addConfigNote and modelAdmin.getConfigNotes API procedures and their MCP tools.

inf deployment list

List deployments for a team with their status, lifecycle, and served model alias.

inf deployment list

Alias: inf deployment ls

Options

Shows the deployment ID (8-char prefix), name, status, lifecycle state, public model identifier, and creation date.

inf deployment get

Show a single deployment's full details — status, lifecycle, model alias, and timestamps. Omit the id to pick interactively.

inf deployment get <id>

Arguments

Options

inf deployment config

Show the effective engine configuration a deployment serves with: the resolved flags, environment variables, and which layer set each.

inf deployment config <id>

Arguments

Options

inf deployment describe

Show everything you can read about a deployment in one view:

• The deployment's settings and config hash.
• The effective engine config. Each flag and env var shows the layer that set it (engine default, engine config, or deployment override) and the value it replaced.
• The endpoint's settings and its resolved list price.
• Routing: the deployment's rank, whether it is the default, its rule as CEL, and every other deployment on the endpoint.
• Instances: counts by status, how many run the current config hash, ready shards, and scheduling failures.
• Recent events.
• The live routing load: in-flight requests against routing slots across the deployment's shards, as the engine reports them now. This needs super-admin.
• The control towers the deployment may be placed on: status, health, GPU counts, and position in the fleet placement order. This needs super-admin.
• Pricing rules, what each team scope pays now, aliases, and margin. These need super-admin.

Sections that need super-admin read {"unavailable":"requires super-admin"} for other callers.

inf deployment describe <id>

# One JSON document for scripts and agents
inf --json deployment describe <id> | jq '.engineConfig.attribution.flags[] | select(.source=="override")'

Alias: inf deployment inspect

Options

--help lists what every field means and which command changes it.

inf deployment history

Show what changed on a deployment and its endpoint, and when:

• configEras: the time ranges when each config hash was the desired config. Instances created in a range ran that config. To see which config a metric came from, match the metric's timestamps to these ranges.
• timeline: deployment and endpoint events, newest first, with a summary of each. Config, settings, and endpoint changes record their before and after values. Routing changes record every deployment's rank and rule before and after.
• instances: instances that were live in the window or created in it.
• instanceEngineConfigs (--instance-configs): instances grouped by the exact engine config stored on each when it was created. A deployment without a pinned engine config picks up new config versions without a deployment event, so this is the record of what actually ran.
• stateAt (--at): the deployment's config, settings, desired instance count, lifecycle, and accepted control towers at that moment, with the endpoint's settings and routing. It starts from the current state and undoes every recorded change after --at.
• engineConfigHistory: the engine config's versions. This needs super-admin.
• pricingHistory: changes in the window to the endpoint's list price, its pricing rules, and this deployment's GPU-hour pricing (billing exemption, GPU-hour override). With --at, pricesAt shows the list price and the rule each team paid at that moment. This needs super-admin.

inf deployment history <id> --since 24h
inf deployment history <id> --at 2026-10-01T14:05:00Z
inf deployment history <id> --type endpoint.routing_changed,deployment.config_changed
inf --json deployment history <id> --since 14d --instance-configs

Options

The history covers only recorded events. A change made before event recording began, or outside the API, does not appear.

inf deployment instances

List a deployment's running and terminated instances with their status, config hash, and stop reason.

inf deployment instances <id>

Alias: inf deployment inst

Arguments

Options

inf deployment watch

Poll a deployment until its rollout converges — status Active and the target config hash Running.

inf deployment watch <id>

Arguments

Options

Exits 0 on convergence; fails if the timeout elapses.

inf deployment stop / start

stop archives a deployment so it stops serving (alias of archive). start restarts a previously archived deployment so it resumes serving. Both are reversible.

inf deployment stop <id> --yes
inf deployment start <id>

Options

stop prompts for confirmation in an interactive terminal; in non-TTY environments (CI, scripts) it refuses to run without -y.

inf deployment archive

Archive a deployment so it stops serving and frees a slot against the team cap. Reversible with start.

inf deployment archive <id> --yes

Arguments

Options

inf deployment create

Create a deployment with its full configuration.

inf deployment create \
  --name <name> \
  --model-id <model-id> \
  --public-model-identifier <team>/<model-slug>

Options

Pass exactly one of --endpoint-id or --public-model-identifier.

These three timeout flags require super-admin access. After creation they belong to the endpoint: change them with inf endpoint update --id <endpoint-id>. Pass null to restore the inherited default.

--accepted-control-tower-ids also requires super-admin access and takes a comma-separated list. An empty list means any eligible tower, which is the default. The set constrains where new instances are placed: instances already running are never moved when it changes. The super-admin CLI spells the same setting as a repeatable inf admin deployment create|update --accepted-control-tower <id>, with --clear-control-towers to empty it.

Among the towers a deployment accepts, super admins can also set a fleet-wide placement preference: an ordered list of control towers that new instances fill first. Among towers of equal standing, a new instance lands on the first listed tower with free GPUs for it, and an unlisted tower is used only when no listed tower has room. Some rules still rank above the list: a team's own towers, in-cluster towers, US-EAST towers with free GPUs, and, for a replacement, the tower of the instance it replaces. A list of US-EAST towers therefore fills in list order. Running instances are never moved.

inf admin control-towers preference get
inf admin control-towers preference set cw-us-east-13a cw-us-east-15a cw-h200-cluster
inf admin control-towers preference clear

set takes control tower IDs or unique names, best first, and replaces the whole list. clear removes the preference.

Prints the new deployment ID. Run inf deployment watch <deployment-id> to track its rollout.

Examples

# Deploy on 1 GPU instance and watch the rollout converge
inf deployment create \
  --name support-model \
  --model-id <model-id> \
  --public-model-identifier acme/support-v1 \
  --desired-instance-count 1
inf deployment watch <deployment-id>

# Override an engine flag to lower max tokens served
inf deployment create \
  --name support-model \
  --model-id <model-id> \
  --public-model-identifier acme/support-v1 \
  --model-config-flag-overrides '{"max_tokens": 4096}'

Generated REST verbs

inf deployment also exposes the full deployment API surface as generated commands — versions, update, events (one page of the raw event timeline; --since / --until bound it), engine-config, engine-configs, inferences, inference-filter-options, public-identifier-available, training-job-for-model, open-source-models, and allowed-instance-types. Run inf deployment --help to list them and their flags.