Endpoints

An endpoint is the public model identifier callers use. One or more deployments serve it. The endpoint owns the model, the serverless pricing, the request-routing settings, the aliases, and the allowlist of extra teams, so these change on the endpoint and apply to every deployment serving it. A deployment's endpointId (from inf deployment get) names its endpoint.

Alias: inf endpoints

inf endpoint get

Get an endpoint's settings and allowed teams.

inf endpoint get --id <endpoint-id>

inf endpoint update

Update an endpoint's settings. Omitted flags leave a setting unchanged; pass null to restore a nullable setting's inherited default.

# Switch the model every deployment on the endpoint serves
inf endpoint update --id <endpoint-id> --model-id <model-id>

# Disable queueing and cap the whole request at 10 minutes (super-admin)
inf endpoint update --id <endpoint-id> \
  --queue-timeout-ms -1 \
  --max-queue-size -1 \
  --total-req-timeout-ms 600000

Changing the model rolls every deployment on the endpoint. It resets each deployment's model-config pin, parallelism, and shard overrides.

Reads and endpoint events return a stored hf_token as ********. Sending ******** back in --tokenizer-options keeps the stored token; leaving hf_token out removes it.

inf endpoint update --id <endpoint-id> \
  --tokenizer-kind HuggingFaceTokenizers \
  --tokenizer-options '{"repo":"Qwen/Qwen3-8B","revision":"<commit-sha>"}'

inf endpoint list

List the team's endpoints. Each row shows the endpoint's deployments with their rank and rule, the default deployment, and how many deployments are running. The list is built from deployments, so an endpoint whose deployments are all archived does not appear.

inf endpoint list
inf endpoint list --project <project-id> --stopped

inf endpoint describe

Show the endpoint's settings and resolved list price, its routingVersion, then every deployment in routing order (p<priority>, * marks the default). Each deployment row shows its rule as CEL, engine and config version, flag overrides, GPUs, instance counts, and live routing load. Pricing rules, what each team scope pays now, and aliases need super-admin.

inf endpoint describe <endpoint-id>
inf --json endpoint describe <endpoint-id> | jq '.deployments[] | {priority, name, ruleText}'
inf endpoint describe <endpoint-id> --status Active,Degraded

inf endpoint instances

Show every deployment on the endpoint and its instances, grouped by the exact engine config frozen onto each instance when it was created. For each group the command shows:

• the config id and version, engine, parallelism, and L3 KV backend
• the launch command the worker builds, with flags in the order the worker passes them. Values that only the pod knows (the weights snapshot path, the served model name, the Mooncake client config) are <placeholders> marked # worker.
• the env vars (secrets masked)
• each instance's ID, status, configHash, and whether it is on the deployment's desired config

The last section diffs each deployment's desired config, and each instance group, against the default deployment's desired config: flags and env added, removed, or changed. By default the command skips Stopped deployments and shows only live instances. Archived deployments never appear.

inf endpoint instances <endpoint-id>
inf endpoint instances <endpoint-id> --stopped --terminated
inf --json endpoint instances <endpoint-id> | jq '.diffs[] | select(.identical | not)'

inf endpoint history

Show how the endpoint and all its current deployments changed:

• routingEras: every deployment's priority and rule over time. Each era names who started it (startedBy: user, clientTag, reason, routingVersion) and the deployments it changed (changes: priority and rule before and after).
• configEras: each deployment's config hash over time.
• timeline: one list of endpoint and deployment events.
• pricingHistory: list-price and pricing-rule changes. This needs super-admin.

--at shows the endpoint and every deployment as they were at that moment. It takes --team, --since, --until, --at, --type, --actor, and --max-events, with the same meanings as in inf deployment history. For --instance, --instance-configs, or --no-endpoint-events, run inf deployment history on one deployment.

inf endpoint history <endpoint-id> --since 3d
inf --json endpoint history <endpoint-id> --at 2026-10-01T14:05:00Z | jq '.stateAt'

inf endpoint pricing

Show the endpoint's pricing:

• The list price: what billing charges when no pricing rule applies.
• The current pricing rules.
• What each team scope pays now. Team null is every team without its own rule.
• Every list-price and rule change in the window.
• With --at, the list price and the rule each team paid at that moment.

Rules, current prices per scope, and history need super-admin. Other callers see only the list price.

inf endpoint pricing <endpoint-id>
inf endpoint pricing <endpoint-id> --since 30d --for-team <team-id>
inf --json endpoint pricing <endpoint-id> --at 2026-10-01T14:05:00Z | jq '.history.pricesAt'

Alias: inf endpoint prices

inf endpoint events

Read one page of the endpoint's raw events, newest first:

• endpoint.changed records { changed, before, after } for settings and list price.
• endpoint.routing_changed records every deployment's { deploymentId, rank, rule } before and after (rank is the priority), plus operation (replace, set, clear), fromVersion / toVersion, reason, clientTag and apiKeyId. Events from before 2026-10-07 have only before and after.
• The other event types are endpoint.created, endpoint.deployment_added, endpoint.alias_added, and endpoint.alias_removed.

The same read is available as REST POST /endpoints/{endpointId}/events and the MCP tool deployment_get_endpoint_events.

inf endpoint events --endpoint-id <endpoint-id> --event-type endpoint.routing_changed --since 2026-10-01T00:00:00Z

Super-admin commands

inf admin endpoints adds admin conveniences over the same settings:

# Serverless pricing in dollars per 1M tokens, and the allowlist
inf admin endpoints update <endpoint-id> \
  --serverless \
  --serverless-cost-input 1.30 \
  --serverless-cost-output 4.40 \
  --allowed-team <team-id>

# Make the endpoint dedicated again (clears its serverless rates)
inf admin endpoints update <endpoint-id> --private

It also owns the endpoint's aliases (generated add-alias, remove-alias, aliases, and alias-availability) and its pricing rules.

Routing: priorities, versions, and safe concurrent edits

Each deployment on an endpoint has a routing priority (routingRank on deployment reads) and an optional rule.

• Priority: an integer; lower wins. Priorities are unique within an endpoint. The engine evaluates deployments in priority order. The default is the deployment with the lowest priority. null = no priority; when every deployment has none, the routing is off.
• Gaps: a full-list save assigns 10, 20, 30, …, so you can move one deployment between two others (e.g. 15) without renumbering. Endpoints saved before gaps existed keep 1, 2, 3, … until their next full-list save.
• routingVersion: inf endpoint describe and inf endpoint get show it. Every write names the version it read. If the routing changed since, the write fails with CONFLICT, saves nothing, and the error shows the current routing and version. Re-read, re-apply your change, retry.
• Audit: every write records who made it, the version before and after, and your optional --reason and --client-tag (script name, agent session) on the endpoint.routing_changed event. inf endpoint history shows the per-deployment diff.

inf endpoint routing set

Changes one deployment's rule, overflow rule and/or priority. Every other deployment keeps its own, so two people editing different deployments never overwrite each other.

inf endpoint routing set <endpoint-id> --deployment <dpl> \
  --rule 'request.input_token_count + request.num_images * 2000 > 150000' \
  --reason 'long-prompt pool' --client-tag 'agent kimi-canary'
inf endpoint routing set <endpoint-id> --deployment <dpl> --priority 15
inf endpoint routing set <endpoint-id> --deployment <dpl> --rule false   # park it
inf endpoint routing set <endpoint-id> --deployment <dpl> --rule any     # no rule
inf endpoint routing set <endpoint-id> --deployment <dpl> \
  --overflow-rule 'request.is_openrouter || target.free_slot_pct >= 30'

The same write is REST POST /endpoints/set-deployment-routing and generated inf admin endpoints set-deployment-routing.

inf admin endpoints update-routing

Replaces the whole order: a priority per deployment, an optional rule per deployment, and the default. Use it to reorder many deployments at once; use inf endpoint routing set for one.

1. A deployment is eligible when its rule is true. A null rule is always eligible. A rule is false when it divides or takes a modulo by zero, or reads a variable the engine can't see for that request (see below). A deployment with no running instances is never eligible.
2. The request may use every eligible deployment and takes any free capacity among them, whatever their priority. When all of them are full, it queues (or gets a 429); it never uses an ineligible deployment, unless the endpoint sets --enable-routing-overflow true (see inf endpoint update).
3. When no deployment is eligible, the default (lowest priority) serves it, whatever its rule says.
4. Engines reload routing at most every 5 minutes, so a save takes effect within 5 minutes. An endpoint whose routing was never saved spreads requests across its deployments by free capacity.

A rule is JSON, not CEL: { "version": 1, "expr": <BoolExpr> }.

• BoolExpr: bool (value), bool_var (name), str_in (name, values: 1 to 32 strings; true when the variable equals one), eq / neq / lt / lte / gt / gte (left, right: NumExpr), and / or (args: one or more BoolExpr), not (arg).
• NumExpr: num (value), var (name), add / sub / mul / div / mod (left, right).
• Variables:

Requests above the size limit for inline bodies reach the engine as a presigned URL, so any rule that reads a body-derived variable is false for them and they fall back to the default.

• Limits: 200 nodes, 20 levels of nesting.

# Prompts up to 256k tokens may use either deployment; larger ones only dep_large
inf admin endpoints update-routing --endpoint-id <endpoint-id> \
  --expected-routing-version "$(inf --json endpoint get --id <endpoint-id> | jq -r .routingVersion)" \
  --deployments '[
  {"deploymentId": "dep_small", "rule": {"version": 1, "expr": {
    "op": "lte",
    "left": {"op": "var", "name": "request.input_token_count"},
    "right": {"op": "num", "value": 262144}}}},
  {"deploymentId": "dep_large", "rule": null}
]'

The dashboard's endpoint page shows and edits the same rules as CEL, e.g. request.input_token_count <= 262144.

inf admin endpoints clear-routing

Turns an endpoint's routing off: clears every priority, rule and the default, so requests spread across all its deployments by free capacity again (within 5 minutes). The endpoint page's "Reset routing" button does the same.

inf admin endpoints clear-routing --endpoint-id <endpoint-id> --expected-routing-version <routingVersion>

Related commands

• inf deployment create --endpoint-id <id> adds another deployment to an endpoint.
• inf deployment update changes one deployment's own settings: name, GPUs, instances, engine overrides.