Signals

Create LLM classifiers that label spans, traces, or sessions — then inspect labels, distributions, and alerts.

A signal is an LLM classifier that labels a sampled fraction of an agent's spans, traces, or sessions. Use signals to track behavior you care about — refusals, sentiment, error categories — and alert when the label mix drifts.

Use inf signals to define classifiers, backfill labels over a past window, inspect the produced labels and their distribution over time, and manage threshold alerts on them.

How signals work

Each signal belongs to one agent (its UUID, not its identity value) and targets one scope: span, trace, or session.
A signal has a classifier: binary (true/false) or string (picks one of 2–10 --label values).
The classifier samples a percentage of matching items (--sample-rate, integer 0–100) and runs only while the signal is active (activate / disable toggle it).
Editing creates a new version of the signal; the current version drives classification.

Quickstart

# 1. Find the agent that owns the signal
inf agent list

# 2. Create a binary classifier sampling 25% of spans
inf signals create \
  --agent-id <agent-uuid> \
  --name "Refusals" \
  --scope span \
  --classifier binary \
  --sample-rate 25 \
  --prompt "Did the assistant refuse the request?"

# 3. Verify the prompt labels a past window correctly before going live
inf signals verify <signal-id> \
  --from 2026-06-01T00:00:00Z \
  --to 2026-06-02T00:00:00Z

# 4. See the signals for an agent and inspect one
inf signals list --agent my-agent
inf signals get <signal-id>

inf signals list prints full signal IDs. Use --json for machine-readable output.

`inf signals list`

List all signals defined for a single agent identity, with classifier type, scope, sample rate, and recent label counts.

inf signals list --agent my-agent

Alias: inf signals ls

Options

Flag	Required	Description	Default
`--agent <identity>`	Yes	Agent identity value to list signals for, e.g. `my-agent`	—
`--include-archived`	No	Also show archived (soft-deleted) signals	Off

`inf signals list-by-project`

List every signal across all agents in a project, as a lightweight summary with no version detail. Use when you don't know which agent owns a signal.

inf signals list-by-project

Options

Flag	Required	Description	Default
`--project <id>`	No	Project UUID to scan	Active project

`inf signals get`

Show one signal's full config plus its current version — classifier prompt, labels, and sample rate.

inf signals get <signal-id>

Arguments

Argument	Required	Description
`signalId`	Yes	Signal UUID (from `inf signals list`)

`inf signals create`

Create a signal. Pass exactly one of --prompt or --prompt-file.

inf signals create \
  --agent-id <agent-uuid> \
  --name "Sentiment" \
  --scope trace \
  --classifier string \
  --sample-rate 10 \
  --label positive,neutral,negative \
  --prompt-file ./sentiment.txt

Options

Flag	Required	Description	Default
`--agent-id <id>`	Yes	Agent UUID that owns this signal (the agent's id, not its identity value)	—
`--name <name>`	Yes	Human-readable signal name, e.g. "Refusals"	—
`--scope <scope>`	Yes	What each classifier run labels: `span`, `trace`, or `session`	—
`--classifier <type>`	Yes	`binary` (true/false, no `--label`) or `string` (2–10 `--label` values)	—
`--sample-rate <percent>`	Yes	Percent of matching items to classify (integer 0–100)	—
`--prompt <text>`	Exactly one of prompt/prompt-file	Classifier prompt text inline	—
`--prompt-file <path>`	Exactly one of prompt/prompt-file	Read the classifier prompt from a file, e.g. `./refusal-prompt.txt`	—
`--label <label>`	For `string` classifiers	Label value; repeat 2–10 times or comma-separated	`[]`
`--sentiment <outcome=value>`	No	Outcome sentiment, e.g. `true=negative` or `<label>=positive`	`[]`

Examples

# Binary classifier (no labels) sampling 25% of spans
inf signals create --agent-id <agent-uuid> --name "Refusals" \
  --scope span --classifier binary --sample-rate 25 \
  --prompt "Did the assistant refuse the request?"

# String classifier with a fixed label set, prompt read from a file
inf signals create --agent-id <agent-uuid> --name "Sentiment" \
  --scope trace --classifier string --sample-rate 10 \
  --label positive,neutral,negative --prompt-file ./sentiment.txt

`inf signals edit`

Create a new version of a signal by overriding any of its config fields; unspecified fields keep their current values. Pass at most one of --prompt or --prompt-file.

inf signals edit <signal-id> --sample-rate 50

Arguments

Argument	Required	Description
`signalId`	Yes	Signal UUID (from `inf signals list`)

Options

Flag	Required	Description	Default
`--scope <scope>`	No	New scope: `span`, `trace`, or `session`	Keep current
`--classifier <type>`	No	New classifier type: `binary` or `string`	Keep current
`--sample-rate <percent>`	No	New sample rate (integer 0–100)	Keep current
`--prompt <text>`	No	New classifier prompt text inline	Keep current
`--prompt-file <path>`	No	Read the new classifier prompt from a file	Keep current
`--label <labels>`	No	Replace the string classifier label set entirely, e.g. `good,bad`	Keep current
`--sentiment <outcome=value>`	No	Outcome sentiment, e.g. `true=negative`	Keep current

Examples

# Bump just the sample rate
inf signals edit <signal-id> --sample-rate 50

# Rewrite the prompt and swap the label set
inf signals edit <signal-id> --prompt-file ./v2-prompt.txt --label positive,negative

`inf signals activate` / `disable` / `archive`

Manage a signal's state.

# Resume classifying new sampled items
inf signals activate <signal-id>

# Pause classification (reversible)
inf signals disable <signal-id>

# Archive (soft-delete); hidden from `list` unless --include-archived
inf signals archive <signal-id>

Each takes a single required signalId argument (UUID from inf signals list) and no other options.

`inf signals verify`

Preview how a signal's current classifier would label a small sample from a past window, without persisting results. Use to sanity-check a prompt before a full run.

inf signals verify <signal-id> \
  --from 2026-06-01T00:00:00Z \
  --to 2026-06-02T00:00:00Z \
  --sample-size 30

Arguments

Argument	Required	Description
`signalId`	Yes	Signal UUID (from `inf signals list`)

Options

Flag	Required	Description	Default
`--from <iso>`	Yes	Window start, ISO-8601	—
`--to <iso>`	Yes	Window end, ISO-8601	—
`--sample-size <n>`	No	Number of items to classify (integer 1–100)	`20`

`inf signals run`

Create and inspect manual backfill runs.

`inf signals run create`

Create a manual signal run over a window.

inf signals run create <signal-id> \
  --from 2026-06-01T00:00:00Z \
  --to 2026-06-02T00:00:00Z \
  --sample-rate 100

Flag	Required	Description	Default
`--from <iso>`	Yes	Start time, ISO-8601	—
`--to <iso>`	Yes	End time, ISO-8601	—
`--sample-rate <percent>`	Yes	Sample rate percent, 0–100	—

`inf signals run list`

List manual runs for a signal.

`inf signals run poll`

Poll a manual signal run until it reaches a terminal state.

inf signals run poll <manual-run-id> --signal-id <signal-id>

Flag	Required	Description	Default
`--signal-id <id>`	Yes	Signal the run belongs to	—
`--timeout <seconds>`	No	Max seconds to poll before giving up	`600`
`--interval <seconds>`	No	Seconds between polls	`10`

`inf signals backfill`

Queue a manual backfill run over a window.

inf signals backfill \
  --signal-id <signal-id> \
  --from 2026-06-01T00:00:00Z \
  --to 2026-06-02T00:00:00Z \
  --sample-rate-percent 100

Flag	Required	Description
`--signal-id <id>`	Yes	Signal to backfill
`--from <iso>`	Yes	Window start, ISO-8601
`--to <iso>`	Yes	Window end, ISO-8601
`--sample-rate-percent <n>`	Yes	Sample rate percent (integer)

`inf signals labels list`

List the per-item labels a signal produced, with paging, sorting, and filtering by label value.

inf signals labels list <signal-id> --classifier string --string-label negative --limit 100

Arguments

Argument	Required	Description
`signalId`	Yes	Signal UUID (from `inf signals list`)

Options

Flag	Required	Description	Default
`--classifier <type>`	Yes	Classifier type: `binary`, `string`, or `regex`	—
`--limit <n>`	No	Max rows to return (integer 1–200)	`50`
`--offset <n>`	No	Rows to skip for paging (integer >= 0)	`0`
`--sort-by <field>`	No	`completed_at`, `target_start_time`, `status`, or `label`	`completed_at`
`--sort-dir <dir>`	No	`asc` or `desc`	`desc`
`--bool-label <bool>`	No	Filter to one binary outcome (`true`/`false`; binary classifiers)	—
`--string-label <label>`	No	Filter to one string label value, e.g. `negative` (string classifiers)	—

`inf signals distribution`

Show how a signal's labels break down over time, bucketed by hour or day (computed live from raw labels).

inf signals distribution <signal-id> \
  --from 2026-06-01T00:00:00Z \
  --to 2026-06-08T00:00:00Z \
  --bucket day

Options

Flag	Required	Description	Default
`--from <iso>`	Yes	Window start, ISO-8601	—
`--to <iso>`	Yes	Window end, ISO-8601	—
`--bucket <b>`	No	`hour` or `day`	`hour`

`inf signals distribution-trends`

Get a materialized-view-backed label distribution for a signal (grid-optimized for larger windows).

inf signals distribution-trends <signal-id> --from <iso> --to <iso> --bucket day

Takes the same --from, --to, and --bucket options as inf signals distribution.

`inf signals alerts`

Manage alerts on a signal — threshold predicates over a rolling window, Slack delivery, and firing history. The full subtree runs under inf signals alerts.

`inf signals alerts create`

inf signals alerts create \
  --signal-id <signal-id> \
  --name "true-rate drop" \
  --metric true_rate \
  --operator lte \
  --threshold 0.8 \
  --window 1h

Flag	Required	Description	Default
`--signal-id <id>`	Yes	Signal to alert on	—
`--name <name>`	Yes	Alert name	—
`--metric <metric>`	Yes	`label_volume`, `true_rate`, `value_count`, or `value_share`	—
`--operator <op>`	Yes	Fire when the value is `gte` or `lte` the threshold	—
`--threshold <number>`	Yes	Threshold value	—
`--window <window>`	Yes	Rolling window: `5m`, `10m`, `15m`, `30m`, `1h`, `3h`, `6h`, `24h`, `48h`	—
`--comparison <mode>`	No	How the threshold is interpreted: `pct_change` or `absolute`	`pct_change`
`--string-label <label>`	For `value_count`/`value_share`	Target value; required for `value_*` metrics, rejected for `label_volume`/`true_rate`	—
`--min-label-count <n>`	No	Min labels in-window before firing	`0`
`--cooldown <minutes>`	No	Cooldown minutes between firings: `0`, `15`, `60`, `360`, or `1440`	`0`
`--slack-channel-id <id>`	No	Slack channel ID (with `--slack-channel-name`)	—
`--slack-channel-name <name>`	No	Slack channel name (with `--slack-channel-id`)	—

`inf signals alerts` — other verbs

edit <alert-id> --signal-id <id> — change an alert's predicate, Slack target, or enabled toggle (same flags as create).
archive <alert-id> --signal-id <id> — archive an alert.
get <alert-id> --signal-id <id>, list --signal-id <id> — fetch alerts.
list-by-team / list-by-project — list alerts across the active team's signals or within a project.
events <alert-id> --signal-id <id> — list an alert's firing history.
state <alert-id> --signal-id <id> — get an alert's current state snapshot.
backtest --signal-id <id> … — replay a predicate over the last N days (read-only preview).

edit, archive, get, events, and state all require --signal-id in addition to the alert ID.

Run inf signals alerts --help for the full flag list on each.

Common workflows

# Backfill labels over a past window, then poll the manual run
RUN_ID=$(inf signals run create <signal-id> \
  --from 2026-06-01T00:00:00Z --to 2026-06-02T00:00:00Z \
  --sample-rate 100 --json | jq -r '.id')
inf signals run poll "$RUN_ID" --signal-id <signal-id>

# Alert when the true-rate drops to/below 0.8 over the last hour
inf signals alerts create --signal-id <signal-id> --name "true-rate drop" \
  --metric true_rate --operator lte --threshold 0.8 --window 1h

# Preview how that predicate would have fired over the last 7 days (read-only)
inf signals alerts backtest --signal-id <signal-id> \
  --metric true_rate --operator lte --threshold 0.8 --window 1h

inf agent lists the auto-discovered agents a signal can target.
inf span and inf trace inspect the items signals label.

Signals

On this page