Skip to main content
Inference.net currently relies on JSONL for the two main file-driven workflows in the platform.

​
Batch API input files

Each line defines one request. Required fields:
  • custom_id
  • method
  • body
url is not required in the JSONL line format Inference.net currently validates. The request type is inferred from the body shape on each line.

​
Source-backed Batch API line

This shape comes from inference/apps/relay/tests/e2e/utils/batch-test.utils.ts. 
{"custom_id":"chat-1","method":"POST","body":{"model":"meta-llama/llama-3.2-3b-instruct/fp-16","n":null,"temperature":null,"top_p":null,"messages":[{"role":"user","content":"What is the capital of France?"}]}}
Typical use:
  • large offline queues for chat completions or completions
  • nightly processing jobs
  • translation, extraction, or synthetic-data workloads at scale

​
Historical dataset uploads

Each line is one request or request/response record. Typical fields:
  • request
  • optional response

​
Source-backed historical upload line

This shape matches the upload processor tests in inference/apps/llm-ops-consumer/tests/unit/inference-upload.vitest.test.ts. 
{"request":{"model":"gpt-4","messages":[{"role":"user","content":"Hello"}],"temperature":0.7,"max_tokens":100},"response":{"id":"chatcmpl-123","object":"chat.completion","created":1700000000,"model":"gpt-4","choices":[{"index":0,"message":{"role":"assistant","content":"Hi there!"},"finish_reason":"stop"}]}}
Typical use:
  • importing logs from another provider
  • backfilling Observe with historical traffic
  • creating eval or training datasets from older request history

​
JSONL rules

  • one valid JSON object per line
  • no trailing commas across lines
  • keep each line self-contained