Skip to main content
Learn how to use our Group API to submit multiple inference requests together, perfect for processing related tasks that need to be tracked as a unit. The Group API supports both chat completions and text completions with up to 50 requests per group.
Group API is available for both /v1/slow/group/chat/completions and /v1/slow/group/completions endpoints.
You should not mix completion and chat-completion requests in the same group.

​
Overview

The Group API provides a streamlined way to submit multiple asynchronous inference requests as a single unit. Unlike the Batch API which requires JSONL file uploads, the Group API accepts requests directly in the request body, making it ideal for:
  • Small to medium batches: Process up to 50 requests at once
  • Related tasks: Group related inference requests together
  • Webhook notifications: Get notified when all requests in a group complete
  • Simpler integration: No file uploads or JSONL formatting required
  • Faster implementation: Direct JSON API calls without file management

​
Group API vs Batch API

FeatureGroup APIBatch API
Maximum requests501,000,000
Input formatJSON array in request bodyJSONL file upload
File managementNot requiredRequired
Use caseSmall batches, quick implementationLarge-scale processing
Webhook supportYesYes
Completion time1-72 hours1-72 hours

​
Getting Started

​
1. Submit a Group Request

Submit multiple requests together by sending them as an array in the request body: 
const response = await fetch(
  "https://api.inference.net/v1/slow/group/chat/completions",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      requests: [
        {
          model: "google/gemma-3-27b-instruct/bf-16",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of France?" },
          ],
          max_tokens: 100,
        },
        {
          model: "google/gemma-3-27b-instruct/bf-16",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of Germany?" },
          ],
          max_tokens: 100,
        },
      ],
      webhook_id: "my-webhook-123", // Optional: attach a webhook for notifications
    }),
  },
);

const result = await response.json();
console.log(result); // { groupId: "group_abc123", groupSize: 2 }
The response will include a group ID and the number of requests:
JSON
{
  "groupId": "group_xY3kL9mN2pQ",
  "groupSize": 2
}

​
2. Retrieve Group Results

Once your group is processed, retrieve all generation results using the group ID: 
const response = await fetch(
  `https://api.inference.net/v1/slow/group/${groupId}/generations`,
  {
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
    },
  },
);

const result = await response.json();
console.log(result.generations); // Array of all completed generations
The response includes all generations in the group:
JSON
{
  "generations": [
    {
      "_id": "gen_abc123",
      "state": "Success",
      "request": {
        "model": "google/gemma-3-27b-instruct/bf-16",
        "messages": [
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is the capital of France?"}
        ],
        "max_tokens": 100
      },
      "response": {
        "id": "gen_abc123",
        "object": "chat.completion",
        "choices": [
          {
            "message": {
              "role": "assistant",
              "content": "The capital of France is Paris."
            },
            "finish_reason": "stop"
          }
        ],
        "usage": {
          "prompt_tokens": 25,
          "completion_tokens": 8,
          "total_tokens": 33
        }
      }
    },
    {
      "_id": "gen_def456",
      "state": "Success",
      "request": {
        "model": "google/gemma-3-27b-instruct/bf-16",
        "messages": [
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is the capital of Germany?"}
        ],
        "max_tokens": 100
      },
      "response": {
        "id": "gen_def456",
        "object": "chat.completion",
        "choices": [
          {
            "message": {
              "role": "assistant",
              "content": "The capital of Germany is Berlin."
            },
            "finish_reason": "stop"
          }
        ],
        "usage": {
          "prompt_tokens": 25,
          "completion_tokens": 8,
          "total_tokens": 33
        }
      }
    }
  ]
}

​
Using Webhooks

Attach a webhook to receive notifications when your group completes processing. Include the webhook_id when submitting the group: 
const response = await fetch(
  "https://api.inference.net/v1/slow/group/chat/completions",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      requests: [
        /* your requests */
      ],
      webhook_id: "my-webhook-123",
    }),
  },
);
When all requests in the group complete, your webhook will receive a notification with:
  • Group ID
  • Completion status
  • Summary of successful and failed requests
  • Custom IDs for each request (if provided)
See our Webhook Documentation for setup instructions.

​
Text Completions Support

The Group API also supports text completions: 
const response = await fetch(
  "https://api.inference.net/v1/slow/group/completions",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      requests: [
        {
          model: "google/gemma-3-27b-instruct/bf-16",
          prompt: "The capital of France is",
          max_tokens: 10,
        },
        {
          model: "google/gemma-3-27b-instruct/bf-16",
          prompt: "The capital of Germany is",
          max_tokens: 10,
        },
      ],
    }),
  },
);

​
Limits and Constraints

  • Maximum requests per group: 50
  • Request format: Direct JSON (no JSONL files required)
  • Supported endpoints:
    • /v1/slow/group/chat/completions
    • /v1/slow/group/completions
  • Completion time: 24-72 hours
  • Request expiration: Groups expire after 72 hours if not completed

​
Best Practices

  1. Group related requests: Use groups for requests that logically belong together (e.g., analyzing multiple documents from the same source).
  2. Use webhooks for notifications: Instead of polling, configure webhooks to be notified when your group completes.
  3. Handle individual failures: Some requests in a group may fail while others succeed. Check each generation’s status.
  4. Stay under limits: Keep groups to 50 requests or less. For larger batches, use the Batch API.
  5. Include metadata: Add custom IDs or metadata to your requests for easier tracking:
    JSON
    {
  "model": "google/gemma-3-27b-instruct/bf-16",
  "messages": [{"role": "user", "content": "..."}],
  "metadata": {
    "custom_id": "doc_123",
    "type": "summary"
  }
}

​
Error Handling

The API validates your request structure immediately. Common errors include:
JSON
{
  "error": {
    "message": "Invalid request body.",
    "type": "BadRequestError",
    "fields": {
      "_errors": ["Unrecognized key(s) in object: 'webhook_url'"]
    }
  }
}
Ensure you use the correct field names:
  • webhook_id (correct)
  • webhook_url (incorrect — this is for the Batch API)

​
When to Use Group API

Choose the Group API when you need:
  • Quick implementation without file management
  • To process 50 or fewer related requests
  • Webhook notifications for a set of requests
  • Simple JSON-based integration
For larger workloads (50+ requests), consider using the Batch API instead.