Function Calling

Function calling provides a powerful and flexible way for OpenAI models to interface with your code or external services, and has two primary use cases:

Fetching Data	Retrieve up-to-date information to incorporate into the model’s response (RAG). Useful for searching knowledge bases and retrieving specific data from APIs (e.g. current weather data).
Taking Action	Perform actions like submitting a form, calling APIs, modifying application state (UI/frontend or backend), or taking agentic workflow actions (like handing off the conversation).

​

Getting Started

You’ll need an Inference Cloud account and API key to use the Batch API. See our Quick Start Guide for instructions on how to create an account and get an API key.

Install the OpenAI SDK for your language of choice. To connect to Inference Cloud using the OpenAI SDK, you will need to set thebaseURL to https://api.inference.net/v1 and the apiKey to your Inference Cloud API key, as shown below:

​

Overview

You can extend the capabilities of OpenAI models by giving them access to tools.

This guide will cover how you can give the model access to your own functions through function calling.

With function calling, you’ll tell the model what functions are available to it, and it will decide which one to use.

You’ll then execute the function code, send back the results, and the model will incorporate them into its final response.

​

Sample function

Let’s look at the steps to allow a model to use a real get_weather function defined below:

Sample get_weather function implemented in your codebase

Unlike the diagram earlier, this function expects precise latitude and longitude instead of a general location parameter.

​

Step By Step Example

​

Step 1: Call model with get_weather tool defined

​

Step 2: Pull the selected function call from the model’s response

​

Step 3: Execute the get_weather function

​

Step 4: Supply result and call model again

​

Output

​

Defining functions

Functions can be set in the tools parameter of each API request inside a function object.

A function is defined by its schema, which informs the model what it does and what input arguments it expects. It comprises the following fields:

Field	Description
name	The function’s name (e.g. get_weather)
description	Details on when and how to use the function
parameters	JSON schema defining the function’s input arguments

Take a look at this example:

Because the parameters are defined by a JSON schema, you can leverage many of its rich features like property types, enums, descriptions, nested objects, and, recursive objects.

​

SDK Helpers

While you can define function schemas directly, OpenAI’s SDKs have helpers to convert pydantic and zod objects into schemas. Not all pydantic and zod features are supported. Here is an example of how to use the SDK to define a schema.

​

Best practices for defining functions

1. Write clear and detailed function names, parameter descriptions, and instructions.

   • Explicitly describe the purpose of the function and each parameter (and its format), and what the output represents.
   • Use the system prompt to describe when (and when not) to use each function. Generally, tell the model exactly what to do.
   • Include examples and edge cases, especially to rectify any recurring failures.

2. Apply software engineering best practices.

   • Make the functions obvious and intuitive. (principle of least surprise)
   • Use enums and object structure to make invalid states unrepresentable. (e.g. toggle_light(on: bool, off: bool) allows for invalid calls)
   • Pass the intern test. Can an intern/human correctly use the function given nothing but what you gave the model? (If not, what questions do they ask you? Add the answers to the prompt.)

3. Offload the burden from the model and use code where possible.

   • Don’t make the model fill arguments you already know. For example, if you already have an order_id based on a previous menu, don’t have an order_id param – instead, have no params submit_refund() and pass the order_id with code.
   • Combine functions that are always called in sequence. For example, if you always call mark_location() after query_location(), just move the marking logic into the query function call.

4. Keep the number of functions small for higher accuracy.

   • Evaluate your performance with different numbers of functions.
   • Aim for fewer than 20 functions at any one time, though this is just a soft suggestion.

​

Additional configurations

​

Tool choice

By default the model will determine when and how many tools to use. You can force specific behavior with the tool_choice parameter.

1. Auto: (Default) Call zero, one, or multiple functions. tool_choice: "auto"
2. Required: Call one or more functions. tool_choice: "required"
3. Forced Function: Call exactly one specific function. tool_choice: {"type": "function", "function": {"name": "get_weather"}}

You can also set tool_choice to "none" to imitate the behavior of passing no functions.

​

Strict mode

Setting strict to true will ensure function calls reliably adhere to the function schema, instead of being best effort. We recommend always enabling strict mode.

Under the hood, strict mode works by leveraging our structured outputs feature and therefore introduces a couple requirements:

1. additionalProperties must be set to false for each object in the parameters.
2. All fields in properties must be marked as required.

You can denote optional fields by adding null as a type option (see example below).

Strict mode enabled

While we recommend you enable strict mode, it has a few limitations. Some features of JSON schema are not supported. (See supported schemas.)

​

Streaming

Streaming can be used to surface progress by showing which function is called as the model fills its arguments, and even displaying the arguments in real time.

Streaming function calls is very similar to streaming regular responses: you set stream to true and get chunks with delta objects.

Output of delta.tool_calls:

Instead of aggregating chunks into a single content string, however, you’re aggregating chunks into an encoded arguments JSON object.

When the model calls one or more functions the tool_calls field of each delta will be populated. Each tool_call contains the following fields:

Field	Description
index	Identifies which function call the delta is for
id	Tool call id.
function	Function call delta (name and arguments)
type	Type of tool_call (always function for function calls)

Many of these fields are only set for the first delta of each tool call, like id, function.name, and type.

Below is a code snippet demonstrating how to aggregate the deltas into a final tool_calls object.

Accumulated final_tool_calls[0]