Function Calling
Enable models to fetch data and take actions.
Support for Function Calling is in beta and will be available soon.
Introduction
Function calling provides a powerful and flexible way for OpenAI models to interface with your code or external services, and has two primary use cases:
Fetching Data | Retrieve up-to-date information to incorporate into the model’s response (RAG). Useful for searching knowledge bases and retrieving specific data from APIs (e.g. current weather data). |
Taking Action | Perform actions like submitting a form, calling APIs, modifying application state (UI/frontend or backend), or taking agentic workflow actions (like handing off the conversation). |
If you only want the model to produce JSON, see our docs on structured outputs.
Getting Started
You’ll need an Inference.net account and API key to use Function Calling. See our Quick Start Guide for instructions on how to create an account and get an API key.
Install the OpenAI SDK for your language of choice.
To connect to Inference.net using the OpenAI SDK, you will need to set the base URL to https://api.inference.net/v1
.
In this example, we are reading the API key from the environment variable INFERENCE_API_KEY
.
Overview
You can extend the capabilities of OpenAI models by giving them access to functions that you define called tools
. This is also called “function calling”.
While OpenAI’s models have access to built-in tools, our models only support function calling for functions that you define.
With function calling, you’ll tell the model what tools are available to it, and it will decide which one to use.
You’ll then execute the function code, send back the results, and the model will incorporate them into its final response.
Sample function
Let’s look at the steps to allow a model to use a real get_weather
function defined below:
All functions must return strings. You can format the string as JSON or another format if you like, but the return type itself must be a string.
Unlike the diagram earlier, this function expects precise latitude
and longitude
instead of a general location
parameter.
Step By Step Example
Step 1: Call model with get_weather tool defined
Step 2: Pull the selected function call from the model’s response
Step 3: Execute the get_weather
function
Step 4: Supply result and call model again
Output
Defining functions
Functions can be set in the tools
parameter of each API request.
A function is defined by its schema, which informs the model what it does and what input arguments it expects. It comprises the following fields:
Field | Description |
---|---|
name | The function’s name (e.g. get_weather) |
description | Details on when and how to use the function |
parameters | JSON schema defining the function’s input arguments |
Take a look at this example:
Because the parameters
are defined by a JSON schema, you can leverage many of its rich features like property types, enums, and descriptions.
SDK Helpers
While you can define function schemas directly, OpenAI’s SDKs have helpers to convert pydantic
and zod
objects into schemas.
pydantic
and zod
features are currently supported by Function Calling, but simple, flat schemas are supported.Best practices for defining functions
-
Write clear and detailed function names, parameter descriptions, and instructions.
- Explicitly describe the purpose of the function and each parameter (and its format), and what the output represents.
- Use the system prompt to describe when (and when not) to use each function. Generally, tell the model exactly what to do.
- Include examples and edge cases, especially to rectify any recurring failures.
-
Apply software engineering best practices.
- Make the functions obvious and intuitive. (principle of least surprise)
- Use enums and object structure to make invalid states unrepresentable. (e.g.
toggle_light(on: bool, off: bool)
allows for invalid calls) - Pass the intern test. Can an intern/human correctly use the function given nothing but what you gave the model? (If not, what questions do they ask you? Add the answers to the prompt.)
-
Offload the burden from the model and use code where possible.
- Don’t make the model fill arguments you already know. For example, if you already have an
order_id
based on a previous menu, don’t have anorder_id
param – instead, have no paramssubmit_refund()
and pass theorder_id
with code. - Combine functions that are always called in sequence. For example, if you always call
mark_location()
afterquery_location()
, just move the marking logic into the query function call.
- Don’t make the model fill arguments you already know. For example, if you already have an
-
Keep the number of functions small for higher accuracy.
- Evaluate your performance with different numbers of functions.
- Aim for fewer than 20 functions at any one time, though this is just a soft suggestion.
Streaming
Streaming can be used to surface progress by showing which function is called as the model fills its arguments, and even displaying the arguments in real time.
Streaming function calls is very similar to streaming regular responses: you set stream
to true
and get chunks with delta
objects.
Output of delta.tool_calls
:
Instead of aggregating chunks into a single content
string, however, you’re aggregating chunks into an encoded arguments
JSON object.
When the model calls one or more functions the tool_calls
field of each delta
will be populated. Each tool_call
contains the following fields:
Field | Description |
---|---|
index | Identifies which function call the delta is for |
id | Tool call id. |
function | Function call delta (name and arguments) |
type | Type of tool_call (always function for function calls) |
Many of these fields are only set for the first delta
of each tool call, like id
, function.name
, and type
.
Below is a code snippet demonstrating how to aggregate the delta
objects into a final tool_calls
object.
Accumulated final_tool_calls[0]