Tool Calling

Tool calling allows models to generate structured JSON arguments for functions that you define. This enables the model to interact with external APIs, databases, or local code.

Defining Tools

Tools are defined using JSON Schema. The model will decide whether to call a tool based on the user's query.

Example Tool Definition

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather in a given location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "The city and state, e.g. San Francisco, CA"
        }
      },
      "required": ["location"]
    }
  }
}

How to use Tools

Python

from openai import OpenAI

client = OpenAI(base_url="https://llm.siraya.pro/v1", api_key="<API_KEY>")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather like in Boston?"}],
    tools=tools
)

tool_call = response.choices[0].message.tool_calls[0]
print(f"Model wants to call: {tool_call.function.name} with {tool_call.function.arguments}")

Tool Calling Features

• Parallel Tool Calls: Some models can call multiple tools in a single turn.
• Structured Outputs: Combine tools with response_format for even more control.
• Streaming Tool Calls: Receive tool arguments incrementally as they are generated.

For more details on implementing tool execution logic, see our Features: Tool Calling guide.