Chapter 4: Tool Use & Function Calling: Extending LLM Capabilities

Welcome back, future Applied AI Engineer! In our previous chapters, we mastered foundational programming, system thinking, and the art of crafting effective prompts to guide Large Language Models (LLMs). We learned how LLMs are incredible text generators, capable of understanding and producing human-like language. But what if an LLM needs to do more than just talk? What if it needs to act in the real world, fetch live data, or perform calculations beyond its inherent knowledge?

That’s precisely what this chapter is about! We’re diving into the exciting world of Tool Use and Function Calling. These techniques are foundational for building truly dynamic and capable AI agents that can interact with external systems, retrieve real-time information, and perform complex, multi-step tasks. By the end of this chapter, you’ll understand how to empower your LLMs to go beyond mere conversation and become proactive problem-solvers.

Get ready to roll up your sleeves, because this is where AI applications start to get really powerful and interactive! We’ll build on your Python skills and understanding of LLM APIs to integrate external functionalities seamlessly.

The Limitations of a “Vanilla” LLM

Imagine you’re talking to a brilliant linguist who knows everything about language, history, and literature. They can write amazing stories, summarize complex topics, and even generate creative ideas. However, if you ask them “What’s the weather like in London right now?” or “What’s 1234 times 5678?”, they might struggle. Why? Because their knowledge is confined to what they’ve learned from books and training data. They can’t look out a window, check a weather app, or use a calculator.

Similarly, a “vanilla” LLM, for all its linguistic prowess, is limited to the data it was trained on and its text generation capabilities. It can’t:

• Access real-time information (like current stock prices, news, or weather).
• Perform precise calculations.
• Interact with external APIs (like sending an email, booking a flight, or querying a database).
• Execute code in the traditional sense.

This is where the concept of Tool Use comes in to save the day!

Introducing Tools: LLMs with Superpowers!

Think of “tools” as external capabilities or functions that you provide to an LLM. These tools are like extensions that grant the LLM new “senses” or “actions” in the real world. When an LLM is given access to tools, it gains the ability to:

1. Understand when a tool might be useful to answer a user’s request.
2. Determine which tool to use.
3. Figure out the correct arguments to pass to that tool based on the user’s prompt.

This process is often facilitated by a mechanism called Function Calling.

What is Function Calling?

Function calling, or tool-use API, is the specific way we enable an LLM to “call” or “use” these external tools. It’s a special capability offered by many modern LLM providers (like OpenAI, Anthropic, Google GenAI) that allows you to describe available functions to the LLM.

Here’s the magic:

1. You define a tool: You write a regular Python function (e.g., get_current_weather(location, unit)) that does something useful.
2. You describe the tool to the LLM: You provide the LLM with a schema (a structured description, usually in JSON) of your function. This schema includes the function’s name, its purpose, and the parameters it expects (along with their types and descriptions).
3. The LLM decides: When a user asks a question, the LLM analyzes the query and the available tool descriptions. If it thinks a tool can help, it generates a structured output (not natural language!) that looks exactly like a call to one of your defined functions, complete with the arguments it believes are necessary.
4. Your application executes: This is crucial! The LLM does not execute the function itself. Your application receives the LLM’s “suggested function call.” Your code then takes this suggestion, executes the actual Python function, and captures its output.
5. The LLM gets the result: Your application sends the result of the tool’s execution back to the LLM.
6. The LLM responds: With the tool’s output in hand, the LLM can now formulate a complete, accurate, and contextually relevant natural language response to the user.

Let’s visualize this flow:

This cycle of prompt -> LLM suggests tool -> application executes tool -> LLM gets result -> LLM responds is the core of how LLMs interact with the world. It’s the first step towards building truly agentic AI systems that can plan, execute, and adapt.

Benefits of Tool Use & Function Calling

• Real-time Data: Access current information for up-to-date responses.
• Enhanced Accuracy: Leverage precise external calculations or structured data.
• Automation: Trigger actions in other systems (e.g., send emails, update databases).
• Extended Capabilities: Overcome the inherent limitations of an LLM’s training data.
• Foundation for Agents: Essential building block for creating sophisticated AI agents that can perform complex, multi-step tasks.

Step-by-Step Implementation: Building a Weather Tool

Let’s get hands-on! We’ll build a simple application that allows an LLM to “check the weather” using a mock function.

1. Setup Your Environment

First, ensure you have Python 3.10 or newer installed. We’ll use the OpenAI Python SDK for this example, as it has robust function calling capabilities.

Create a new directory for your project and install the necessary library:

mkdir llm-tools-chapter
cd llm-tools-chapter
python -m venv venv
source venv/bin/activate # On Windows, use `venv\Scripts\activate`
pip install openai~=1.14.3 python-dotenv~=1.0.1

Note on Versions (as of 2026-01-16): The openai Python SDK is actively developed. As of January 2026, 1.14.3 or a similar version in the 1.x.x series is a reasonable estimate for a stable release. python-dotenv is used for securely managing API keys.

Next, create a .env file in your project root to store your OpenAI API key:

# .env
OPENAI_API_KEY="YOUR_OPENAI_API_KEY_HERE"

Replace "YOUR_OPENAI_API_KEY_HERE" with your actual key from the OpenAI developer dashboard. Never hardcode API keys directly in your code!

2. Define Your Tool Function

Let’s create a Python function that simulates fetching weather data. In a real application, this would call a weather API. For now, we’ll keep it simple.

Create a file named main.py:

# main.py

import os
import json
from dotenv import load_dotenv
from openai import OpenAI

# Load environment variables
load_dotenv()

# Initialize OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# --- Our Tool Definition ---
def get_current_weather(location: str, unit: str = "fahrenheit") -> str:
    """Get the current weather in a given location."""
    print(f"--- Calling get_current_weather for {location} in {unit} ---")
    if "tokyo" in location.lower():
        return json.dumps({"location": "Tokyo", "temperature": "10", "unit": "celsius"})
    elif "san francisco" in location.lower():
        return json.dumps({"location": "San Francisco", "temperature": "72", "unit": "fahrenheit"})
    elif "london" in location.lower():
        return json.dumps({"location": "London", "temperature": "5", "unit": "celsius"})
    else:
        return json.dumps({"location": location, "temperature": "unknown", "unit": unit})

# We'll add the LLM interaction here next!

Explanation:

• load_dotenv(): Loads variables from your .env file.
• client = OpenAI(...): Initializes the OpenAI client.
• get_current_weather(location, unit): This is our mock tool. It takes a location (string) and an optional unit (string, defaulting to “fahrenheit”).
• It prints a message to show when it’s called (useful for debugging!).
• It returns a JSON string simulating weather data. Using json.dumps is common when tools return structured data.

3. Describe the Tool to the LLM (Tool Schema)

Now, we need to tell the LLM about our get_current_weather function. We do this by providing a structured description called a tool schema. This schema tells the LLM the tool’s name, what it does, and what parameters it expects.

Add the following dictionary to your main.py file, right after the get_current_weather function definition:

# main.py (continued)

# --- Tool Schema for the LLM ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g., San Francisco, CA",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

Explanation:

• tools: This is a list because an LLM can potentially have access to multiple tools.
• "type": "function": Specifies that this tool is a function.
• "function": Contains the details of the function.
  • "name": "get_current_weather": Must exactly match the Python function name.
  • "description": A human-readable description. This is CRITICAL for the LLM to understand when to use the tool. Make it clear and concise!
  • "parameters": Describes the function’s arguments using a JSON Schema format.
    • "type": "object": Parameters are grouped as an object.
    • "properties": Each key here corresponds to a parameter of your Python function.
      • "location": Describes the location parameter.
      • "unit": Describes the unit parameter and uses "enum" to specify allowed values (optional, but good practice).
    • "required": ["location"]: Lists the parameters that must be provided for the function to be called.

4. Make an LLM Call with Tools and Handle the Response

Now for the grand finale! We’ll send a user query to the LLM, along with our tool descriptions. The LLM will then decide whether to call our get_current_weather tool.

Add the following logic to your main.py file, after the tools definition:

# main.py (continued)

# --- LLM Interaction Logic ---
def run_conversation(user_message: str):
    messages = [{"role": "user", "content": user_message}]

    # Step 1: Send the user message and available tools to the LLM
    print(f"\nUser: {user_message}")
    response = client.chat.completions.create(
        model="gpt-4o-2024-05-13", # Using a recent, capable model
        messages=messages,
        tools=tools,
        tool_choice="auto", # Let the LLM decide if it needs a tool
    )
    response_message = response.choices[0].message

    # Step 2: Check if the LLM wants to call a tool
    if response_message.tool_calls:
        print(f"LLM wants to call a tool: {response_message.tool_calls[0].function.name}")
        print(f"Arguments: {response_message.tool_calls[0].function.arguments}")

        # Add the LLM's tool call message to the conversation history
        messages.append(response_message)

        # Step 3: Execute the tool based on the LLM's suggestion
        available_functions = {
            "get_current_weather": get_current_weather,
        }
        
        tool_call = response_message.tool_calls[0]
        function_name = tool_call.function.name
        function_to_call = available_functions[function_name]
        function_args = json.loads(tool_call.function.arguments)
        
        # Call the actual Python function
        function_response = function_to_call(
            location=function_args.get("location"),
            unit=function_args.get("unit")
        )

        # Step 4: Send the tool's output back to the LLM
        messages.append(
            {
                "tool_call_id": tool_call.id,
                "role": "tool",
                "name": function_name,
                "content": function_response,
            }
        )

        # Step 5: Get the final response from the LLM
        second_response = client.chat.completions.create(
            model="gpt-4o-2024-05-13",
            messages=messages,
        )
        print(f"LLM Final Response: {second_response.choices[0].message.content}")
        return second_response.choices[0].message.content
    else:
        # If no tool call, just print the LLM's direct response
        print(f"LLM Direct Response: {response_message.content}")
        return response_message.content

if __name__ == "__main__":
    print("--- Starting LLM Tool Interaction ---")
    run_conversation("What's the weather like in San Francisco?")
    run_conversation("What about Tokyo in Celsius?")
    run_conversation("Just tell me a fun fact about AI.")
    run_conversation("What's the weather in Berlin?")
    print("--- Finished LLM Tool Interaction ---")

Explanation:

• run_conversation(user_message): This function encapsulates the entire interaction.
• messages = [{"role": "user", "content": user_message}]: We start with the user’s initial message.
• client.chat.completions.create(...): This is where we call the OpenAI chat completion API.
  • model="gpt-4o-2024-05-13": We’re using a modern, capable model. Always check the latest recommended models in the official OpenAI documentation.
  • messages=messages: Our conversation history.
  • tools=tools: We pass our tools schema here! This is how the LLM knows what’s available.
  • tool_choice="auto": This tells the LLM to automatically decide whether to use a tool or just respond directly. You can also force a tool call or prevent it.
• if response_message.tool_calls:: This is the critical check. If the LLM decides to use a tool, response_message.tool_calls will contain the suggested function call(s).
• messages.append(response_message): We add the LLM’s tool call message to the conversation history. This is important so the LLM remembers what it suggested.
• available_functions = {...}: A dictionary mapping tool names (strings) to their actual Python function objects. This allows us to dynamically call the correct function.
• function_name = tool_call.function.name: Extracts the name of the function the LLM wants to call.
• function_args = json.loads(tool_call.function.arguments): The LLM provides arguments as a JSON string; we parse it back into a Python dictionary.
• function_response = function_to_call(...): We execute our actual Python function with the arguments provided by the LLM.
• messages.append({"tool_call_id": tool_call.id, "role": "tool", ...}): After executing the tool, we send its output back to the LLM. The role: "tool" is specific for tool outputs.
• second_response = client.chat.completions.create(...): We make a second LLM call, this time with the tool’s output included in the messages history. The LLM uses this output to formulate its final, natural language response.
• else: If the LLM doesn’t suggest a tool call (e.g., “Tell me a fun fact about AI.”), it will directly generate a response.

5. Run Your Application

Execute your main.py file:

python main.py

You should see output similar to this (the exact LLM responses may vary slightly):

--- Starting LLM Tool Interaction ---

User: What's the weather like in San Francisco?
LLM wants to call a tool: get_current_weather
Arguments: {"location": "San Francisco"}
--- Calling get_current_weather for San Francisco in fahrenheit ---
LLM Final Response: The current weather in San Francisco is 72 degrees Fahrenheit.

User: What about Tokyo in Celsius?
LLM wants to call a tool: get_current_weather
Arguments: {"location": "Tokyo", "unit": "celsius"}
--- Calling get_current_weather for Tokyo in celsius ---
LLM Final Response: The current weather in Tokyo is 10 degrees Celsius.

User: Just tell me a fun fact about AI.
LLM Direct Response: Did you know that the term "artificial intelligence" was coined in 1956 by John McCarthy at the Dartmouth Conference, which is considered the birth of AI as a field?

User: What's the weather in Berlin?
LLM wants to call a tool: get_current_weather
Arguments: {"location": "Berlin"}
--- Calling get_current_weather for Berlin in fahrenheit ---
LLM Final Response: I don't have specific weather data for Berlin, but I can tell you that the temperature for Berlin is currently unknown.

--- Finished LLM Tool Interaction ---

Notice how for “San Francisco”, “Tokyo”, and “Berlin”, the LLM correctly identified that it needed to use the get_current_weather tool, extracted the location and unit, and then used the tool’s output to form a natural language response. For the “fun fact” query, it correctly decided not to use any tools and responded directly.

This is a powerful concept! You’ve just enabled an LLM to interact with an external system.

Mini-Challenge: Build a Simple Calculator Tool

Now it’s your turn! Let’s add another tool to our LLM’s arsenal: a simple calculator that can add two numbers.

Challenge:

1. Create a new Python function called add_numbers(a: float, b: float) -> float that takes two floating-point numbers and returns their sum.
2. Define a tool schema for your add_numbers function. Remember to include its name, description, and parameters (with type and required fields).
3. Add your new tool schema to the tools list in main.py.
4. Update the available_functions dictionary to include your new add_numbers function.
5. Add a new run_conversation call in the if __name__ == "__main__": block to test it, e.g., run_conversation("What is 15.3 plus 20.7?").

Hint:

• Your add_numbers function should be straightforward: return a + b.
• The tool schema for parameters will look similar to get_current_weather, but with two parameters (a and b), both of type: "number".
• Remember to json.dumps the return value of your add_numbers function, just like get_current_weather.

What to Observe/Learn:

• How easily you can extend the LLM’s capabilities by simply defining new Python functions and their corresponding schemas.
• The importance of clear description fields in your tool schema for the LLM to correctly identify when to use a tool.
• How the LLM correctly parses numerical arguments for your new tool.

Take your time, experiment, and don’t be afraid to make mistakes! That’s how we learn best.

Common Pitfalls & Troubleshooting

1. Incorrect Tool Schema:

   • Problem: The LLM doesn’t call your tool when it should, or calls it with incorrect arguments. This is often because the description is unclear, or the parameters definition in the JSON schema is wrong (e.g., wrong type, missing required fields).
   • Solution: Review your description carefully. Is it unambiguous about what the tool does and when it should be used? Double-check your JSON schema against the OpenAI API documentation for function calling. Ensure name matches the Python function exactly.
   • Tip: Try making the description more specific. For example, instead of “Gets weather,” use “Retrieves the current weather conditions for a specified city and state.”

2. Tool Execution Errors:

   • Problem: The LLM correctly identifies the tool and arguments, but your Python function fails when executed.
   • Solution: This is a standard Python debugging problem. Add print() statements inside your tool function to see the arguments it receives. Use a debugger if necessary. Ensure your function handles edge cases (e.g., None values, incorrect types if not strictly enforced).
   • Tip: Wrap your tool function’s execution in a try...except block in your run_conversation logic to gracefully handle errors and potentially report them back to the LLM.

3. LLM Hallucinating Tool Calls or Arguments:

   • Problem: The LLM might try to call a tool that doesn’t exist, or it might invent arguments that aren’t part of your schema.
   • Solution: While less common with modern models, ensure your tool schema is precise. If tool_choice="auto" is too permissive, you can try tool_choice={"type": "function", "function": {"name": "your_tool_name"}} to force a specific tool, or tool_choice="none" to prevent any tool calls. For hallucinated arguments, often refining the tool’s description or the user’s prompt can help.
   • Security Note: Always validate arguments received from the LLM before executing the tool. Never trust LLM-generated input blindly, especially if your tool interacts with sensitive systems.

4. Forgetting to Send Tool Output Back to LLM:

   • Problem: The LLM calls a tool, but then just stops or gives a generic response instead of using the tool’s data.
   • Solution: You must make a second LLM API call, including the tool’s output in the messages history with role="tool". Without this, the LLM won’t know the outcome of its suggested action.

Summary

Congratulations! You’ve just unlocked a superpower for your LLMs. In this chapter, we covered:

• The inherent limitations of LLMs and why they need external tools.
• The concept of Tool Use, enabling LLMs to interact with the real world.
• Function Calling as the primary mechanism to describe and invoke these tools.
• A step-by-step implementation of a weather tool using the OpenAI Python SDK, including defining the tool function, creating its JSON schema, and orchestrating the LLM interaction.
• Practical tips for troubleshooting common issues.

By mastering tool use and function calling, you’re laying a crucial cornerstone for building sophisticated AI agents. This ability to extend an LLM’s capabilities beyond its training data is what transforms a conversational model into a true problem-solving agent.

In the next chapter, we’ll dive deeper into Retrieval-Augmented Generation (RAG), another powerful technique that allows LLMs to access and utilize vast amounts of external, up-to-date information, further enhancing their intelligence and accuracy. Get ready to give your LLMs a memory!

References

• OpenAI Documentation: Function calling
• OpenAI Python Library GitHub Repository
• Python-dotenv GitHub Repository
• JSON Schema Official Website

This page is AI-assisted and reviewed. It references official documentation and recognized resources where relevant.