Tool calling

Tool calling (also called function calling) lets a model ask your code to do something, such as look up an order or search the web. You describe each tool with a name, a description and its parameters. When the model needs one, it replies with the tool name and the arguments instead of text. Your code runs the tool and sends the result back, and the model uses it in its answer. Agents, including the bitsrun coding agent, depend on it.

Check that your model supports it

Not every model was trained for tools. Check before you deploy:

bitsrun size qwen3:8b --json

Look for "tools": true in the result. false means the model was not set up for tools. Qwen3, Qwen2.5 Instruct, Llama 3.1 and newer, and Mistral instruct models generally do. Embedding models never do.

Call a tool

import json, os
from openai import OpenAI

client = OpenAI(
    base_url="https://01k8y4000d3m00000000000011.endpoint.bitsrun.com/v1",
    api_key=os.environ["BITSRUN_API_KEY"],
)
tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the shipping status of an order by its number.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string", "description": "The order number, like A-1042"}},
            "required": ["order_id"],
        },
    },
}]
messages = [{"role": "user", "content": "Where is my order A-1042?"}]
reply = client.chat.completions.create(model="qwen3:8b", messages=messages, tools=tools)
call = reply.choices[0].message.tool_calls[0]
print(call.function.name, json.loads(call.function.arguments))

Send the result back

Add the model's tool call and your result to the conversation, then ask again:

messages.append(reply.choices[0].message)
messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps({"status": "Out for delivery"})})
final = client.chat.completions.create(model="qwen3:8b", messages=messages, tools=tools)
print(final.choices[0].message.content)

You can also send tool_choice (for example "auto", "none" or a specific function) and parallel_tool_calls.

Tips

• Write clear descriptions. Smaller models pick tools by their descriptions. Say what each tool does and when to use it.
• Keep the list short. Every tool description uses part of the model's memory on each request. A handful of focused tools works better than dozens.
• Stream long tool calls. A model writing a large argument (a whole file, for example) can be quiet for a while. Streaming keeps the connection open; BitsRun sends keep-alive comments the SDKs ignore.
• Validate arguments. Treat the arguments as input from a user: check them before your code acts on them.

What can go wrong

• The model answers in text instead of calling the tool: the model may not support tools, or the description is unclear. Check with bitsrun size <model> --json, or try a larger model.
• 400 context_too_long: long tool lists and results fill the model's memory. Trim old tool results or raise Memory length under Performance.