# Chat and completions

> Send a list of messages to /v1/chat/completions and get the model's reply, all at once or streamed word by word.

Source: https://bitsrun.com/docs/api/chat/
Updated: 2026-11-09

Chat is the main way to talk to a language model. You send the conversation so far as a list of messages, and the model writes the next one.

## Send a chat request

Each message has a `role` and `content`. Roles are `system` (instructions for the model), `user` (the person) and `assistant` (earlier replies from the model).

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://01k8y4000d3m00000000000011.endpoint.bitsrun.com/v1",
    api_key=os.environ["BITSRUN_API_KEY"],
)
reply = client.chat.completions.create(
    model="qwen3:8b",
    messages=[
        {"role": "system", "content": "You answer support questions for an online bike shop. Be brief."},
        {"role": "user", "content": "How do I return a helmet?"},
    ],
    max_tokens=300,
    temperature=0.3,
)
print(reply.choices[0].message.content)
```

The model does not remember earlier calls. To continue a conversation, send all earlier messages again, including the assistant's replies.

## Stream the answer

Streaming sends the reply in small pieces as the model writes it, so people see text right away. While a sleeping endpoint starts, any request waits on the open connection; streaming shows the first words as soon as it is ready.

```python
stream = client.chat.completions.create(
    model="qwen3:8b",
    messages=[{"role": "user", "content": "Write a haiku about trains."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

```javascript
const stream = await client.chat.completions.create({
  model: "qwen3:8b",
  messages: [{ role: "user", content: "Write a haiku about trains." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

With curl, add `"stream": true` and `-N` so curl prints each piece as it arrives:

```bash
curl -N https://01k8y4000d3m00000000000011.endpoint.bitsrun.com/v1/chat/completions \
  -H "Authorization: Bearer $BITSRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3:8b", "stream": true, "messages": [{"role": "user", "content": "Count to five."}]}'
```

> [!NOTE]
> While a model starts or thinks for a long time, a stream may carry short comment lines (they start with `:`). The OpenAI SDKs ignore them. If you parse the stream yourself, skip lines that start with `:`.

Closing the connection does not stop the model. It finishes the answer, up to `max_tokens`, and that answer is billed like any other. To keep an answer you might abandon cheap, set `max_tokens` to what you need.

## Fields you can send

BitsRun passes these fields to the model and drops any others:

- `messages`, `stream`, `stream_options`
- `max_tokens` or `max_completion_tokens` (the longest reply, capped at 32,768 tokens)
- `temperature`, `top_p`, `seed`, `stop`, `presence_penalty`, `frequency_penalty`
- `n` (number of replies, up to 4), `logprobs`, `top_logprobs`
- `tools`, `tool_choice`, `parallel_tool_calls` (see [Tool calling](https://bitsrun.com/docs/api/tools/))
- `response_format` (for JSON output)
- `chat_template_kwargs` with `enable_thinking` only (see below)

A token is a piece of a word, about four characters of English text on average.

## Turn thinking on or off

Some models, such as Qwen3, think out loud before they answer. Thinking gives better answers to hard questions but takes longer. Where the model supports it, switch it off for quick replies:

```python
reply = client.chat.completions.create(
    model="qwen3:8b",
    messages=[{"role": "user", "content": "What is the capital of Peru?"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
```

## Plain completions

`/v1/completions` continues a piece of text instead of a conversation. It accepts `prompt`, `suffix`, `echo` and the same sampling fields. Use chat unless you know you need this.

```bash
curl https://01k8y4000d3m00000000000011.endpoint.bitsrun.com/v1/completions \
  -H "Authorization: Bearer $BITSRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3:8b", "prompt": "The three primary colors are", "max_tokens": 30}'
```

## LangChain

LangChain's OpenAI chat model works with the same two settings:

```python
import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://01k8y4000d3m00000000000011.endpoint.bitsrun.com/v1",
    api_key=os.environ["BITSRUN_API_KEY"],
    model="qwen3:8b",
)
print(llm.invoke("Give me one tip for writing clear emails.").content)
```

## Ollama clients

GGUF models (deployed from an Ollama name or a `.gguf` file, served by llama.cpp) also speak Ollama's own API. Models served by vLLM or TEI answer only the OpenAI paths. Use the endpoint host without `/v1` and send your key in the `Authorization` header. The Ollama Python library takes it in `headers`:

```python
import os
from ollama import Client

client = Client(
    host="https://01k8y4000d3m00000000000011.endpoint.bitsrun.com",
    headers={"Authorization": f"Bearer {os.environ['BITSRUN_API_KEY']}"},
)
reply = client.chat(model="qwen3:8b", messages=[{"role": "user", "content": "Hello"}])
print(reply.message.content)
```

The `ollama` command line app (`ollama run`) cannot send that header, so it gets a `401`. Use a library or call the API directly.

Or call it directly:

```bash
curl https://01k8y4000d3m00000000000011.endpoint.bitsrun.com/api/chat \
  -H "Authorization: Bearer $BITSRUN_API_KEY" \
  -d '{"model": "qwen3:8b", "messages": [{"role": "user", "content": "Hello"}], "stream": false}'
```

Only the chat, generate, embed and tags paths answer. Commands that change the server, such as `pull`, `create` or `delete`, are not available.

## What can go wrong

- **`model_starting`**: the endpoint was asleep and took longer than 8 minutes to start (in a `200` body), or you sent `X-BitsRun-Wait: 0` (a `503`). Send the request again.
- **`400 context_too_long`**: the conversation is longer than the model can read at once. Drop older messages, or raise **Memory length** under [Performance](https://bitsrun.com/docs/endpoints/performance/).
- **`401 unauthorized`**: the key is missing, mistyped or revoked. Check the `Authorization: Bearer` header.
- **`429 rate_limited`**: your account has too many requests open at once. See [Limits](https://bitsrun.com/docs/api/limits/).

Every code is explained on [Error codes](https://bitsrun.com/docs/troubleshooting/errors/).