# Cold starts and what is billed

> A sleeping endpoint wakes on the next request; the wait (seconds on a warm GPU, a few minutes on a new one) is never billed, and billing runs only while it is awake.

Source: https://bitsrun.com/docs/get-started/cold-starts/
Updated: 2026-11-09

Endpoints sleep when nobody uses them, so you pay nothing at night or on quiet days. The price of sleeping is a short wait the next time someone calls: a cold start.

## What one wake costs


- Asleep: Free
- Starting a GPU: Free
- Loading the model: Free
- Answering requests: Billed per second
- Waiting before sleep: Billed, you pick 60 s to 1 h


- **Free:** while asleep, while a GPU machine starts, while the model downloads and loads, and while you wait for a free GPU.
- **Billed by the second:** from the moment the model is ready until it goes back to sleep. That includes the wait after the last request (the **Sleep after** time), because the GPU stays on, ready to answer at once.

## How long a cold start takes

| Model kind | Warm GPU (ran recently) | New GPU machine |
|---|---|---|
| GGUF models on llama.cpp, for example `qwen3:8b` | A few seconds to about 40 seconds | About 2 to 3 minutes |
| Embedding models on Text Embeddings Inference | A few seconds to about 40 seconds | About 2 to 3 minutes |
| Safetensors chat models on vLLM | About 40 seconds | About 3 to 4 minutes |

Large models take longer, mostly to download. When every GPU of a size is busy, the start waits for a free one and keeps trying in every zone and region. If none is free for 30 minutes, it stops trying and tells you (`no_capacity`). You are not billed while it waits.

## What your code sees

- **Every request waits** on the open connection for up to 8 minutes, then answers normally. You do not need to retry.
- **While it waits,** BitsRun sends something your client ignores, so no proxy closes the connection: keep-alive comments for streaming requests, spaces before the JSON for other requests (JSON parsers skip them), and empty progress lines for Ollama native streams.
- **If the model is still not ready after 8 minutes** (a very large model or a slow download), or it fails to start, the answer has status `200` and an error body with `"code": "model_starting"` or `"model_unavailable"`. The wake carries on, so send the request again.
- **Up to 4 requests wait at once** per endpoint for your account. More than that get `503 model_starting` with `Retry-After: 30`, so a retry loop cannot pile up answers nobody reads. Requests that time out on your side still run once the model is ready, and are billed.
- **Prefer a 503?** Send the header `X-BitsRun-Wait: 0`. A request that finds the model asleep then waits about 45 seconds and gets `503` with `"code": "model_starting"` and `Retry-After: 30`. Use it when your code retries on its own or must see real status codes.

Your own retry loop in Python, with the header:

```python
import time
from openai import OpenAI, APIStatusError

# X-BitsRun-Wait: 0 answers 503 while the model starts, so this loop decides how long to wait.
client = OpenAI(base_url="https://<id>.endpoint.bitsrun.com/v1", api_key="brk_...",
                default_headers={"X-BitsRun-Wait": "0"}, max_retries=0)

def ask(**kwargs):
    for attempt in range(16):
        try:
            return client.chat.completions.create(**kwargs)
        except APIStatusError as e:
            if e.status_code != 503:
                raise
            time.sleep(30)
    raise RuntimeError("The endpoint did not wake in 20 minutes")
```

## Avoiding the wait

- Select **Start** on the endpoint card a few minutes before you need it.
- Choose a longer **Sleep after** (up to 1 hour) for bursty traffic. You pay while it waits.
- Turn on **Priority** so a cold start never waits for a spare GPU.
- Turn on **Always on** for no cold starts at all.
- Tick "Notify me when it is ready" on the card, or turn on the ready email.

These are explained in [Sleep, always-on and priority](https://bitsrun.com/docs/endpoints/sleep-and-always-on/) and [Notifications](https://bitsrun.com/docs/endpoints/notifications/).

## A worked example

A support chat on an NVIDIA L4 ($0.000264 a second) with **Sleep after** set to 2 minutes gets a burst of questions that keeps it answering for 10 minutes.

- Cold start, 3 minutes: free.
- Answering, 10 minutes (600 seconds): $0.16.
- Waiting before sleep, 2 minutes (120 seconds): $0.03.
- Total for the burst: about $0.19.