Embeddings

An embedding is a list of numbers (a vector) that captures what a piece of text means. Texts with similar meaning get vectors that sit close together. Apps use them for semantic search, for RAG (retrieval-augmented generation: finding the right documents to hand a chat model), for grouping similar items and for spotting duplicates.

Deploy an embedding model

Deploy an embedding model like any other model, for example the Hugging Face repo Snowflake/snowflake-arctic-embed-l-v2.0 or Qwen/Qwen3-Embedding-0.6B:

bitsrun deploy hf:Snowflake/snowflake-arctic-embed-l-v2.0 --name "Docs search"

Embedding models are small, so they fit the smallest GPU size (NVIDIA L4). The card shows Model type: Embeddings once it is created. See Deploy your first model for the dashboard steps.

Get vectors

Send one string or a list of strings as input:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://01k8y4000d3m00000000000012.endpoint.bitsrun.com/v1",
    api_key=os.environ["BITSRUN_API_KEY"],
)
result = client.embeddings.create(
    model="snowflake-arctic-embed-l-v2.0",
    input=["How do I reset my password?", "Where can I change my login email?"],
)
for item in result.data:
    print(len(item.embedding), item.embedding[:4])

curl https://01k8y4000d3m00000000000012.endpoint.bitsrun.com/v1/embeddings \
  -H "Authorization: Bearer $BITSRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "arctic", "input": "How do I reset my password?"}'

BitsRun passes input, encoding_format and dimensions (for models that can shorten their vectors) and drops other fields.

Use them in LangChain

import os
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    base_url="https://01k8y4000d3m00000000000012.endpoint.bitsrun.com/v1",
    api_key=os.environ["BITSRUN_API_KEY"],
    model="snowflake-arctic-embed-l-v2.0",
    check_embedding_ctx_length=False,
)
vector = embeddings.embed_query("How do I reset my password?")

check_embedding_ctx_length=False stops LangChain from splitting text with an OpenAI-only tokenizer before it sends it.

Tips

• Use one model for both sides. Vectors from different models cannot be compared. Embed your documents and your search queries with the same endpoint.
• Send batches. One request with 64 short texts is faster than 64 requests.
• Keep it awake while you index. A big import sends many requests in a row, so the endpoint stays awake and you pay only for the time it works. It sleeps again after its idle time.
• Give each app its own key. A key limited to the embedding endpoint cannot reach your chat endpoints. See API keys.

What can go wrong

• 404 not_found on /v1/chat/completions: embedding endpoints only answer /v1/embeddings and /v1/models.
• 400 context_too_long or a cut-off result: each model reads a limited amount of text at once. Split long documents into chunks of a few hundred words.
• model_starting: the endpoint took longer than 8 minutes to start, or you sent X-BitsRun-Wait: 0 (a 503). Send the request again; embedding models usually start in seconds on a warm GPU.