LFM2.5-ColBERT-350M

← Back to Liquid Nanos LFM2.5-ColBERT-350M is a late-interaction retrieval model. It creates 128-dimensional vectors per token and scores query/document matches with MaxSim, which improves retrieval quality and generalization at the cost of a larger index.

HF GGUF MLX

Use LFM2.5-ColBERT-350M when you want stronger retrieval or reranking quality and can afford a larger per-token index. Use LFM2.5-Embedding-350M when you need the smallest, fastest dense-vector index.

Specifications

Property	Value
Parameters	~353M
Type	Late interaction
Document Length	512 tokens
Query Length	32 tokens
Output	128-dimensional vector per token
Similarity	MaxSim
Supported Languages	English, Spanish, German, French, Italian, Portuguese, Arabic, Swedish, Norwegian, Japanese, Korean

High-Quality Retrieval

Better matching from token-level interactions.

Reranking

Reorder candidates from a first-stage retriever.

Enterprise RAG

Strong multilingual document matching.

Quick Start

Run the model with Sentence Transformers (MultiVectorEncoder), with PyLate for indexing, retrieval, and reranking, or with llama.cpp through the GGUF checkpoint.

Sentence Transformers
PyLate
Reranking
GGUF

Install:

pip install "sentence-transformers>=6.0.0"

Run:

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("LiquidAI/LFM2.5-ColBERT-350M", trust_remote_code=True)

query = "Which planet is known as the Red Planet?"
documents = [
    "Venus is often called Earth's twin because of its similar size and proximity.",
    "Mars, known for its reddish appearance, is often referred to as the Red Planet.",
    "Jupiter, the largest planet in our solar system, has a prominent red spot.",
    "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.",
]

query_embeddings = model.encode_query([query])
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (32, 128) (17, 128)

# MaxSim late-interaction scoring (the Mars document ranks highest)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[27.1621, 28.2578, 27.7266, 28.1992]])

Install:

pip install -U pylate

Index and retrieve documents:

from pylate import indexes, models, retrieve

model = models.ColBERT(
    model_name_or_path="LiquidAI/LFM2.5-ColBERT-350M",
    trust_remote_code=True,
)
model.tokenizer.pad_token = model.tokenizer.eos_token

index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
    override=True,
)

documents_ids = ["1", "2", "3"]
documents = [
    "Paris is the capital of France.",
    "Tokyo is the capital of Japan.",
    "Berlin is the capital of Germany.",
]

document_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,
    show_progress_bar=True,
)
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=document_embeddings,
)

query_embeddings = model.encode(
    ["Which city is Japan's capital?"],
    batch_size=32,
    is_query=True,
    show_progress_bar=True,
)

retriever = retrieve.ColBERT(index=index)
results = retriever.retrieve(queries_embeddings=query_embeddings, k=10)
print(results)

from pylate import models, rank

model = models.ColBERT(
    model_name_or_path="LiquidAI/LFM2.5-ColBERT-350M",
    trust_remote_code=True,
)

queries = ["Which city is Japan's capital?"]
documents = [[
    "Paris is the capital of France.",
    "Tokyo is the capital of Japan.",
    "Berlin is the capital of Germany.",
]]
document_ids = [["fr", "jp", "de"]]

query_embeddings = model.encode(queries, is_query=True)
document_embeddings = model.encode(documents, is_query=False)

reranked = rank.rerank(
    documents_ids=document_ids,
    queries_embeddings=query_embeddings,
    documents_embeddings=document_embeddings,
)
print(reranked)

Start llama-server:

llama-server -hf LiquidAI/LFM2.5-ColBERT-350M-GGUF --embeddings

The /embedding endpoint returns one vector per token. The client applies the ColBERT pre- and post-processing defined in the model’s config_sentence_transformers.json:

Prefix queries with [Q] and documents with [D] .
Pad queries to 32 tokens with the pad token. Truncate documents to 512 tokens and drop the vectors of punctuation tokens listed in skiplist_words.
L2-normalize each token vector, then score query/document pairs with MaxSim.

The GGUF model card has a complete client script (colbert-rerank.py) that does this against a local llama-server.

Models

Fine-tuning

Edge Inference

GPU Inference

Cloud Inference

Guides

Help

Specifications

High-Quality Retrieval

Reranking

Enterprise RAG

Quick Start

​Specifications

High-Quality Retrieval

Reranking

Enterprise RAG

​Quick Start

Specifications

Quick Start