Vector

The Vector data engine is DataK³’s vector data plane: a vector collection is a table with a vector<n> column on the tables-gateway, written and read with the protocol your client already speaks. There is no separate vector service and no engine to provision — a vector<n> column is just another column type on the tabled plane.

Speak SQL, Postgres, Qdrant or Pinecone to it. KNN is ORDER BY "col" <=> '[…]' LIMIT k over the pg wire (pg.uk-lon-1.dodil.io:5432, sslmode=require), or a stock Qdrant / Pinecone client at qdrant.uk-lon-1.dodil.io / pinecone.uk-lon-1.dodil.io — the api-key header carries the db id (the bucket name), not a credential. See Connect & wire adapters.

Search by meaning (semantic search)

Store an embedding next to each row, embed the user’s question with the same model, and order by distance. It’s one SQL statement, and because it’s the same table, the same WHERE clause that scopes a person’s rows also scopes their search results.


# try it from the CLI: embeds the text with an Ignite model, then runs KNN
dodil data vsearch -b <bucket> --table docs --column embedding \
  --metric cosine --top-k 10 --text "late delivery complaints"

From an app:


# 1. embed the question (OpenAI-compatible; same model as the stored vectors)
q = openai.embeddings.create(model="arctic-embed-m-v2", input=question).data[0].embedding
 
# 2. KNN over the pg wire, scoped to the caller like any other query
rows = await db.fetch(
    """SELECT id, title, embedding <=> $1 AS distance
         FROM docs
        WHERE tenant_id = $2
        ORDER BY embedding <=> $1
        LIMIT 10""",
    str(q), caller["tenant"])

Embeddings: Ignite Models → Embeddings , from the app’s own service account with ignite.model-user. Use the same model for stored vectors and queries.
Auto-embed uploaded files (PDFs, images) with a vector pipeline instead of writing vectors yourself.
Speed and quality: add an ANN index once tables grow.
One-call managed search (dodil data search / POST /:bucket/search/vector) is not working yet: it returns a per-collection failReason until its read path moves off Milvus, and it takes no row filter. Use the KNN path above. See Feature Status.
Dense only. There is no dense+BM25 hybrid on the plane today.
Multi-user apps: keep the tenant/owner filter in the KNN query. See Keep each customer’s data separate .

What lives here

The SQL data model — the vector<n> / halfvec<n> / bit<n> types, distance metrics and their operator / opclass mapping, and the HNSW / IVFFlat ANN index. See Core Concepts.
KNN over the wire — pgvector operators in SQL (<-> L2, <#> inner product, <=> cosine), the Qdrant / Pinecone adapters, or dodil data vsearch over the typed TablesVector.QueryVectors facet.
The managed search route — POST /:bucket/search/vector, the control-plane dispatcher that embeds a text/file query server-side and searches pipeline-created collections. See API Reference → Search.

What does NOT live here — auto-embed-on-ingest

Creating a collection that chunks + embeds + indexes uploaded objects for you is a control-plane / ingest concern, not a data-engine one. That whole surface — CreateVectorPipeline, VectorConfig, dodil data vector collection add, the *_embedding_index templates, and the ingest rules that feed them — lives in Pipelines:

Pipelines → Vector Collections (API) — CreateVectorPipeline and the pipeline CRUD that reads/deletes a collection
Pipelines → dodil data vector collection (CLI)
Pipelines → Pipeline Collection recipe — the canonical RAG-ingest flow
Pipelines → External Collection recipe — BYO embeddings over the wire adapters
Pipelines → Templates — the embedding template catalog

A pipeline-created collection is still a vector<n> table underneath — once it exists, you query it with the same SQL/KNN paths documented here.

Two ways vectors arrive in a table

Where vectors come from	How you set it up	Read them with
You write them (BYO embeddings)	`CREATE TABLE … (id VARCHAR PRIMARY KEY, embedding vector(N))`, then upsert with a stock Qdrant / Pinecone client or SQL `INSERT`	SQL KNN · `dodil data vsearch` · Qdrant / Pinecone search
A pipeline writes them (auto-embed)	`CreateVectorPipeline` with a `*_embedding_index` template + an ingest rule	the managed search route · or SQL KNN on the physical `k3_<uuid>` table

In this section

Quickstart — create a vector<n> table, insert, and run SQL KNN in 5 minutes
Core Concepts — the SQL data model: types, metrics, operators, opclasses, the ANN index
API Reference — the managed search route + the tabled data path (TablesVector)
CLI Guide — dodil data vsearch
Recipes — multi-collection search, hybrid + rerank, multimodal search