dodil data vector collection

Manage vector collections. Four subcommands — add, get, list, delete. A vector collection is a pipeline with a vector facet, so get and delete take a pipeline id. Mirrors Collections — API Reference.

Persistent flag on the whole group: --bucket / -b.

There is no add-manual. The old external-embedding mode was retired with the Milvus/VBase decommission; AddVectorCollection no longer exists and EMBEDDING_SOURCE_EXTERNAL is read-only legacy. For bring-your-own embeddings, create a table with a vector(N) column and push over the wire adapters — see API Reference → BYO embeddings and Recipes → External Collection.

dodil data vector collection add

dodil data vector collection add [name] -b BUCKET -t TEMPLATE_ID [-d DESC] [--set k=v]

Creates a template-driven collection. K3 writes three rows — the vector destination, the index pipeline, and (best-effort, by the *_index → *_search convention) the search pipeline — and talks to no data plane. The physical collection materializes lazily on first ingest.

Schema is owned by the template’s ScriptContract and the caller cannot override it: dimensions, distance_metric, embedding_type, sparse_mode and embed_model are all resolved server-side. Collections provision on demand — no engine setup precedes this call.

# Text / PDF / docx → embeddings (canonical RAG)
dodil data vector collection add docs -b kb-prod \
  --description "RAG corpus" \
  --template text_embedding_index
 
# Code → embeddings (AST-aware chunking)
dodil data vector collection add code -b kb-prod \
  --description "Source-code search" \
  --template code_embedding_index
 
# Multimodal (images / video / audio / PDF page renders)
dodil data vector collection add assets -b kb-prod \
  --description "Mixed-media library" \
  --template visual_embedding_index

The response is a Pipeline with its destination attached. Capture pipelineId — get, delete and every ingest command key off it.

--set only sends strings. vcAddCmd wraps each value with structpb.NewStringValue, so a template input that needs a number, boolean or array (e.g. object_embedding_index’s labels: [string]) cannot be supplied from the CLI. Use the API with a typed templateInputs map for those.

The two inputs that genuinely are caller-tunable — chunk_size (default 500) and chunk_overlap (default 50) — are read from template_inputs, so --set chunk_size=800 works.

Pipeline.name is the template id, not your collection name. The handler sets the pipeline row’s name to template_id (create_vector.rs:254), so collection add prints back Vector collection 'text_embedding_index' (pipeline …) created and collection list shows NAME and TEMPLATE as the same string. Your name is on the destination — read it from .destination.name. This is a source bug, already logged.

It does not create an ingest rule

CreateVectorPipeline derives no rule; callers own rule scope. Nothing is ever embedded until you add one:

dodil data ingest add docs-corpus -b kb-prod \
  -c "$PIPELINE_ID" \
  -i '**/*.pdf' -i '**/*.txt'

-c / --collection on ingest add sets CreateRuleRequest.pipeline_id — pass the pipeline id. Browse templates with dodil data vector templates to see the acceptedExtensions your globs should mirror — see templates.

dodil data vector collection get

dodil data vector collection get [pipeline-id] -b BUCKET

Takes a pipeline id, not a name and not a collection id. Always prints JSON.

dodil data vector collection get "$PIPELINE_ID" -b kb-prod \
  | jq '{
      pipelineId, scriptumTemplate,
      name: .destination.name,
      status: .destination.status,
      vector: .destination.vector
    }'

Useful for confirming dimensions, embedModel and physicalName before searching on the data plane.

dodil data vector collection list

dodil data vector collection list -b BUCKET

ListPipelines with facet = PIPELINE_FACET_VECTOR. Table output is PIPELINE_ID · NAME · TEMPLATE — and per the note above, NAME and TEMPLATE will be the same string. Use -o json and read .destination.name for the name you chose.

dodil data vector collection list -b kb-prod -o json \
  | jq '.pipelines[] | {pipelineId, name: .destination.name, dims: .destination.vector.dimensions, model: .destination.vector.embedModel}'

Note the response key is pipelines, not collections. Over HTTP the facet is a query parameter and only vector, table, object and graph are accepted — warehouse and tables are rejected with INVALID_ARGUMENT.

dodil data vector collection delete

dodil data vector collection delete [pipeline-id] -b BUCKET

Deletes the pipeline row. This is wiring only — it does not drop the physical vector collection on the plane, which is plane state you drop yourself through the tables-gateway.

# 1. Delete the ingest rule you created
RULE_ID=$(dodil data ingest list -b kb-prod -p "$PIPELINE_ID" -o json | jq -r '.rules[0].ruleId')
dodil data ingest delete "$RULE_ID" -b kb-prod
 
# 2. Delete the pipeline
dodil data vector collection delete "$PIPELINE_ID" -b kb-prod

For a whole recipe-installed chain, POST /:bucket/pipelines/_batch-delete (BatchDeleteArtifacts) unwinds rules → pipelines → destinations in dependency order and is idempotent.

Common gotchas

See also

• Collections — API Reference — full surface incl. typed template_inputs
• dodil data vector templates — discover templates for add
• dodil data vsearch — KNN on the data plane
• Core Concepts → Collection — the pipeline / destination / VectorConfig model
• Recipes → Pipeline Collection + External Collection — end-to-end worked examples