Skip to Content
We are live but in Staging 🎉
PipelinesAPI ReferencePipelines

Pipelines — API Reference

Package: dodil.k3.pipeline.v1 · Service: PipelineService

A pipeline is a Scriptum template + options + optional destination — the recipe that rules trigger against an object. Pipelines are bucket-scoped and first-class rows; rules reference them by pipeline_id. See Core Concepts → Pipeline.

RPCHTTP
CreatePipelinePOST /:bucket/pipelines
GetPipelineGET /:bucket/pipelines/:pipeline_id
ListPipelinesGET /:bucket/pipelines
UpdatePipelinePATCH /:bucket/pipelines/:pipeline_id
DeletePipelineDELETE /:bucket/pipelines/:pipeline_id

gRPC setup — grpcurl, endpoints, reflection, and field-name casing — is covered once in Conventions → Using gRPC.

CreatePipeline

Two ways to specify the script — oneof script_source:

  • scriptum_template — reference an existing Scriptum script by name.
  • spawn_from_template — pass a template ID; the server creates a fresh Scriptum script from it and persists the resulting script name on the pipeline row.

Request

Reference an existing Scriptum script:

curl -sS -X POST "https://k3.dev.dodil.io/kb-prod/pipelines" \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "name": "embed-contracts",
    "scriptumTemplate": "text_embedding_index",
    "options": { "chunk_size": "1000", "chunk_overlap": "150" },
    "storeEntityId": "ent_a1b2..."
  }'

Spawn a new script from a template ID:

curl -sS -X POST "https://k3.dev.dodil.io/kb-prod/pipelines" \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "name": "contracts-pii",
    "spawnFromTemplate": "entity_pii_extraction",
    "storeEntityId": "ent_c3d4..."
  }'

Response

A Pipeline row — store_entity_kind is derived server-side from the destination (vector, warehouse, or "" for free pipelines). See Core Concepts → Pipeline.

GetPipeline

Request

curl -sS "https://k3.dev.dodil.io/kb-prod/pipelines/pipe_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN"

Response

A Pipeline row — see Core Concepts → Pipeline.

ListPipelines

Filter by destination kind (vector / warehouse / free) or by specific store_entity_id. Cursor pagination.

Request

All pipelines:

curl -sS "https://k3.dev.dodil.io/kb-prod/pipelines?page_size=50" \
  -H "Authorization: Bearer $DODIL_TOKEN"

Only vector pipelines:

curl -sS "https://k3.dev.dodil.io/kb-prod/pipelines?store_entity_kind=vector" \
  -H "Authorization: Bearer $DODIL_TOKEN"

All pipelines bound to one collection:

curl -sS "https://k3.dev.dodil.io/kb-prod/pipelines?store_entity_id=ent_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN"

Response

{
  "pipelines": [
    {
      "pipelineId": "pipe_a1b2...",
      "bucket": "kb-prod",
      "name": "embed-contracts",
      "scriptumTemplate": "text_embedding_index",
      "options": { "chunk_size": "1000", "chunk_overlap": "150" },
      "storeEntityId": "ent_a1b2...",
      "storeEntityKind": "vector"
    }
  ],
  "pagination": { "nextPageToken": "", "totalCount": "1" }
}

UpdatePipeline

Patch-style. optional fields use proto3 presence; the options map uses the StringMap wrapper from dodil.k3.common.v1 so you can distinguish “leave alone” (absent) from “clear” (present + empty) from “replace” (present + non-empty).

Request

Rename + tweak options:

curl -sS -X PATCH "https://k3.dev.dodil.io/kb-prod/pipelines/pipe_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "pipelineId": "pipe_a1b2...",
    "name": "embed-contracts-v2",
    "options": { "entries": { "chunk_size": "1500", "chunk_overlap": "200" } }
  }'

Re-bind to a different collection:

curl -sS -X PATCH "https://k3.dev.dodil.io/kb-prod/pipelines/pipe_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "pipelineId": "pipe_a1b2...",
    "storeEntityId": "ent_new..."
  }'

Response

A Pipeline row — see Core Concepts → Pipeline.

DeletePipeline

Request

curl -sS -X DELETE "https://k3.dev.dodil.io/kb-prod/pipelines/pipe_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN"

Response

Empty (DeletePipelineResponse {}).

Deleting a pipeline doesn’t cascade to rules — any rule whose pipeline_id was this pipeline will keep existing with its binding empty (server can’t resolve the deleted pipeline). New ingest jobs for those rules will fail. Either delete dependent rules first or re-bind them with UpdateRule.

See also

CredentialsTemplates