Rules — API Reference

Package: dodil.data.ingest.v1 · Service: IngestService

A rule is a trigger: glob/MIME/size filters that bind a source to a pipeline. When an object matches, K3 spawns an ingest job that runs the bound pipeline. See Core Concepts → Rule.

Nothing ingests until a rule exists. Creating a pipeline — including via CreateVectorPipeline, CreateTablePipeline or CreateObjectPipeline — writes no ingest rule. The facet creators deliberately leave rule scope to the caller, so a collection or table with no rule bound to it is real, listable, and will never receive a single object. CreateRule is the step that turns a pipeline on.

RPC	HTTP
`CreateRule`	`POST /:bucket/rules`
`GetRule`	`GET /:bucket/rules/:rule_id`
`ListRules`	`GET /:bucket/rules`
`UpdateRule`	`PATCH /:bucket/rules/:rule_id`
`DeleteRule`	`DELETE /:bucket/rules/:rule_id`

gRPC setup — grpcurl, endpoints, reflection, and field-name casing — is covered once in Conventions → Using gRPC.

`CreateRule`

Request

dodil data


dodil data ingest add pdf-contracts \
  -b kb-prod \
  -s src_a1b2... \
  -c pipe_a1b2... \
  -i 'contracts/**/*.pdf'

-c/--collection takes a pipeline id, not a collection id. The CLI assigns it straight to CreateRuleRequest.pipeline_id; the flag name predates the pillar split and was never renamed. Pass the value you would send as pipelineId.

The sibling -p/--pipeline-type flag (documented as embedding, extraction) is declared but never read — it is dead and setting it changes nothing.

The CLI covers only a subset of the request: it always sends enabled: true and exposes no --exclude, --include-mime, --min-size, --max-size, --priority or --description on add. Those are settable afterwards via dodil data ingest update, or up front over HTTP/gRPC.

HTTP


curl -sS -X POST "https://api.data.dodil.io/kb-prod/rules" \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "sourceId": "src_a1b2...",
    "name": "pdf-contracts",
    "description": "All PDFs in contracts/",
    "includePatterns": ["contracts/**/*.pdf"],
    "excludePatterns": ["contracts/**/draft-*"],
    "includeMimeTypes": ["application/pdf"],
    "minSizeBytes": "1024",
    "maxSizeBytes": "0",
    "enabled": true,
    "priority": 100,
    "pipelineId": "pipe_a1b2..."
  }'

gRPC


rpc CreateRule(CreateRuleRequest) returns (IngestRule);
 
message CreateRuleRequest {
  string bucket = 1;
  string source_id = 2;
  string name = 3;
  string description = 4;
  repeated string include_patterns = 5;
  repeated string exclude_patterns = 6;
  repeated string include_mime_types = 7;
  repeated string exclude_mime_types = 8;
  int64 min_size_bytes = 9;
  int64 max_size_bytes = 10;                  // 0 = no limit
  bool enabled = 11;
  int32 priority = 12;                        // higher = matched first
  string pipeline_id = 13;                    // required; must exist in same bucket
  optional string recipe_name = 14;           // provenance stamp for recipe installs
}


grpcurl \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -d '{
    "bucket": "kb-prod",
    "source_id": "src_a1b2...",
    "name": "pdf-contracts",
    "description": "All PDFs in contracts/",
    "include_patterns": ["contracts/**/*.pdf"],
    "exclude_patterns": ["contracts/**/draft-*"],
    "include_mime_types": ["application/pdf"],
    "min_size_bytes": "1024",
    "max_size_bytes": "0",
    "enabled": true,
    "priority": 100,
    "pipeline_id": "pipe_a1b2..."
  }' \
  $K3_GRPC \
  dodil.data.ingest.v1.IngestService/CreateRule

Matching is conjunctive across axes (path globs ∧ MIMEs ∧ size). Within an axis, includes OR together; excludes always win. See Core Concepts → Glob pattern syntax for the full grammar (tokens, case-insensitivity, MIME wildcard, evaluation order) — important: patterns are ASCII case-insensitive.

Field semantics worth stating outright:

Field	Behaviour
`max_size_bytes`	`0` means no limit, not “reject everything”. `min_size_bytes` of `0` likewise means no floor.
`priority`	Higher is matched first. It is an `int32`, so negatives are legal and sort last.
`enabled`	A disabled rule stays in the table and keeps its id — it simply stops firing. Prefer `enabled: false` over `DeleteRule` when you may want the rule back.
`pipeline_id`	Required, and the pipeline must exist in the same bucket.
`recipe_name`	Provenance only — a stable `recipe.id` stamped when the rule came from a recipe install. Empty for hand-installed rules. Do not route on it.

One more matcher applies that is not a field on this message: ingest rules skip source objects under any object destination’s excluded_prefix. That is what stops an object pipeline re-ingesting its own output forever, and it applies regardless of what your include patterns say.

Response

An IngestRule row — see Core Concepts → Rule.

`GetRule`

Request

HTTP


curl -sS "https://api.data.dodil.io/kb-prod/rules/rule_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN"

Response

An IngestRule row with server-derived binding (pipeline name, kind, destination) for display. Always route on pipeline_id — binding is empty if the pipeline was deleted. See Core Concepts → Rule.


message PipelineBinding {
  string pipeline_name = 1;
  string pipeline_kind = 2;                   // "vector" | "warehouse" | "free"
  string destination_id = 3;                  // empty for free pipelines
  string destination_kind = 4;
  string destination_name = 5;                // collection or table name
}

PipelineBinding.pipeline_kind reports the stored vocabulary — a table pipeline reads "warehouse", not "table". That differs from the ?facet= query value on ListPipelines, which is table and explicitly rejects warehouse. Note also that IngestJob.pipeline_kind adds "object" to the same set, while this message’s comment lists only three.

`ListRules`

Filter by source_id and/or pipeline_id.

ListRules does not paginate over HTTP. The handler reads only source_id and pipeline_id and sends pagination: None, so ?page_size= and ?page_token= are ignored and the response carries every matching rule. Pagination is reachable over gRPC only.

Request

dodil data


dodil data ingest list -b kb-prod
dodil data ingest list -b kb-prod -s src_a1b2...
dodil data ingest list -b kb-prod -p pipe_a1b2...

Rules are the dodil data ingest command group: add, get, list, update, delete.

HTTP

All rules tied to one source:


curl -sS "https://api.data.dodil.io/kb-prod/rules?source_id=src_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN"

All rules tied to one pipeline:


curl -sS "https://api.data.dodil.io/kb-prod/rules?pipeline_id=pipe_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN"

gRPC


rpc ListRules(ListRulesRequest) returns (ListRulesResponse);
 
message ListRulesRequest {
  string bucket = 1;
  optional string source_id = 2;
  optional string pipeline_id = 3;            // filter by pipeline
  dodil.data.common.v1.PaginationRequest pagination = 4;
}

All rules tied to one source:


grpcurl \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -d '{
    "bucket": "kb-prod",
    "source_id": "src_a1b2..."
  }' \
  $K3_GRPC \
  dodil.data.ingest.v1.IngestService/ListRules

All rules tied to one pipeline:


grpcurl \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -d '{
    "bucket": "kb-prod",
    "pipeline_id": "pipe_a1b2..."
  }' \
  $K3_GRPC \
  dodil.data.ingest.v1.IngestService/ListRules

Response

HTTP


{
  "rules": [
    {
      "ruleId": "rule_a1b2...",
      "bucket": "kb-prod",
      "sourceId": "src_a1b2...",
      "name": "pdf-contracts",
      "includePatterns": ["contracts/**/*.pdf"],
      "enabled": true,
      "priority": 100,
      "pipelineId": "pipe_a1b2..."
    }
  ],
  "pagination": { "nextPageToken": "", "totalCount": "1" }
}

`UpdateRule`

Patch-style. Repeated fields use the StringList wrapper from dodil.data.common.v1 so absent / empty / non-empty have distinct meanings.

Request

HTTP

Disable a rule without deleting it:


curl -sS -X PATCH "https://api.data.dodil.io/kb-prod/rules/rule_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "bucket": "kb-prod", "ruleId": "rule_a1b2...", "enabled": false }'

Tighten the include set:


curl -sS -X PATCH "https://api.data.dodil.io/kb-prod/rules/rule_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "ruleId": "rule_a1b2...",
    "includePatterns": { "values": ["contracts/2026/*.pdf"] }
  }'

Re-bind to a new pipeline:


curl -sS -X PATCH "https://api.data.dodil.io/kb-prod/rules/rule_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "bucket": "kb-prod", "ruleId": "rule_a1b2...", "pipelineId": "pipe_new..." }'

Response

An IngestRule row — see Core Concepts → Rule.

`DeleteRule`

Request

HTTP


curl -sS -X DELETE "https://api.data.dodil.io/kb-prod/rules/rule_a1b2..." \
  -H "Authorization: Bearer $DODIL_TOKEN"

Response

Empty (DeleteRuleResponse {}).

Deleting a rule stops new ingests under it. Existing jobs stay in the system with their rule_id pointing at the deleted row.

Rules — API Reference

CreateRule

Request

dodil data

HTTP

gRPC

Response

GetRule

Request

HTTP

gRPC

Response

ListRules

Request

dodil data

HTTP

gRPC

Response

HTTP

gRPC

UpdateRule

Request

HTTP

gRPC

Response

DeleteRule

Request

HTTP

gRPC

Response

See also

`CreateRule`

`GetRule`

`ListRules`

`UpdateRule`

`DeleteRule`