Recipes
Self-contained, runnable scenarios using K3’s production Scriptum catalog. Every recipe is grounded in a real template (no inventions) and uses the same dodil data flow you saw in the Quickstart.
What’s here
| Recipe | Template used | Destination | When you need it |
|---|---|---|---|
| PDF → Vector | text_embedding_index | Vector collection | Auto-index PDFs / docx / HTML for semantic search (the canonical RAG ingest) |
| Pipeline Collection | text_embedding_index | Vector collection | The full auto-embed collection flow — create, bind a rule, ingest, search |
| External Collection | — (BYO embeddings) | Vector collection (a plain table) | Compute embeddings yourself and write vectors over the Qdrant / Pinecone adapters or SQL |
| Documents → Warehouse | entity_pii_extraction | Warehouse table | Extract structured entities + PII into rows you can query with SQL |
| Replay & Retry | — | — | Recover from FAILED / PARTIAL ingest jobs, replay after pipeline changes |
The one-command shortcut
Before wiring anything by hand, check whether a built-in recipe already covers it. dodil data recipe install provisions the pipeline and binds the ingest rule in a single command — the only path that does both:
| Recipe | Stands up | Hand-wired equivalent |
|---|---|---|
document-rag | text_embedding_index + doc-glob rule | PDF → Vector |
code-rag | code_embedding_index + source-extension rule | same shape |
image-rag | visual_embedding_index + image-glob rule | same shape |
invoice-intake | invoice_parsing → table + rule | Documents → Warehouse |
transcription | audio_transcription → table, then text_embedding_index → vector | — |
summarize-to-markdown | summarization → object destination summaries/ | — |
summarize-then-rag | the above, then indexes summaries/ for search | — |
generic | vector index + classification/summarization tables | — |
Those eight ids are the complete list. See dodil data recipe. The pages below build the same chains step by step, so you can see — and vary — each piece.
Other templates worth knowing
The recipes here pick one template each as the worked example. The same flow works with any of the 34 catalog templates — swap the --template argument and use the creator matching the destination facet:
| Goal | Template | Destination | Create with |
|---|---|---|---|
| Code search across a repo | code_embedding_index | Vector | vector collection add |
| Multimodal asset library | visual_embedding_index | Vector | vector collection add |
| Face recognition | face_embedding_index | Vector | vector collection add |
| Open-vocabulary object indexing | object_embedding_index | Vector | vector collection add |
| Object detection → rows | object_detection | Warehouse | table pipeline create |
| Audio transcription | audio_transcription | Warehouse | table pipeline create |
| Document classification | classification | Warehouse | table pipeline create |
| Document triage / routing | document_triage | Warehouse | table pipeline create |
| Sentiment dashboards | sentiment_intent_analysis or review_analysis | Warehouse | table pipeline create |
| Image understanding (OCR + detection + reasoning) | image_understanding | Warehouse | table pipeline create |
| Translation | translation | Warehouse | table pipeline create |
| Code intelligence (symbols, deps) | code_intelligence | Warehouse | table pipeline create |
| Product catalog enrichment | product_catalog_enrichment | Warehouse | table pipeline create |
| Financial KPI extraction | financial_statement_extraction | Warehouse | table pipeline create |
| Verbatim XLSX row ingest | tabular_data_ingest | Warehouse | table pipeline create |
| Video surveillance | video_surveillance | Warehouse | table pipeline create |
| Summaries written back as files | summarization | Object | object destination create |
dodil data table templates prints the authoritative warehouse-compatible subset (17 templates, filtered server-side by facet=TABLE). Full descriptions, modalities and labels: API Reference → Templates → The catalog.
Conventions used in these recipes
- CLI-first — every step has a
dodil data ...command. Anything not in the CLI today drops tocurlagainst the API; where a recipe ends by querying what it built, it shows the native client (SQL over the Postgres wire,dodil data search) rather than curl. - Endpoint examples use the production control plane
https://api.data.dodil.io. Table and vector data operations do not live there — they go to the tables-gateway. - Buckets use the placeholder
kb-prod; replace with your real bucket name. - Internal source is automatic — every bucket has one from
CreateBucket, namedinternalwith providerSOURCE_PROVIDER_INTERNAL_S3. You only look up itssource_idto wire rules. ItsstatusreadsSOURCE_STATUS_PENDINGuntil it first syncs; that’s fine. - A pipeline without a rule never runs. No facet creator derives an ingest rule for you — binding one is a separate, mandatory step in every recipe here. Only
recipe installdoes both. - Destinations are created by facet creators, not by
pipeline create.CreatePipelinebinds to a destination that already exists, or makes a free pipeline; it never creates one. - External sources are Preview — these recipes use the internal source exclusively; the same pipeline + rule + job model works once external sources go GA.
See also
- Pipelines — the pipelines engine overview
- Quickstart — first pipeline + first ingest in 5 minutes
- Core Concepts — Source, Credential, Pipeline, Template, Rule, IngestJob
- API Reference — Source / Pipeline / Ingest contracts
- CLI Guide — every
dodil datacommand in this domain