Data Engines
Every K3 bucket is backed by four engines — one namespace, four ways to hold and query the same data. You don’t provision them individually: Objects and SQL are live at bucket creation, Vector and Graph materialize on demand the first time you create a collection or a graph.
The four engines
| Engine | What it holds | Backend | Wire protocols |
|---|---|---|---|
| Objects | Raw bytes — files, media, uploads | CephS3 byte plane | S3 (SigV4) at object.uk-lon-1.dodil.io — aws-cli, boto3, rclone, any S3 SDK |
| SQL | Structured rows — two lanes: DB (OLTP point reads/writes) and Warehouse (HTAP analytics) on the tabled data plane | tabled | Postgres at pg.uk-lon-1.dodil.io:5432, gRPC at table-rpc.uk-lon-1.dodil.io:443, GraphQL at gql.uk-lon-1.dodil.io/graphql |
| Vector | Embeddings — semantic / similarity search collections | tabled data plane | Qdrant at qdrant.uk-lon-1.dodil.io, Pinecone at pinecone.uk-lon-1.dodil.io — stock clients |
| Graph | Nodes + edges projected over SQL tables — defined via CREATE GRAPH in SQL | tabled data plane | Bolt (Cypher) at bolt+s://bolt.uk-lon-1.dodil.io:7687 — neo4j drivers, cypher-shell |
Three services behind the four engines
The four engines are the product view. They are served by three services, and knowing which one owns an operation tells you which host and which protocol to reach for:
- k3-api — the control plane. HTTP/JSON on
api.data.dodil.io, gRPC onrpc.data.dodil.io:443(dodil.data.*.v1). Buckets & provisioning, quotas & billing, the connect flow, sources, ingest rules, pipelines (including the vector and object pipeline facets), ingest jobs, templates, actions, reservation intent, and the search dispatcher. This is what thedodil dataCLI and the docs’ control-planecurlexamples talk to. It does not serve table/vector/graph data or S3 bytes. - tables-gateway — the SQL / Vector / Graph data plane. Table DDL + DML, ad-hoc SQL, maintenance, and the vector + graph facets (
dodil.tables.v1), exposed over the Postgres, Bolt, Qdrant, Pinecone, GraphQL, and Tables REST/gRPC wire adapters. - object-gateway — the Objects data plane. S3 byte operations over the S3 protocol.
The data planes are per-protocol wire adapters: the same bucket speaks the wire protocol of the tool you already use, and the db id is the bucket name on every wire (Postgres dbname, S3 bucket, Bolt database, the Qdrant/Pinecone api-key).
control plane (global) api.data.dodil.io / rpc.data.dodil.io:443 (k3-api)
─────────────────────────────────────────────────────────────────
Objects data plane object.uk-lon-1.dodil.io (S3) object-gateway
SQL data plane pg.uk-lon-1.dodil.io:5432 (Postgres) tables-gateway
table-rpc.uk-lon-1.dodil.io:443 (gRPC)
table.uk-lon-1.dodil.io (HTTP)
gql.uk-lon-1.dodil.io/graphql (GraphQL)
Vector data plane qdrant.uk-lon-1.dodil.io (Qdrant) tables-gateway
pinecone.uk-lon-1.dodil.io (Pinecone)
Graph data plane bolt+s://bolt.uk-lon-1.dodil.io (Bolt/Cypher) tables-gatewayData-plane doors are region-scoped (<service>.<region>.dodil.io); uk-lon-1 is the current production region. Endpoints, credentials (API keys / service accounts), and copy-paste client snippets live in Connect & Adapters.
How the engines fit together
- Uploads land in Objects; Pipelines fan each object out into SQL rows and Vector embeddings automatically.
- SQL’s two lanes share one table surface — transactional writes coalesce into analytical reads (HTAP); see SQL → Concepts.
- Graph is a projection:
CREATE GRAPH g NODES (people KEY id) EDGES (follows SRC src DST dst)over existing SQL tables, then traverse withgraph_*SQL functions or Cypher over Bolt — and converge them: KNN candidates in SQL, graph walks over the hits, or all three pillars in one GraphQL query. - Hot capacity for the SQL / Vector / Graph planes is governed per bucket — free shared tier by default, dedicated reservations when you need guarantees. See Reservation & Hot Cache.
Where to next
- Objects — buckets, S3 compatibility, presigned URLs
- SQL — tables, the Execute/Data APIs, DuckDB-dialect SQL
- Vector — collections and KNN search (no dense+BM25 hybrid — see Feature Status)
- Graph — CREATE GRAPH, the Cypher subset & traversals, analytics
- Converging engines — KNN + joins + graph traversal + GraphQL over one bucket
- Connect & Adapters — every endpoint, every credential mode
- Reservation & Hot Cache — resident-capacity guarantees per bucket