Skip to Content
We are live but in Staging 🎉
Data EnginesOverview

Data Engines

Every K3 bucket is backed by four engines — one namespace, four ways to hold and query the same data. You don’t provision them individually: Objects and SQL are live at bucket creation, Vector and Graph materialize on demand the first time you create a collection or a graph.

The four engines

EngineWhat it holdsBackendWire protocols
ObjectsRaw bytes — files, media, uploadsCephS3 byte planeS3 (SigV4) at object.uk-lon-1.dodil.io — aws-cli, boto3, rclone, any S3 SDK
SQLStructured rows — two lanes: DB (OLTP point reads/writes) and Warehouse (HTAP analytics) on the tabled data planetabledPostgres at pg.uk-lon-1.dodil.io:5432, gRPC at table-rpc.uk-lon-1.dodil.io:443, GraphQL at gql.uk-lon-1.dodil.io/graphql
VectorEmbeddings — semantic / similarity search collectionstabled data planeQdrant at qdrant.uk-lon-1.dodil.io, Pinecone at pinecone.uk-lon-1.dodil.io — stock clients
GraphNodes + edges projected over SQL tables — defined via CREATE GRAPH in SQLtabled data planeBolt (Cypher) at bolt+s://bolt.uk-lon-1.dodil.io:7687 — neo4j drivers, cypher-shell

Three services behind the four engines

The four engines are the product view. They are served by three services, and knowing which one owns an operation tells you which host and which protocol to reach for:

  • k3-api — the control plane. HTTP/JSON on api.data.dodil.io, gRPC on rpc.data.dodil.io:443 (dodil.data.*.v1). Buckets & provisioning, quotas & billing, the connect flow, sources, ingest rules, pipelines (including the vector and object pipeline facets), ingest jobs, templates, actions, reservation intent, and the search dispatcher. This is what the dodil data CLI and the docs’ control-plane curl examples talk to. It does not serve table/vector/graph data or S3 bytes.
  • tables-gateway — the SQL / Vector / Graph data plane. Table DDL + DML, ad-hoc SQL, maintenance, and the vector + graph facets (dodil.tables.v1), exposed over the Postgres, Bolt, Qdrant, Pinecone, GraphQL, and Tables REST/gRPC wire adapters.
  • object-gateway — the Objects data plane. S3 byte operations over the S3 protocol.

The data planes are per-protocol wire adapters: the same bucket speaks the wire protocol of the tool you already use, and the db id is the bucket name on every wire (Postgres dbname, S3 bucket, Bolt database, the Qdrant/Pinecone api-key).

control plane (global) api.data.dodil.io / rpc.data.dodil.io:443 (k3-api) ───────────────────────────────────────────────────────────────── Objects data plane object.uk-lon-1.dodil.io (S3) object-gateway SQL data plane pg.uk-lon-1.dodil.io:5432 (Postgres) tables-gateway table-rpc.uk-lon-1.dodil.io:443 (gRPC) table.uk-lon-1.dodil.io (HTTP) gql.uk-lon-1.dodil.io/graphql (GraphQL) Vector data plane qdrant.uk-lon-1.dodil.io (Qdrant) tables-gateway pinecone.uk-lon-1.dodil.io (Pinecone) Graph data plane bolt+s://bolt.uk-lon-1.dodil.io (Bolt/Cypher) tables-gateway

Data-plane doors are region-scoped (<service>.<region>.dodil.io); uk-lon-1 is the current production region. Endpoints, credentials (API keys / service accounts), and copy-paste client snippets live in Connect & Adapters.

How the engines fit together

  • Uploads land in Objects; Pipelines fan each object out into SQL rows and Vector embeddings automatically.
  • SQL’s two lanes share one table surface — transactional writes coalesce into analytical reads (HTAP); see SQL → Concepts.
  • Graph is a projection: CREATE GRAPH g NODES (people KEY id) EDGES (follows SRC src DST dst) over existing SQL tables, then traverse with graph_* SQL functions or Cypher over Bolt — and converge them: KNN candidates in SQL, graph walks over the hits, or all three pillars in one GraphQL query.
  • Hot capacity for the SQL / Vector / Graph planes is governed per bucket — free shared tier by default, dedicated reservations when you need guarantees. See Reservation & Hot Cache.

Where to next