Create

Two creation paths, on two different planes:

What	Plane	Surface
`CreateTable` — you define the schema	data plane	`dodil.tables.v1.Tables/CreateTable`, or `CREATE TABLE` SQL via `Execute`
`CreateTablePipeline` — a Scriptum template owns the schema	control plane	`dodil.data.pipeline.v1.PipelineService/CreateTablePipeline` · `POST /:bucket/pipelines/table`

See the Tables hub for the full RPC list.

`CreateTable`

Caller defines columns / partition columns / primary key; the plane creates the Delta table eagerly. CreateTableRequest carries a oneof ddl — a typed TableSpec or a raw sql string. Both normalize to the same executor, so CREATE TABLE SQL through Execute is equivalent.

Request

SQL


CREATE TABLE events (
  id           BIGINT      NOT NULL,
  user_id      VARCHAR     NOT NULL,
  occurred_at  TIMESTAMP   NOT NULL,
  event_type   VARCHAR     NOT NULL,
  payload      JSON,
  PRIMARY KEY (id, user_id)
)
PARTITIONED BY (event_type);

CREATE TABLE IF NOT EXISTS, CREATE OR REPLACE TABLE and CREATE TABLE … AS SELECT are all accepted — they set if_not_exists / or_replace / the CTAS source on the same plan.

dodil data


dodil data table create events -b kb-prod \
  --description "Click + purchase events" \
  --columns-json '[
    {"name":"id",          "type":"bigint",    "nullable":false},
    {"name":"user_id",     "type":"varchar",   "nullable":false},
    {"name":"occurred_at", "type":"timestamp", "nullable":false},
    {"name":"event_type",  "type":"varchar",   "nullable":false},
    {"name":"payload",     "type":"json",      "nullable":true}
  ]' \
  --partition-column event_type \
  --merge-key id --merge-key user_id

--merge-key populates TableSpec.pk_columns — a composite primary key. Column type is a SQL type string; the CLI normalizes a few aliases (int/integer/long/int64 → BIGINT, string/text → VARCHAR, float/real → DOUBLE, bool → BOOLEAN, bytes → BLOB) and passes anything else through uppercased, so VECTOR(768) and DECIMAL(18,2) work.

gRPC (typed spec)


rpc CreateTable(CreateTableRequest) returns (CreateTableResponse);
 
message CreateTableRequest {
  string db_id = 1;
  oneof ddl {
    TableSpec spec = 2;
    string sql = 3;                    // "CREATE TABLE …" — same executor
  }
}
 
message TableSpec {
  string table_name = 1;
  repeated ColumnDef columns = 2;
  repeated string partition_columns = 3;
  repeated string pk_columns = 4;      // composite PK supported
  optional string description = 5;
  bool if_not_exists = 6;
  bool or_replace = 7;
}
 
message ColumnDef {
  string name = 1;
  string type = 2;                     // BIGINT, VARCHAR, DOUBLE, TIMESTAMP…
  bool nullable = 3;
  optional string default_expr = 4;    // GENERATE_UUID(), CURRENT_TIMESTAMP(), literal…
}


{
  "db_id": "acme--kb-prod",
  "spec": {
    "table_name": "events",
    "columns": [
      { "name": "id",          "type": "BIGINT",    "nullable": false },
      { "name": "user_id",     "type": "VARCHAR",   "nullable": false },
      { "name": "occurred_at", "type": "TIMESTAMP", "nullable": false },
      { "name": "event_type",  "type": "VARCHAR",   "nullable": false },
      { "name": "payload",     "type": "JSON",      "nullable": true }
    ],
    "partition_columns": ["event_type"],
    "pk_columns": ["id", "user_id"],
    "description": "Click + purchase events"
  }
}

HTTP

There is no dedicated create-table HTTP route. Send the DDL through the SQL door:


curl -sS -X POST "https://table.uk-lon-1.dodil.io/v1/databases/acme--kb-prod/sql/execute" \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sql":"CREATE TABLE events (id BIGINT NOT NULL, user_id VARCHAR NOT NULL, occurred_at TIMESTAMP NOT NULL, event_type VARCHAR NOT NULL, payload JSON, PRIMARY KEY (id, user_id)) PARTITIONED BY (event_type)"}'

Response: { "kind": "ddl" }.

Response


message CreateTableResponse {
  string table_name = 1;
  string location = 2;                 // s3://… path
  int64 version = 3;
  repeated string partition_columns = 4;
  repeated string pk_columns = 5;
  uint32 column_count = 6;             // TOTAL columns
  bool already_existed = 7;            // IF NOT EXISTS hit an existing table
  bool replaced = 8;                   // OR REPLACE dropped and rebuilt
}


{
  "table_name": "events",
  "location": "s3://…/events",
  "version": 0,
  "partition_columns": ["event_type"],
  "pk_columns": ["id", "user_id"],
  "column_count": 5,
  "already_existed": false,
  "replaced": false
}

Through Execute, the same information arrives as ExecuteResponse.ddl (a DdlResult, which adds columns_added, objects_deleted and rows_backfilled).

Defaults via ColumnDef.default_expr — BigQuery-aligned SQL applied at row commit when the column is absent: GENERATE_UUID(), ULID(), CURRENT_TIMESTAMP(), 42, 'pending'. The expression is validated against the column type at DDL time, not on first insert. Note GENERATE_UUID() and ULID() return strings — declare those columns VARCHAR. See Core Concepts → Column.

Keyless tables: a table created without a primary key gets a hidden _rowid VARCHAR DEFAULT generate_ulid() PK so bag semantics and Postgres parity hold. It is hidden from SELECT * and from the pg wire, but it means keyed DML addresses _rowid, not your columns.

`CreateTablePipeline`

Pipeline-generated mode — the schema is owned by a Scriptum analytics template and materializes lazily on first ingest. This is a control-plane RPC, on dodil.data.pipeline.v1.PipelineService, not on Tables. It returns a Pipeline, not a table.

RPC	HTTP
`dodil.data.pipeline.v1.PipelineService/CreateTablePipeline`	`POST https://api.data.dodil.io/:bucket/pipelines/table`

It does not create an ingest rule. CreateTablePipeline writes two rows — the warehouse destination and the pipeline binding — and stops. Nothing will ever fire until you add a rule yourself (dodil-k3/bin/api/src/services/pipeline/create_table.rs:12-15, whose comment says so verbatim). This is the single most common reason a pipeline table stays empty.

Request

dodil data


# 1. create the pipeline + its warehouse destination
dodil data table pipeline create entities -b kb-prod \
  --template entity_pii_extraction \
  --description "Auto-extracted entities + PII"
 
# 2. MANDATORY — scope an ingest rule at it, or nothing ever runs
dodil data ingest add entities-intake -b kb-prod \
  -c "$PIPELINE_ID" -i 'intake/**/*.pdf'

Browse templates with dodil data table templates (flags: --search, --label).

gRPC


rpc CreateTablePipeline(CreateTablePipelineRequest) returns (Pipeline);
 
message CreateTablePipelineRequest {
  string bucket = 1;
  string name = 2;                     // the table name
  string description = 3;
  // REQUIRED. The template MUST declare @accepts_extension — a table
  // pipeline whose template declares none can never fire.
  string template_id = 4;
  // Optional. Writes a 0-byte S3 folder marker so the prefix renders in
  // the object explorer before the first upload. Must not contain `..`
  // or `**`. It is NOT rule scoping.
  optional string folder_prefix = 5;
  optional string recipe_name = 6;
}

HTTP


curl -sS -X POST "https://api.data.dodil.io/kb-prod/pipelines/table" \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "name": "entities",
    "description": "Auto-extracted entities + PII",
    "template_id": "entity_pii_extraction"
  }'

Response

A Pipeline — see Pipelines. Its id is what you pass to dodil data ingest add -c.

folder_prefix is not a glob. It writes a 0-byte S3 folder marker so the prefix shows up in the object explorer before the first upload. It never scopes a rule and can never match a glob — rule scope is the -i pattern you pass to ingest add.

Precondition: the template must declare @accepts_extension. Without it the create is refused (dodil-k3/bin/api/src/services/pipeline/facets.rs:112-124) — the derived globs have nothing to derive from.

The schema is template-defined and learned on first ingest. K3 deliberately does not derive columns up front (create_table.rs:74-79), so DESCRIBE returns nothing useful until the first document lands.

Create

CreateTable

Request

SQL

dodil data

gRPC (typed spec)

HTTP

Response

CreateTablePipeline

Request

dodil data

gRPC

HTTP

Response

See also

`CreateTable`

`CreateTablePipeline`