dodil k3 table — maintenance

Operational table maintenance. Three commands — optimize packs small Delta files, vacuum reclaims space past retention, compact drains the write log into Delta on demand. K3 runs the log → Delta drain automatically in the background; you’ll reach for these for explicit packing, space reclamation, and post-batch sequencing.

Persistent flag on the group: --bucket / -b (required).

CLI doesn’t yet cover Restore (time travel) or History (commit log) — both live on the Maintenance API. See Recipes → Time travel + Restore for the workflow.

dodil k3 table optimize

dodil k3 table optimize [name] -b BUCKET [--target-file-size-mb N] [--z-order-column COL ...]

Two modes:

• Bin-pack (default) — coalesce small Delta files up to --target-file-size-mb per partition.
• Z-order — set --z-order-column to rewrite all files clustered by those columns. Improves read locality for queries that filter on them.

# Bin-pack with default 128 MB target
dodil k3 table optimize events --bucket kb-prod
 
# Larger target (256 MB) for analytical workloads
dodil k3 table optimize events --bucket kb-prod --target-file-size-mb 256
 
# Z-order for queries that filter on (event_type, user_id)
dodil k3 table optimize events --bucket kb-prod \
  --z-order-column event_type --z-order-column user_id

Reading the response:

When to call: after large bulk writes / nightly batches / migrations. Routinely-compacted HTAP tables stay well-packed automatically.

dodil k3 table vacuum

dodil k3 table vacuum [name] -b BUCKET [--retention-hours N] [--dry-run] [--disable-retention-check]

Permanently delete old Delta file versions past retention. Destructive — vacuumed files are gone, and the corresponding Delta versions can no longer be restored.

# ALWAYS dry-run first
dodil k3 table vacuum events --bucket kb-prod --dry-run -o json \
  | jq '{filesDeleted, filesDeletedPaths: .filesDeletedPaths[0:5]}'
 
# Real run after reviewing
dodil k3 table vacuum events --bucket kb-prod
 
# Custom retention
dodil k3 table vacuum events --bucket kb-prod --retention-hours 720   # 30 days

--disable-retention-check bypasses Delta’s safety net. Delta refuses to vacuum files younger than 168 h because in-flight readers and time-travel queries break once the files are gone. Set only for forced cleanups on quiesced tables where you accept that risk.

Vacuum interacts with Restore — once you vacuum past version N, restore to version < N is no longer possible. The safe pattern is documented at Recipes → Time travel + Restore.

dodil k3 table compact

dodil k3 table compact [name] -b BUCKET [--batch-size N]

Force the write log to drain into Delta now. K3 runs this in the background automatically; call it manually for:

• After bulk writes — materialize rows in Delta immediately for analytical reads
• Before maintenance — compact → optimize is the canonical sequence post-batch
• Tests / e2e — drain the log to assert against describe’s lastDrain* counters

# Single drain tick
dodil k3 table compact events --bucket kb-prod
 
# Loop until fully drained — `truncated: false` means done
while dodil k3 table compact events --bucket kb-prod -o json \
        | jq -e '.truncated == true' > /dev/null; do
  echo "  more entries to drain..."
done

Response fields:

walUniqueKeys < walEntriesProcessed is normal — the compactor dedupes multiple writes to the same PK (newest-wins) before MERGE.

Canonical post-batch sequence

After a big bulk write:

# 1. Drain the write log into Delta
while dodil k3 table compact events --bucket kb-prod -o json \
        | jq -e '.truncated == true' > /dev/null; do :; done
 
# 2. Bin-pack the resulting small files
dodil k3 table optimize events --bucket kb-prod
 
# 3. (Optional) Periodically — vacuum old versions past retention
dodil k3 table vacuum events --bucket kb-prod --dry-run     # always preview first
dodil k3 table vacuum events --bucket kb-prod

See also

• Maintenance — API Reference — full surface incl. Restore and History
• dodil k3 table — lifecycle — describe to see drain-lag + last-drain stats
• dodil k3 table — data — what produces write-log entries in the first place
• Recipes → Time travel + Restore — vacuum / restore interaction