Skip to Content
We are live but in Staging 🎉
PipelinesCLI Guidedodil data source

dodil data source

You usually don’t need create. Every bucket gets an auto-created internal S3 source on CreateBucket — direct uploads trigger pipelines without you registering anything. You do routinely need list — it’s how you resolve the internal source’s source_id when wiring a rule or triggering discovery.

Preview — external sources (Google Drive, SharePoint, Confluence, GitHub) are still solidifying for production. Direct S3 upload through the auto-created internal source is the production ingest channel today.

All source subcommands take --bucket / -b (required).

dodil data source list

dodil data source list -b BUCKET [-o json]

Lists every source on the bucket. The auto-created source appears with name: "internal", provider: "SOURCE_PROVIDER_INTERNAL_S3", description: "Auto-created internal S3 source", enabled: true and syncIntervalSeconds: "300".

Its status on a fresh bucket is SOURCE_STATUS_PENDING, not ACTIVE — the source hasn’t run a sync yet. That’s expected, and direct uploads still fire your rules. Don’t gate a setup script on seeing ACTIVE.

The human (non-JSON) output is tab-separated source_id / name / provider / enabled=….

The most common use — capture the internal source’s ID:

SOURCE_ID=$(dodil data source list -b kb-prod -o json \
  | jq -r '.sources[] | select(.name == "internal") | .sourceId')

dodil data source get

dodil data source get <source-id> -b BUCKET [-o json]

Fetches one source by ID — provider, root path, sync interval, enabled state, and status.

dodil data source get "$SOURCE_ID" -b kb-prod -o json \
  | jq '{sourceId, name, provider, status, enabled}'

dodil data source create

dodil data source create [name] -b BUCKET

Creates a new source row on the bucket. The CLI takes only name and --bucket — there are no flags for provider, root path, sync interval, or provider-specific account info.

FlagShortTypeDescription
--bucket-bstringRequired. Bucket to attach the source to.

Example:

dodil data source create contracts-source -b kb-prod

The result is barely usable on its own. Because the CLI sends no provider, the row is created as SOURCE_PROVIDER_UNSPECIFIED (stored as the provider string "unknown"), with root_path empty and sync_interval_seconds = 0. It only sets enabled: true. Nothing syncs from a source with no provider. Since the CLI also has no source update, you cannot fix it afterwards from the CLI — you’d need PATCH /:bucket/sources/{source_id}. Create external sources through the API in the first place, as shown below.

dodil data source has no update and no delete — the group is create, list, get only. Both operations exist on the API: PATCH /:bucket/sources/{source_id} and DELETE /:bucket/sources/{source_id}.

Adding a real external source (via API)

The CLI doesn’t yet expose provider/root_path/sync_interval flags. For external sources, hit the API directly:

# Google Drive source
curl -sS -X POST "https://api.data.dodil.io/kb-prod/sources" \
  -H "Authorization: Bearer $DODIL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket": "kb-prod",
    "provider": "SOURCE_PROVIDER_GOOGLE_DRIVE",
    "name": "finance-drive",
    "rootPath": "/Q2",
    "providerAccount": "drive-id-123",
    "syncIntervalSeconds": "3600",
    "enabled": true
  }'

You’ll typically also need to store a credential for the source.

See also

Overviewdodil data credential