Core concepts
Jobs
A Job represents a single ingestion run (or batch). You create a job, stream inputs into it, then finalize it.
TaskType
A TaskType represents the function as Embedding can be used for multiple purposes.
Inputs
- text
- URLs
- s3
- auto
- datasets
Outputs
Outputs are the produced artifacts:
- chunk index
- chunk text
- embeddings
- metadata + spans
When to use Vector Ingestion
Use VNG when you want:
- a reliable ingestion pipeline
- ingestion for multiple file types
- reprocessing when content changes
- distributed workers for throughput
- repeatable ingestion that’s ready for production
If you only need to embed a single short string once, you may not need VNG. But if you’re building a system where data constantly changes and needs to stay searchable, VNG is essential.
Last updated on