Dynamic Fields

Dynamic fields let you store extra attributes with each record without predefining them in your collection schema.

This is useful when:

Your metadata varies per item (different products have different attributes).
You iterate quickly and don’t want to redeploy / migrate schema for every new field.
You want to keep a stable core schema (ID + vector + a few required scalars) and attach optional metadata on demand.

How it works

When dynamic fields are enabled, the underlying vector store adds a hidden JSON field called $meta to each entity. Any key you insert that is not declared in the schema is automatically stored inside that hidden JSON field. You don’t need to manage \$meta yourself—this happens transparently. citeturn4view2

Enable dynamic fields in a collection

In Dodil, you enable dynamic fields when you build the collection schema.

Below is a minimal example that defines only:

a primary key
a dense vector field

…and allows everything else to be stored dynamically.


from dodil import Client
from dodil.vbase import VBaseConfig, DataType
 
# Authorize
c = Client(
    service_account_id="...",
    service_account_secret="...",
)
 
# Connect to VBase
vbase = c.vbase.connect(
    VBaseConfig(
        host="vbase-db-<id>.infra.dodil.cloud",
        port=443,
        scheme="https",
        db_name="db_<id>",
    )
)
 
# Build a schema with dynamic fields enabled
schema = vbase.raw.create_schema(
    auto_id=False,
    enable_dynamic_field=True,
)
 
schema.add_field(field_name="id", datatype=DataType.INT64, is_primary=True)
schema.add_field(field_name="vector", datatype=DataType.FLOAT_VECTOR, dim=768)
 
# Create the collection
vbase.create_collection(
    collection_name="products",
    schema=schema,
)

Insert records with extra dynamic keys

After enabling dynamic fields, you can insert records with keys that do not exist in the schema. Those keys will be stored automatically in the dynamic JSON field. citeturn4view2


vbase.insert(
    collection_name="products",
    data=[
        {
            "id": 1,
            "vector": [0.1] * 768,
 
            # These are NOT in the schema → stored dynamically
            "overview": "Great product",
            "words": 150,
            "dynamic_json": {
                "varchar": "some text",
                "nested": {"value": 42.5},
                "string_price": "99.99",
            },
        }
    ],
)

Filter using dynamic keys

Once inserted, dynamic keys can be used in filter expressions like normal scalar fields. For keys that are JSON objects, use JSON path syntax to access nested values. citeturn4view2


filter_1 = 'overview == "Great product"'              # simple dynamic key
filter_2 = 'words >= 100'                             # numeric dynamic key
filter_3 = 'dynamic_json["nested"]["value"] < 50'   # nested JSON key
 
# Example: filter + return dynamic keys in the result
results = vbase.search(
    collection_name="products",
    data=[[0.1] * 768],
    filter=filter_1,
    limit=10,
    output_fields=[
        "overview",
        'dynamic_json["varchar"]',
    ],
)
 
print(results)

Index keys inside the dynamic field

If you frequently filter on a dynamic key (especially at scale), you can create a JSON-path index for faster filtering.

Indexing dynamic keys is optional.
For JSON path indexing, the index type must be AUTOINDEX or INVERTED. citeturn4view2


index_params = vbase.raw.prepare_index_params()
 
# Index a simple string key
index_params.add_index(
    field_name="overview",
    index_type="AUTOINDEX",
    index_name="overview_index",
    params={
        "json_cast_type": "varchar",
        "json_path": "overview",
    },
)
 
# Index a numeric key
index_params.add_index(
    field_name="words",
    index_type="AUTOINDEX",
    index_name="words_index",
    params={
        "json_cast_type": "double",
        "json_path": "words",
    },
)
 
# Index a nested JSON key
index_params.add_index(
    field_name="dynamic_json",
    index_type="AUTOINDEX",
    index_name="json_nested_index",
    params={
        "json_cast_type": "double",
        "json_path": "dynamic_json['nested']['value']",
    },
)
 
vbase.create_index(
    collection_name="products",
    index_params=index_params,
)

When to promote a dynamic key into the schema

Dynamic fields are great for flexibility, but for core fields you use all the time, consider adding them explicitly to your schema:

You frequently return the field in output_fields.
You filter on it constantly and want predictable performance.
You want stronger typing and more maintainable schema design.

That way you get both: a stable, explicit schema for the essentials + dynamic fields for everything else.