Skip to content

id_manager

sqllocks_spindle.engine.id_manager

ID Manager — tracks generated PKs and provides FK resolution.

Classes

PKPool

Bases: Protocol

Protocol for PK pool implementations.

RangePKPool

Memory-efficient PK pool for contiguous integer ranges.

Instead of storing all PK values (16 GB for 2B int64s), stores only (start, count) — 16 bytes total. FK sampling uses rng.integers() for O(1) memory.

Attributes
end property

Exclusive end of the range.

Methods:
extend(additional_count)

Grow the range by additional_count contiguous values.

IDManager

Track all generated primary keys and provide foreign key resolution.

The ID Manager is the backbone of relational integrity in Spindle. It maintains pools of generated PK values per table and provides various FK lookup strategies (random, weighted, constrained).

Methods:
register_table(table_name, df, pk_columns)

Register a generated table's PKs for FK resolution (thread-safe).

Tables with no primary_key (pk_columns=[]) are registered as data-only so lookups and constrained FKs still work, but get_random_fks() will raise KeyError since they have no PK pool.

register_range(table_name, start, count)

Register a contiguous integer PK range without storing all values.

Uses RangePKPool for O(1) memory regardless of count.

append_pks(table_name, new_pks)

Grow a table's PK pool incrementally with new values (thread-safe).

For RangePKPool, extends the range. For ndarray pools, concatenates.

get_random_fks(table_name, count, distribution='uniform', params=None)

Get random FK values from a parent table's PK pool.

Supports uniform, zipf, and pareto-weighted selection.

get_constrained_fks(table_name, constraint_column, constraint_values, nullable=False)

Get FK values constrained by a parent column value.

Example: get address_id values that belong to specific customer_ids.

When nullable=True, rows where no matching parent exists get None instead of a random fallback. Set nullable=True for nullable FK columns.

get_filtered_fks(table_name, filter_column, filter_value, count)

Get FK values filtered by a column condition.

Example: get order_ids where status='completed'.

get_sampled_fks(table_name, sample_rate, filter_column=None, filter_value=None)

Get a sample of PKs (e.g., 8% of orders for returns).

Returns the actual FK values, not indices.

lookup_values(table_name, lookup_column, fk_values, pk_column)

Look up column values from a parent table via FK.

Example: get product.unit_price for each order_line.product_id.