fabric

sqllocks_spindle.fabric

Spindle Fabric integration — OneLake, Eventstream, Lakehouse, SQL Database, Semantic Model.

Classes

OneLakePaths

Constructs proper OneLake paths for Fabric Lakehouse items.

Supports the standard landing zone layout::

Files/landing/<domain>/<entity>/dt=YYYY-MM-DD/hour=HH/
Files/landing/<domain>/<entity>/_control/
Files/quarantine/<domain>/<run_id>/

Works both inside a Fabric runtime (auto-detected via FABRIC_RUNTIME or TRIDENT_RUNTIME_VERSION environment variables) and locally with a configurable base path.

Parameters:

Name	Type	Description	Default
base_path	str | Path | None	Root path for the Lakehouse Files area. If None, auto-detects: inside Fabric uses /lakehouse/default/Files, otherwise defaults to ./lakehouse_files.	None

Attributes

base property

Return the resolved base path (Path for local, str URI for remote).

Methods:

landing_zone_path(domain, entity, dt, hour=None)

Build the landing zone partition path.

Parameters:

Name	Type	Description	Default
domain	str	Domain name (e.g. "retail").	required
entity	str	Entity / table name (e.g. "order").	required
dt	str	Date string in YYYY-MM-DD format.	required
hour	str | int | None	Optional hour partition ("00"–"23"). When provided the path includes an hour=HH segment.	None

Returns:

Type	Description
Path | str	Fully qualified landing zone directory path.

quarantine_path(domain, run_id)

Build the quarantine directory for failed / rejected rows.

Parameters:

Name	Type	Description	Default
domain	str	Domain name.	required
run_id	str	Unique identifier for the ingestion run.	required

Returns:

Type	Description
Path | str	Quarantine directory path.

tables_path(table_name)

Build the Tables path for a Delta table (sibling of Files).

This points to <lakehouse_root>/Tables/<table_name> — the standard location that DeltaWriter targets.

Parameters:

Name	Type	Description	Default
table_name	str	Name of the Delta table.	required

Returns:

Type	Description
Path	Path to the table directory under Tables.

control_path(domain, entity)

Build the control directory path for manifests and done flags.

Parameters:

Name	Type	Description	Default
domain	str	Domain name.	required
entity	str	Entity / table name.	required

Returns:

Type	Description
Path | str	Control directory path.

manifest_path(domain, entity, dt)

Build the manifest file path for a given date partition.

Parameters:

Name	Type	Description	Default
domain	str	Domain name.	required
entity	str	Entity / table name.	required
dt	str	Date string in YYYY-MM-DD format.	required

Returns:

Type	Description
Path | str	Path to the manifest JSON file.

done_flag_path(domain, entity, dt)

Build the done-flag (sentinel) file path for a given date partition.

Parameters:

Name	Type	Description	Default
domain	str	Domain name.	required
entity	str	Entity / table name.	required
dt	str	Date string in YYYY-MM-DD format.	required

Returns:

Type	Description
Path | str	Path to the _SUCCESS sentinel file.

LakehouseFilesWriter

Write data files into Lakehouse Files landing zones.

This writer targets the Files/ area of a Fabric Lakehouse (flat files in landing zones), as opposed to :class:~sqllocks_spindle.output.DeltaWriter which writes Delta tables into the Tables/ area.

Uses :class:OneLakePaths for consistent path construction and works both inside a Fabric runtime and locally with a configurable base path.

Parameters:

Name	Type	Description	Default
base_path	str | Path | None	Root path for the Lakehouse Files area. Passed through to :class:OneLakePaths. If None, auto-detects Fabric or falls back to ./lakehouse_files.	None
default_format	str	Default output format — "parquet", "csv", or "jsonl" (default "parquet").	'parquet'

Example::

from sqllocks_spindle.fabric import LakehouseFilesWriter, OneLakePaths

writer = LakehouseFilesWriter(base_path="./test_output")
paths = OneLakePaths(base_path="./test_output")
dest = paths.landing_zone_path("retail", "order", "2025-01-15", hour=10)

writer.write_partition(df, dest, format="parquet")
writer.write_done_flag(paths.done_flag_path("retail", "order", "2025-01-15"))

Attributes

paths property

Return the underlying :class:OneLakePaths instance.

Methods:

write_all(tables, format=None, **kwargs)

Write each table as a bulk file (Parquet/CSV/JSONL).

Conforms to the SpindleWriter protocol so LakehouseFilesWriter can be used with MultiWriter.

Parameters:

Name	Type	Description	Default
tables	dict[str, DataFrame]	Mapping of table_name -> DataFrame.	required
format	str | None	Output format (defaults to writer's default_format).	None
**kwargs	Any	Extra args passed to write_partition.	{}

Returns:

Type	Description
LakehouseWriteResult	LakehouseWriteResult with per-table row counts and any errors.

write_partition(df, path, format=None, file_naming_template=None)

Write a DataFrame as a data file into a landing-zone partition.

Parameters:

Name	Type	Description	Default
df	DataFrame	DataFrame to write.	required
path	str | Path	Target directory for the partition.	required
format	str | None	Output format — "csv", "parquet", or "jsonl". Falls back to default_format if None.	None
file_naming_template	str | None	Template for the output file name. Supports {format} placeholder. Defaults to "part-0001.{format}".	None

Returns:

Type	Description
Path	Path to the written file.

Raises:

Type	Description
ValueError	If format is unsupported.

write_manifest(manifest_dict, path)

Write a manifest JSON file to the control directory.

Parameters:

Name	Type	Description	Default
manifest_dict	dict[str, Any]	Manifest contents (serialised as JSON).	required
path	str | Path	Full path to the manifest file.	required

Returns:

Type	Description
Path	Path to the written manifest file.

write_done_flag(path)

Write an empty _SUCCESS sentinel file.

Parameters:

Name	Type	Description	Default
path	str | Path	Full path to the done-flag file.	required

Returns:

Type	Description
Path	Path to the written sentinel file.

LakehouseWriteResult dataclass

Result of a Lakehouse write_all() operation.

MultiWriter

Write generated data to multiple Fabric stores concurrently.

Fans out to all configured writers using ThreadPoolExecutor. Each writer runs concurrently. Partial failures are captured without aborting other stores.

Example::

from sqllocks_spindle.fabric import (
    EventhouseWriter, WarehouseBulkWriter,
    LakehouseFilesWriter, MultiWriter,
)

mw = MultiWriter(
    eventhouse=EventhouseWriter(...),
    lakehouse=LakehouseFilesWriter(...),
)
result = mw.write(tables)
print(result.summary())

Methods:

write(tables, **kwargs)

Write tables to all configured stores concurrently.

Parameters:

Name	Type	Description	Default
tables	dict[str, DataFrame]	Mapping of table_name -> DataFrame.	required
**kwargs	Any	Extra args forwarded to each writer's write_all().	{}

Returns:

Type	Description
MultiWriteResult	MultiWriteResult aggregating per-store results.

MultiWriteResult dataclass

Aggregated result from writing to multiple stores.