Migration Toolkits

The repeatable parts of a warehouse migration, done: inventory the source and tag what’s dead, translate the DDL, copy the data from a manifest, and prove parity before you cut over. Covers SQL Server / SSIS, Snowflake, and Databricks. Pairs with the migration playbooks.

What’s included

• inventory-sqlserver.sql / inventory-snowflake.sql — one row per table: size, rows, last write, dependencies, likely-dead flag
• translate_ddl.py — SQL Server / Snowflake DDL → Fabric DDL (sqlglot when available, regex fallback), with warnings for what it drops
• metadata_driven_copy.py — copy every source table into bronze from one manifest; full snapshot or watermark-incremental
• validate_parity.py — row counts + aggregate checksums, source vs. migrated, per table; fails outside tolerance
• migration-manifest.example.json — the table list with per-table decision, watermark, and checksum columns
• function-cheatsheet.md — Snowflake/T-SQL → Fabric translation reference for the logic the translator can’t do
• cutover-runbook.md — the parallel-run, freeze, cut-over, and decommission checklist

File tree

README.md
config/migration-manifest.example.json
docs/cutover-runbook.md
docs/function-cheatsheet.md
notebooks/metadata_driven_copy.py
notebooks/validate_parity.py
scripts/translate_ddl.py
sql/inventory-snowflake.sql
sql/inventory-sqlserver.sql

The arc

1. Inventory the source; tag every table keep / retire / rebuild.
2. Move one thin vertical slice end to end first.
3. Translate DDL, review every file.
4. Fill the manifest; run the copy (history once, then incremental).
5. Run parity nightly — green for a full business cycle.
6. Cut over per the runbook. Done = the old pipeline is deleted.