# Docs

- [Start here](/docs): What fabdocs.dev is, who it is for, and how to navigate the manual.

- **Core Documentation**
- OneLake & Delta Lake: Storage architecture, Delta tuning, and shortcuts.
  - [OneLake & Delta Lake](/docs/onelake): The one copy of data for the whole tenant — and how to keep its Delta tables fast and cheap.
  - [Delta table optimization](/docs/onelake/delta-optimization): Compaction, target file size, partitioning vs. Z-order, and a maintenance schedule that fits inside your capacity.
  - [V-Order tuning](/docs/onelake/v-order): V-Order trades write cost for read speed. Here is how to decide per table.
  - [VACUUM & retention](/docs/onelake/vacuum-and-retention): Reclaim storage without breaking time travel, Direct Lake, or in-flight readers.
  - [Multi-cloud shortcuts](/docs/onelake/shortcuts): S3, ADLS, GCS, and Dataverse shortcuts — caching behavior, egress cost, and consistency gotchas.
  - [Database mirroring](/docs/onelake/mirroring): Near-real-time replicas of Snowflake, Azure SQL, Cosmos DB, and Postgres landing as Delta tables in OneLake — how it works, what it costs, and when to use it over a shortcut or a pipeline.
- Data Engineering: Notebooks, PySpark, and the execution engine.
  - [Data Engineering](/docs/data-engineering): Production PySpark on Fabric — patterns that survive Runtime upgrades and keep CU consumption predictable.
  - [Production PySpark patterns](/docs/data-engineering/pyspark-patterns): Idempotent upserts, safe schema evolution, parameterized notebooks, and structured logging for Fabric.
  - [notebookutils reference](/docs/data-engineering/notebookutils-reference): The practical reference for notebookutils (formerly mssparkutils) — filesystem, notebook orchestration, secrets, lakehouse, and runtime helpers in Fabric notebooks.
  - [Native Execution Engine](/docs/data-engineering/native-execution-engine): NEE runs Spark operators as native (Velox/Gluten) code. Here is where it helps, where it falls back, and how to verify.
  - [Resource profiles](/docs/data-engineering/resource-profiles): Spark resource profiles change dozens of configs at once to match the workload shape. Pick the right one instead of hand-tuning.
  - [Runtime upgrades](/docs/data-engineering/runtime-upgrades): Fabric Runtime bumps Spark, Delta, Python, and Java together. Treat it like a dependency upgrade, not a surprise.
- CI/CD & Git: Git integration, pipelines, and infrastructure as code.
  - [CI/CD & Git](/docs/cicd): Treat Fabric workspaces as deployable artifacts — Git-backed, promoted through environments, parameterized per stage.
  - [Git integration](/docs/cicd/git-integration): How workspace items serialize to a repo, what round-trips cleanly, and the branching model that avoids merge pain.
  - [Deployment pipelines](/docs/cicd/deployment-pipelines): Stages, item bindings, deployment rules, and how to gate promotion so prod never gets a surprise.
  - [Variable libraries](/docs/cicd/variable-libraries): One item, per-stage value sets. Notebooks and pipelines read a variable; the active value set switches on deployment.
  - [GitHub Actions for Fabric](/docs/cicd/github-actions): Validate on PR, deploy on merge. A service principal, fabric-cli, and two workflows.
- Governance & Security: OneLake security, identity, and audit.
  - [Governance & Security](/docs/governance): Patterns for OneLake security, workspace identity, and audit logging that pass an enterprise compliance review.
  - [OneLake security](/docs/governance/onelake-security): Define row-, column-, and table-level access once; every engine enforces it — Spark, SQL endpoint, and Direct Lake.
  - [Workspace identity](/docs/governance/workspace-identity): A managed Entra identity per workspace — trusted access to firewalled storage with no secrets to rotate.
  - [Audit logs](/docs/governance/audit-logs): Where Fabric activity is recorded, how long it lives, and how to get it into a SIEM for compliance.

- **Applied**
- Tools & Calculators: Interactive estimators and local workflow guides.
  - [Tools & Calculators](/docs/tools): Interactive estimators for capacity planning, plus practical local-development guides.
  - [CU cost & capacity calculator](/docs/tools/cu-cost-calculator): Estimate and compare Capacity Unit burn for Spark jobs versus SQL warehouse workloads by compute size and concurrency.
  - [Spark pool sizing estimator](/docs/tools/spark-pool-sizer): Estimate how many nodes and what node size a Fabric Spark job needs to finish in a target window — plus a memory sanity check and CU cost per run.
  - [Fabric capacity SKU reference](/docs/tools/capacity-sku-reference): F2 to F2048 — Capacity Units, the Power BI SKU equivalence, autoscale behavior, and which workloads each tier realistically supports.
  - [VS Code sync & extension guide](/docs/tools/vscode-sync): Edit Fabric notebooks, environments, and pipelines from VS Code — with Git, real linting, and no browser tab.
- Migration Playbooks: Step-by-step conversions into Fabric.
  - [Migration Playbooks](/docs/playbooks): Opinionated, step-by-step conversions from Databricks, Snowflake, and SSIS into Microsoft Fabric.
  - [From Databricks](/docs/playbooks/from-databricks): Move notebooks, jobs, Unity Catalog tables, and DLT pipelines to Fabric with the fewest rewrites.
  - [From Snowflake](/docs/playbooks/from-snowflake): Move warehouses, roles, tasks, streams, and SnowSQL to Fabric Warehouse and OneLake.
  - [From SSIS](/docs/playbooks/from-ssis): Convert control flow, data flow, and SSISDB packages to Fabric Data Pipelines and Dataflows Gen2.