Unity Catalog Migration Kit

$69

Step-by-step migration toolkit from Hive Metastore to Unity Catalog. Assessment scripts, automated migration, validation tests, and rollback procedures.

📁 14 files🏷 v1.0.0

PythonTerraformJSONMarkdownAWSAzureGCPDatabricksRedis

📁 File Structure 14 files

unity-catalog-migration-kit/ ├── README.md ├── guides/ │ ├── day2_operations.md │ ├── migration_guide.md │ └── rollback_procedures.md ├── notebooks/ │ ├── automated_table_migration.py │ ├── permission_mapper.py │ ├── post_migration_validation.py │ └── pre_migration_assessment.py ├── templates/ │ └── migration_planning_spreadsheet.csv ├── terraform/ │ └── unity-catalog-setup/ │ ├── main.tf │ ├── outputs.tf │ └── variables.tf └── tools/ ├── migration_planner.py └── timeline_estimator.py

📖 Documentation Preview README excerpt

Unity Catalog Migration Kit

By [Datanest Digital](https://datanest.dev) | Version 1.0.0 | $69

A comprehensive, production-ready toolkit for migrating from Databricks Hive Metastore to Unity Catalog. Includes automated assessment, migration notebooks, Terraform infrastructure-as-code, planning tools, and operational guides.

---

What's Included

Databricks Notebooks

| Notebook | Purpose |

|----------|---------|

| pre_migration_assessment.py | Scans Hive Metastore, catalogs all databases/tables/views/permissions, generates assessment report |

| automated_table_migration.py | Migrates tables from Hive to Unity Catalog using CTAS with full validation |

| permission_mapper.py | Maps legacy Hive permissions to Unity Catalog grants |

| post_migration_validation.py | Validates migration integrity: row counts, schema comparison, data sampling |

Terraform Module

| File | Purpose |

|------|---------|

| unity-catalog-setup/main.tf | Full Unity Catalog metastore, catalogs, schemas, external locations |

| unity-catalog-setup/variables.tf | Configurable variables for all resources |

| unity-catalog-setup/outputs.tf | Outputs for downstream consumption |

CLI Tools

| Tool | Purpose |

|------|---------|

| migration_planner.py | Generates migration waves from assessment output |

| timeline_estimator.py | Estimates migration timeline based on table count and data volume |

Guides

| Guide | Purpose |

|-------|---------|

| migration_guide.md | Step-by-step migration walkthrough |

| day2_operations.md | Post-migration Unity Catalog operations |

| rollback_procedures.md | Rollback procedures for each migration stage |

Templates

| Template | Purpose |

|----------|---------|

| migration_planning_spreadsheet.csv | CSV template for migration wave planning |

---

Quick Start

1. Run the Pre-Migration Assessment

Import notebooks/pre_migration_assessment.py into your Databricks workspace and run it. This produces a JSON assessment report cataloging every database, table, view, and permission in your Hive Metastore.

2. Plan Migration Waves

Use the CLI planner to generate migration waves from the assessment:


python tools/migration_planner.py \
  --assessment-file assessment_report.json \
  --max-tables-per-wave 50 \
  --output waves.json

*... continues with setup instructions, usage examples, and more.*

📄 Code Sample .py preview

notebooks/automated_table_migration.py # Databricks notebook source # MAGIC %md # MAGIC # Automated Table Migration # MAGIC **Unity Catalog Migration Kit — Datanest Digital** # MAGIC # MAGIC Migrates tables from Hive Metastore to Unity Catalog using CREATE TABLE AS SELECT (CTAS) # MAGIC with built-in validation, error handling, and progress tracking. # MAGIC # MAGIC **Prerequisites:** # MAGIC - Unity Catalog metastore assigned to workspace # MAGIC - Target catalog and schemas created (use the Terraform module or create manually) # MAGIC - Run `pre_migration_assessment.py` first to generate the assessment report # COMMAND ---------- # MAGIC %md # MAGIC ## Configuration # COMMAND ---------- dbutils.widgets.text("target_catalog", "", "Target Unity Catalog Name") dbutils.widgets.text("source_database", "", "Source Hive Database") dbutils.widgets.text("target_schema", "", "Target Schema (leave empty to match source)") dbutils.widgets.text("table_filter", "*", "Table Filter (glob pattern)") dbutils.widgets.dropdown("migration_mode", "ctas", ["ctas", "clone", "sync"], "Migration Mode") dbutils.widgets.dropdown("dry_run", "true", ["true", "false"], "Dry Run") dbutils.widgets.text("checkpoint_path", "/tmp/unity_catalog_migration/checkpoints", "Checkpoint Path") dbutils.widgets.dropdown("overwrite_existing", "false", ["true", "false"], "Overwrite Existing Tables") target_catalog: str = dbutils.widgets.get("target_catalog") source_database: str = dbutils.widgets.get("source_database") target_schema: str = dbutils.widgets.get("target_schema") or source_database table_filter: str = dbutils.widgets.get("table_filter") migration_mode: str = dbutils.widgets.get("migration_mode") dry_run: bool = dbutils.widgets.get("dry_run") == "true" checkpoint_path: str = dbutils.widgets.get("checkpoint_path") overwrite_existing: bool = dbutils.widgets.get("overwrite_existing") == "true" assert target_catalog, "target_catalog is required" assert source_database, "source_database is required" print(f"Source: hive_metastore.{source_database}") print(f"Target: {target_catalog}.{target_schema}") print(f"Mode: {migration_mode}") print(f"Dry run: {dry_run}") print(f"Overwrite: {overwrite_existing}") # COMMAND ---------- # MAGIC %md # ... 282 more lines ...

Buy Now — $69 Back to Products