Moving from Hive metastore to Unity Catalog: what to prepare first
Most Unity Catalog migrations are decided before the first table moves. What to settle first, and which migration method fits which kind of table.
Moving a Databricks platform from the legacy Hive metastore to Unity Catalog looks like a metadata exercise: register the tables in a new catalog and point the jobs at them. In practice, the tables are the easy part. What decides whether the migration takes weeks or drags on for months is what has been settled before the first table moves: who the users are, where the data sits, which clusters run what, and how much code assumes the old way of working.
This article covers that preparation.
What goes wrong during and after the move is covered in common Unity Catalog migration mistakes.
What actually changes
Four changes drive most of the work.
A three-level namespace. Hive metastore addresses a table as schema.table. Unity Catalog adds a catalog in front: catalog.schema.table. Every query, job and report that uses two-level names has to be pointed at the right catalog.
One metastore per region, governed at the account level. A Unity Catalog metastore serves the workspaces in its region, and permissions are granted to identities defined at the Databricks account level rather than inside each workspace. Groups created locally in a workspace cannot be granted access to Unity Catalog data.
No DBFS for managed tables, no mount points. Unity Catalog does not store managed tables in DBFS, so managed tables in the Hive metastore have to be copied, not just re-registered. Mount points are not supported; they are replaced by external locations and volumes, and clusters no longer need instance profiles to reach storage.
Cluster access modes matter. Unity Catalog works on clusters in dedicated (formerly single user) or shared access mode. Clusters in “no isolation shared” mode, and clusters with credential passthrough enabled, cannot use it.
The Hive metastore keeps working alongside Unity Catalog during the transition, which is what makes a phased migration possible. Databricks’ own guidance is not to leave both in place for long, because two permission systems have to be kept in step.
Five things to settle before migrating
1. An inventory you can trust
Before anything moves, you need a complete list of what exists: databases and tables, whether each table is managed or external and where its files are, permissions, jobs, clusters, mount points and the service principals that access storage. Databricks Labs provides a tool for this, UCX, and we treat its assessment as a required first step. How UCX fits alongside the other free migration tools is covered in Databricks migration accelerators.
The inventory also answers the questions that size the project: how many tables are managed, how much data sits in DBFS, how many jobs use hard-coded paths, and which workloads nobody runs any more and can simply be retired.
2. Identities at the account level
Users and service principals created in a workspace are synchronised to the account automatically. Groups are not. Every workspace-local group that grants access today needs an account-level equivalent before Unity Catalog permissions can be assigned. The recommended setup is to provision users and groups from the identity provider into the Databricks account through SCIM, and to stop provisioning into individual workspaces.
This step also needs named people: an account admin, the cloud and IAM administrators, the identity provider administrator, the workspace admins and the future metastore admin. Migrations stall more often waiting for one of these people than on any technical issue.
3. A storage design
Unity Catalog reaches cloud storage through storage credentials and external locations. A storage credential on its own grants access to a whole container; an external location narrows that to a specific path. A practical pattern is one external location per container or storage account, with credentials scoped by domain and environment, for example a production credential for sales data.
Two rules are worth knowing early. Unity Catalog does not allow two tables to use the same storage location, which catches platforms where several tables were registered on one path. And the DBFS root bucket should not be reused as managed storage.
4. Clusters and runtimes
Clusters on older Databricks runtimes have to be rebuilt, not reconfigured. The minimum runtime version for Unity Catalog differs across Databricks’ own materials from different years, so check the current documentation for the features you need rather than relying on a number from an old slide. Cluster policies are the practical way to make sure new clusters are created in a Unity Catalog-compatible access mode.
Shared clusters carry limitations that single-user clusters do not, for example around R and some kinds of user-defined functions. If a team depends on those, plan their clusters explicitly.
5. The code
A code review for the upgrade looks for: two-level table names, references to mounts, direct use of dbutils.fs and file paths, sqlContext and sparkContext, user-defined functions in Scala, Python and Java, init scripts, JAR libraries on shared clusters, streaming checkpoints stored in DBFS, and pipelines. The volume of this work depends heavily on whether schemas are parameterised or hard-coded, and on whether the team has CI/CD; projects without it should be estimated separately.
Jobs should run as service principals rather than as individual users, and before an old table is dropped, the dependency check is simple: revoke access to it and re-run the queries that might still use it.
Which method for which table
Databricks’ material maps each kind of table to a method:
| Hive metastore table | Stored in | Target | Method |
|---|---|---|---|
| Any | DBFS root | Managed or external | Copy with CTAS or DEEP CLONE |
| Any | Mounted or cloud storage | External | SYNC, which registers the table in place |
| Any | Mounted or cloud storage | Managed | Copy with CTAS or DEEP CLONE |
| Hive SerDe format | Any | Any | Copy with CTAS or DEEP CLONE |
SYNC upgrades external Delta and Parquet tables without moving data, for a whole schema or table by table, and can be run repeatedly while both metastores coexist. Copying only works for data in Delta format. Views have to be recreated after the tables they depend on.
We scope an upgrade at one of three levels of ambition. The simplest moves external tables and the compute that only reads them, and is easily reversible. The second adds managed tables and the jobs that write to them. The third covers all jobs, pipelines and external orchestration. Starting at the first level and moving high-value, lower-risk workloads first means nobody discovers a limitation in the middle of a project.
For platforms with tens or hundreds of workspaces, a phased pattern keeps a Unity Catalog zone and a legacy zone running side by side while workloads move across. Databricks has also made Hive metastore federation generally available, which lets Unity Catalog govern tables that stay in an existing metastore; whether it suits a given platform is a question for the assessment.
How long it takes
Duration is decided by the variables above: how many managed tables must be copied, how much code uses mounts and two-level names, and how quickly the right people can make identity and storage decisions.
Where we start
Our Migration Scan runs the assessment on your workspaces and turns the inventory into a plan: which tables move by which method, which identities and storage decisions are needed and from whom, and an estimate of the code changes. The migration itself sits in the Modernize stage, and the access model that Unity Catalog makes possible is in Govern.
Sources: Databricks documentation on upgrading to Unity Catalog and UCX by Databricks Labs, read on 16 October 2026. Features and limits change between runtime versions; check the current documentation before planning.