Databricks schema management, done safely

Why "CREATE OR REPLACE is safe" has edges

On Databricks, CREATE OR REPLACE TABLE is the recommended way to rebuild a table — a single transaction on the Delta log, history preserved, time travel intact. So engineers internalize "replace is safe, Delta keeps history," and then the state around the statement quietly proves the rule has edges: a managed table whose files get dropped, a streaming reader whose checkpoint is invalidated, an external table that behaves nothing like a managed one.

State-based schema management makes that state visible before you deploy. You declare desired-state .sql files per object and the engine diffs them against the live catalog — so the danger in the state around a statement becomes a mechanical review check, not a memory check.

The safety classifier knows Databricks

A generic diff tool calls every DROP "destructive" and leaves the judgment to whoever is reading the diff at 5pm on a Friday. DDT classifies each change into four tiers — SAFE, EXPENSIVE, DESTRUCTIVE, UNRECOVERABLE — using Databricks-specific knowledge:

• Replacing a managed table deletes underlying files; external tables retain them → UNRECOVERABLE vs DESTRUCTIVE.
• Replacing a table with streaming readers invalidates their checkpoint — they re-read from earliest → UNRECOVERABLE.
• The classifier reads catalog state (managed/external, readers) — the same diff line classifies differently by what it actually touches.
• A rebuild that rewrites files is flagged EXPENSIVE before you pay for the compute.

$ ddt extract --target databricks://prod --out ./Lakehouse.ddtproj
$ ddt compare --source ./Lakehouse.ddtproj --target databricks://prod

  ~ TABLE  sales.orders       replace (managed, 2 streaming readers)  [UNRECOVERABLE]
  - TABLE  sales.orders_ext   drop   (external; files retained)       [DESTRUCTIVE]
  + TABLE  sales.orders_v2                                            [SAFE]

  1 UNRECOVERABLE change — publish refused.
  Remediation: SHALLOW CLONE before replace; restart readers from new checkpoint.

Anything destructive or unrecoverable refuses to run unless you opt in explicitly — it is a structural gate, not a comment you scroll past.

Start free

DDT is priced identically to its sibling. The deterministic core is free forever; Pro and Team add compare, apply-in-CI, and rollback.