← Blog

SQLMesh and SDT: transformation vs schema deployment

July 7, 2026 · 6 min read

If you are evaluating SQLMesh, you are looking at one of the best-designed tools in the modern data stack. It is a genuine step past dbt: state-aware, with a Terraform-style plan / apply loop, virtual environments, and column-level lineage that ships in the open-source core with no hosted catalog attached. The community around it is sharp and the engineering is honest.

So this is not a takedown. It is a boundary. SQLMesh and SDT/DDT do two different jobs, and the fastest way to waste a week is to expect either one to be the other. This post draws the line precisely, with real SQL, so you can decide what you actually need.

What SQLMesh is genuinely good at

SQLMesh is a transformation framework. You write models — SQL SELECT statements — and SQLMesh manages the graph between them: what depends on what, what needs rebuilding when a model changes, and how to backfill incrementally without reprocessing history you already have.

Its standout ideas are real and worth naming plainly:

• Virtual environments. A dev environment points views at existing physical tables instead of re-materializing everything, so only the models you actually changed get built. Promotion to prod is a view swap, not a re-run.
• Column-level lineage in OSS. It parses your SQL and tells you which upstream column feeds which downstream column — no vendor catalog required.
• Interval-aware incrementals. It tracks which date ranges a model has processed, so restarts and gaps are handled correctly instead of by hand.

If your problem is "I have a DAG of transformations and I want them to rebuild correctly and cheaply," SQLMesh is an excellent answer. It is now stewarded by the Linux Foundation, and Tobiko Data publishes benchmarks showing large speed and cost wins over dbt Core on Snowflake. Take vendor benchmarks with the usual salt, but the architecture behind them is sound.

The scope boundary: models are not the catalog

Here is the boundary. A transformation tool models the SELECT graph — the tables and views your queries read and write. That is a subset of your database. It is not the catalog.

Your production schema contains objects that are not models and never appear in a transformation DAG. On Snowflake: streams, tasks, procedures, masking policies, row-access policies, warehouses, network policies, notification integrations. On Databricks Unity Catalog: the equivalent governance and compute surface. SQLMesh does not deploy these, because they are outside what a transformation framework is designed to own. That is not a defect — it is scope.

Consider a masking policy protecting PII, attached to a column your models read from:

-- Snowflake: a governance object no transformation graph manages.
CREATE MASKING POLICY pii_email AS (val STRING) RETURNS STRING ->
  CASE WHEN CURRENT_ROLE() IN ('COMPLIANCE') THEN val
       ELSE REGEXP_REPLACE(val, '.+@', '****@') END;

ALTER TABLE customers MODIFY COLUMN email
  SET MASKING POLICY pii_email;

SQLMesh can read the customers table in a model. It will not deploy that policy, diff it when the redaction logic changes, or notice if a migration silently detaches it. The same is true of the row-access policy that enforces multi-tenant isolation, the task that orchestrates a load, and the stored procedure that half your pipeline calls. Ask the concrete question: who deploys those, reviews their changes, and stops an unsafe one? For most teams the honest answer is "a hand-written migration script and hope."

Two different jobs, same repo

The framing that actually works is coexistence — the same way SQLMesh coexists with the objects dbt never managed. SQLMesh owns the model layer: the SELECT graph, backfills, incrementals. SDT (Snowflake) and DDT (Databricks) own full-schema deployment and safety for everything: the models' physical tables and every governance, compute, and stateful object around them.

This is not "rip out SQLMesh." If SQLMesh is building your transformation layer well, keep it. SDT/DDT sit underneath, holding the schema-as-code truth for the whole catalog. Both can live in one repo, and the boundary between them is clean: models are SQLMesh's; the rest of the DDL surface is SDT/DDT's.

The safety layer SDT adds

The part no transformation tool is built to do is refuse a dangerous deploy on the full object surface. SDT/DDT classify every proposed change into one of four verdicts, with a reason attached to each finding:

• UNRECOVERABLE — the platform has nothing to restore (a dropped stream's offset, a dropped Snowpipe's load history). Refused by default.
• DESTRUCTIVE — data loss that Time Travel can undo within retention (a dropped column). Refused without an explicit gate.
• EXPENSIVE — correct but costly (a full-table rewrite on a large table).
• WARNING — worth a human's eyes, not a blocker.

Because the classifier runs over the whole catalog, it catches the changes that live outside a model graph entirely:

$ sdt compare --project ./Warehouse --target prod

  - MASKING POLICY  ANALYTICS.SEC.PII_EMAIL     detaches from 3 columns  [DESTRUCTIVE]
  - STREAM          ANALYTICS.PUBLIC.ORD_STREAM cursor will be lost      [UNRECOVERABLE]
  ~ WAREHOUSE       LOAD_WH                     XL -> XS, may stall load [WARNING]

  1 UNRECOVERABLE, 1 DESTRUCTIVE change — publish refused.
  To proceed: --allowUnrecoverableDrop (drain ORD_STREAM first; see remediation)

A dropped masking policy is not a model change and will never surface in a transformation diff — but it is the kind of change that quietly exposes PII in production. The verdict, the reason, and the default refusal are the point.

One tool, both platforms, offline, Apache-2.0

SDT covers Snowflake; DDT covers Databricks Unity Catalog; they share one object model and one safety contract, so the compare-and-refuse behavior is identical across both. The deterministic core — compare, safe deploy, extract, lint, lineage — is Apache-2.0, runs fully offline, and never ships your schema to a server. SQLMesh's OSS core is likewise self-hostable; on that axis the two tools share a philosophy.

When SQLMesh alone is enough

Be honest with yourself here. If everything you deploy is transformation models — you have no streams, no tasks, no procedures, no masking or row-access policies, no warehouse or network config under version control, and your governance objects are managed elsewhere or not at all — then SQLMesh alone may be all you need. Adding a second tool to guard a surface you do not deploy is just overhead.

The case for SDT/DDT gets stronger exactly as your non-model surface grows: the more stateful and governance objects sit in production, the more you need something that deploys them as code and refuses to break them. Most teams past a certain size have that surface whether or not they have named it.

SDT and DDT are what I build. But the useful takeaway is platform-independent: your transformation tool models the SELECT graph, and your catalog is bigger than that graph. Whatever you use, make sure something deploys the rest of it — and can say no to a change that cannot be undone.

# Try the deterministic core against a schema you already have:
npm i -g @sdt-tools/cli
sdt schema extract --target prod --out ./Warehouse
sdt compare --project ./Warehouse --target prod

Independent tool, not affiliated with or endorsed by Tobiko Data or the SQLMesh project. “SQLMesh” and “Tobiko” are trademarks of their respective owners. “Snowflake” is a trademark of Snowflake Inc.; “Databricks” is a trademark of Databricks Inc.