dbt is where data work becomes software work

The warehouse changed. The workflow had to catch up.

For a long time, analytics lived in a weird middle place.

The business wanted trusted numbers. Analysts wrote SQL to produce them. Engineers owned production systems. BI tools became the place where definitions quietly multiplied. Everyone agreed that revenue should mean one thing, until the finance dashboard, sales dashboard, and board deck each had a slightly different answer.

The data usually existed.

The workflow was the problem.

dbt became popular because it gave analytics work a better operating model: version-controlled SQL, modular models, tests, documentation, lineage, and repeatable production workflows.

That sounds like tooling. It is really a shift in responsibility.

Analytics code stopped being a pile of useful queries.

It became software.

The old way: SQL as scattered knowledge

Every data team knows this version of the story.

A stakeholder asks, “What was net revenue last month?”

Someone finds a query from the last time this came up. It references a table with a name like orders_cleaned_final. There is a case when block that looks important. Nobody is totally sure who wrote it. The logic gets copied into a dashboard. Two weeks later, someone else copies a slightly different version into another dashboard.

Now the business has three answers to one question.

Nobody meant to create a metrics problem. They were just trying to move quickly.

This is how analytics debt forms:

• Business logic gets copied instead of reused

• Transformations live inside dashboards

• Metric definitions drift across teams

• Changes break downstream reports without warning

• Data quality checks happen after someone complains

• Documentation is either missing or written once and forgotten

The result is not just messy code. It is slower decision-making.

Every important question becomes a small investigation into whether the data can be trusted.

The dbt reframe: analytics needs an engineering workflow

dbt’s core idea is simple: analysts and analytics engineers should be able to work more like software engineers.

Not because every analyst needs to become a backend engineer.

Because analytics has the same production problems as software:

• Code needs review

• Logic needs reuse

• Changes need testing

• Dependencies need visibility

• Documentation needs to live close to the work

• Deployments should be repeatable

• Breakages should be caught before users find them

dbt brings those practices into the transformation layer. The project becomes a graph of models, sources, tests, metrics, and documentation.

That context is the real product.

The tables matter. The graph matters more.

Models make logic reusable

A dbt model is usually a SQL file that defines a transformation. Instead of burying logic in dashboards or one-off notebooks, the team builds a chain of models that turn raw data into trusted business objects.

A common pattern looks like this:

1. Sources: raw tables from production systems or ingestion tools

2. Staging models: cleaned, renamed, lightly standardized data

3. Intermediate models: reusable business logic and joins

4. Mart models: final datasets built for reporting, analysis, apps, or AI workflows

For example, a team might start with raw stripe_payments, salesforce_accounts, and product_events.

In dbt, they can turn those into:

• stg_stripe__payments

• stg_salesforce__accounts

• int_customer_revenue

• fct_recurring_revenue

• dim_customers

Now the revenue logic is not trapped in a dashboard tile. It has a name, an owner, dependencies, tests, and documentation.

That changes the conversation.

Instead of asking, “Which query did we use last time?”

The team can ask, “Is fct_recurring_revenue the right model for this question?”

Much better.

Tests move trust upstream

Most data quality issues are found too late.

A dashboard looks wrong. A stakeholder flags it. The analyst opens the SQL. Someone checks the upstream table. The pipeline ran, but a column started returning nulls. Or duplicate IDs appeared. Or a status value changed from paid to completed and broke a filter.

The team fixes it, but the trust damage already happened.

dbt tests help move that work upstream. They let teams express assumptions about models and data before those assumptions become dashboard incidents.

Out of the box, teams can check common expectations:

• A primary key should be unique

• A primary key should not be null

• A field should only contain accepted values

• A foreign key should match records in another model

And when the business has its own rules, teams can write custom tests as SQL.

That is the important part.

Testing is not just “does the pipeline run?” It becomes “does this data still mean what we think it means?”

Lineage turns breakages into investigations, not mysteries

Data pipelines break. That is not the interesting part.

The interesting part is how long it takes to understand what broke and who is affected.

Lineage gives teams a map.

That map is useful in two very normal situations.

First, something breaks.

A dashboard is wrong. Instead of spelunking through random SQL, the data team can move upstream through the DAG and see which models, sources, or tests are involved.

Second, something changes.

Maybe engineering renames a column. Maybe the finance team changes revenue recognition logic. Maybe the data team wants to refactor a staging model. Lineage helps answer the scary question:

“What will this break?”

Without lineage, every change feels risky.

With lineage, change becomes inspectable.

Still risky, because data work is data work. But inspectable.

Documentation works when it lives with the code

Documentation usually fails because it is separate from the work.

Someone creates a data dictionary. It gets stale. Someone makes a Notion page. It gets forgotten. Someone adds dashboard descriptions. They explain the dashboard, but not the transformation logic underneath it.

dbt pushes documentation closer to the models themselves.

That matters because the model is where the meaning lives.

A good model description explains what the dataset represents, how grain works, what each column means, and what assumptions the business logic makes. Good documentation does not need to be poetic. It needs to prevent the next person from making a bad guess.

For example:

This is not glamorous.

It is incredibly useful.

The best data teams do not document because documentation is virtuous. They document because future debugging is expensive.

The semantic layer is the next trust boundary

Once models are clean, another problem appears: metrics.

Models define useful datasets. Metrics define how the business measures itself.

Revenue. Active users. Churn. Gross margin. Pipeline. Retention.

If every downstream tool defines those separately, trust drifts again. The spreadsheet has one version. The BI tool has another. The AI agent, unless grounded properly, invents a third.

The dbt Semantic Layer is designed to centralize metric definitions on top of dbt models. This helps downstream tools and applications work from consistent business definitions instead of duplicating logic everywhere.

This becomes especially important as AI enters the analytics workflow.

An AI assistant that answers business questions is only as trustworthy as the context it can access. If “revenue” is just a word in a prompt, the answer is fragile. If “revenue” is a governed metric with a definition, dependencies, permissions, and lineage, the answer has something real to stand on.

That is the practical AI story for data teams.

Not magic.

Context.

Where dbt fits in the modern data stack

dbt does not ingest your data. It does not replace your warehouse. It does not magically make messy source systems clean.

It sits in the transformation layer.

A typical workflow looks like this:

1. Data lands in a warehouse or lakehouse from tools like Fivetran, Airbyte, custom pipelines, product databases, or event streams

2. dbt transforms that raw data into tested, documented models

3. BI tools, notebooks, reverse ETL tools, apps, and AI agents consume the trusted models and metrics

4. The dbt project becomes the shared context layer for how the business defines and trusts data

This is why dbt has become central to analytics engineering.

It is not just where SQL runs.

It is where the organization writes down what the data means.

What good dbt projects get right

A dbt project can still become a mess. It is very possible to create a beautiful DAG that nobody understands.

The best projects usually share a few habits.

They keep model layers clear

Staging models clean and standardize. Intermediate models handle reusable logic. Marts serve business-facing use cases.

When every model tries to do everything, the DAG becomes a junk drawer.

They test business assumptions

Not just primary keys.

Good teams test the rules that actually matter:

• Paid invoices should have payment dates

• Active subscriptions should have customer IDs

• Order totals should not be negative unless the record is a refund

• Closed-won opportunities should have close dates

Generic tests catch structural issues. Business tests catch semantic ones.

They document the weird parts

Every business has weird logic.

The problem is not weird logic. The problem is undocumented weird logic.

If churn excludes certain customer segments, say so. If revenue recognition changed in Q3, say so. If a column is deprecated but still used by an old dashboard, say so.

The weird parts are where trust usually breaks.

They treat refactoring as normal

A dbt project is never finished.

Models get renamed. Logic gets consolidated. Tests get added. Old marts get deprecated. Metric definitions evolve.

That is healthy. The goal is not to freeze the project. The goal is to make change safer.

The biggest mistake is treating dbt like a SQL folder

dbt can look deceptively simple.

It is SQL files, YAML, tests, docs, and a DAG. If you squint, it can feel like a nicer way to organize warehouse queries.

That is the trap.

A dbt project is not just a folder of transformations. It is the place where the data team encodes how the business works.

That means the hard questions are not only technical:

• What is the grain of this model?

• Who owns this definition?

• Is this logic reusable or one-off?

• Should this live in staging, intermediate, or marts?

• Which assumptions are stable enough to test?

• Which downstream teams depend on this model?

• What should happen when the definition changes?

The SQL matters. But the shape of the project matters more.

A messy dbt project can still produce clean tables. For a while.

Then the model count grows. Naming conventions drift. Intermediate models become permanent. Business logic gets duplicated. Tests exist, but only for easy things. Documentation explains columns, but not decisions.

Now the team has moved the mess from dashboards into dbt.

Better, maybe.

But not fixed.

dbt works best when it becomes a shared contract

The best dbt projects are not just technically correct. They create agreement.

A finance lead should be able to trust that fct_recurring_revenue means the same thing this month as it did last month. A growth PM should know which activation model powers their dashboard. An analyst should be able to change a staging model and understand what might break. An AI assistant should be able to answer a revenue question using the governed metric, not a random interpretation of the word “revenue.”

That is the contract.

Not a giant governance committee. Not a quarterly taxonomy project that everyone politely ignores.

A practical contract, written into the workflow:

• Models define reusable business objects

• Tests define what must stay true

• Docs explain meaning and assumptions

• Lineage shows dependencies

• Reviews catch risky changes

• Metrics centralize the numbers people argue about most

This is why dbt is such a natural home for analytics engineering.

It gives data teams a way to make trust operational.

The AI angle: context beats cleverness

AI makes this more important, not less.

If an AI agent is going to help with analysis, it needs more than access to the warehouse. Raw access is not intelligence. It is just a bigger surface area for confusion.

Ask an ungrounded assistant:

“Why did revenue drop last week?”

It might write a very confident answer. It might even query something. But what does “revenue” mean? Bookings? Recognized revenue? Net revenue? Gross revenue? Does it include refunds? What about test accounts? What about currency conversion? What about the enterprise contract that got backdated?

These are not edge cases.

These are the job.

The value of dbt in an AI workflow is that it gives agents structured context to work with:

• Certified models

• Metric definitions

• Column descriptions

• Model lineage

• Tests and failures

• Business logic

• Permissions and governance

That context changes the quality of the answer.

Without it, AI is guessing over tables.

With it, AI can reason over the way the business has already agreed to define itself.

Still not magic.

Much more useful.

What changes for the data team

When dbt is working well, the data team’s job changes.

They spend less time answering the same question in slightly different SQL. They spend less time reconciling dashboards. They spend less time explaining why two reports disagree.

They spend more time building the shared layer that makes future work faster.

That can look like:

• Turning repeated stakeholder questions into reusable marts

• Moving fragile dashboard logic into tested models

• Adding business-specific tests for high-value datasets

• Cleaning up model naming and ownership

• Documenting the weird parts before they become incidents

• Defining governed metrics for the numbers people use every week

• Reviewing changes before they break downstream workflows

This is the part that can feel invisible.

Nobody celebrates the test that caught a duplicate customer ID before the board dashboard refreshed. Nobody throws a party because a model description saved an analyst thirty minutes. Nobody writes a Slack announcement for a clean lineage graph.

But that is the work that compounds.

Every trusted model makes the next analysis easier.

Every test reduces the blast radius.

Every documented assumption saves someone from guessing.

Every governed metric makes the next dashboard, notebook, app, or AI answer less fragile.

The lesson: dbt is operational memory

Data teams have always carried a lot of organizational memory.

They know why the revenue number changed. They know which source table is unreliable. They know why churn excludes a certain segment. They know which dashboard is deprecated but still somehow used in the Monday meeting.

For a long time, that memory lived in people’s heads, old SQL, Slack threads, and tribal knowledge.

dbt gives teams a better place to put it.

Not all of it. There will always be context outside the transformation layer. But enough of it to make the work more durable.

The model remembers the logic.

The test remembers the assumption.

The documentation remembers the meaning.

The lineage remembers the dependency.

The metric remembers the agreement.

That is the real reason dbt matters.

It is not just a transformation tool. It is a way for data teams to turn scattered knowledge into shared infrastructure.

Where this goes next

The modern data stack has spent years getting data into the warehouse, transforming it, and showing it in dashboards.

The next phase is about making that data usable across more workflows: analysis, apps, operations, embedded reporting, customer-facing experiences, and AI agents that can actually do useful work.

That makes the transformation layer even more important.

Because the more places data goes, the more trust matters.

dbt sits at a critical point in that system. It is close enough to the raw data to shape it, and close enough to the business to encode what it means.

That is a powerful place to be.

The warehouse gave teams a central place to store data.

dbt gave teams a central place to define it.

And in a world where every tool wants to generate answers, that definition layer becomes the difference between faster insight and faster nonsense.

Getting started

If you are starting with dbt, do not begin by trying to model the entire company.

Start with one important business workflow.

Pick a recurring question people already argue about:

• What is revenue?

• Which customers are active?

• How do we define churn?

• What counts as a qualified lead?

• Which product events matter for activation?

Then build the smallest trusted path:

1. Define the raw sources

2. Create clean staging models

3. Build one business-facing model

4. Add tests for the assumptions that matter

5. Document the model and its columns

6. Expose it to the dashboard, notebook, app, or AI workflow that needs it

7. Use lineage to understand what depends on it before changing it

That is enough to start.

Not a perfect data platform. Not a giant transformation program.

Just one trusted path from raw data to a real decision.

Then do it again.