Fabric and Azure

How and When to Use Dataflows in Power BI

Use a dataflow when several Power BI models need the same cleaned tables. How Gen1 and Gen2 differ, what each needs to run, and how to build one.

Austin Levine

·

Updated

A dataflow is a set of Power Query queries that runs in the Power BI or Fabric service instead of inside one report file. Use one when several semantic models need the same cleaned tables, such as Customer, Product or Date, so the cleaning logic is written once and every model reads the result. If only one report uses the data, keep the transformations in Power Query in Power BI Desktop.

There are two kinds. Dataflow Gen1 is the original Power BI dataflow and runs with Power BI Pro. Dataflow Gen2 is the Microsoft Fabric version, needs Fabric or Premium capacity, and can write its output to a lakehouse, warehouse or other destination. Microsoft recommends Gen2 for new work.

What dataflows are in Power BI

A dataflow holds queries written in Power Query (M), the same language as the Power Query Editor in Desktop. When the dataflow refreshes, the service runs the queries and stores the resulting tables.

The output is a set of tables, not a model. A dataflow has no relationships and no measures. A semantic model connects to the dataflow, imports its tables, and adds the relationships and DAX on top.

You build dataflows in the browser, in a workspace in the Power BI service or Fabric. You cannot create one in Power BI Desktop, but Desktop can read from one.

Dataflow Gen1 vs Gen2


Dataflow Gen1

Dataflow Gen2

Runs on

Pro, Premium Per User or Premium

Fabric capacity, Fabric trial or Premium capacity

Where output goes

Dataflow storage, read with the dataflow connector

Dataflow storage, or a destination such as a Fabric lakehouse, warehouse or Azure SQL database

Incremental refresh

Premium or PPU only

Yes

Linked and computed tables, enhanced compute engine

Premium or PPU only

Not applicable; Gen2 uses Fabric compute

DirectQuery from a semantic model to the dataflow

Yes, on Premium

No

Works in Fabric pipelines

No

Yes

Git integration and deployment

No

Yes, by default for new items

Two practical rules follow:

  • If your organization has only Power BI Pro licenses, Gen1 is what you can use.

  • If you have Fabric capacity, build new dataflows as Gen2. Existing Gen1 dataflows can be moved with Save As, or by exporting the queries as a template and importing them.

How to use dataflows in Power BI

Create the dataflow

  1. In the workspace, select New item and choose the dataflow type: Gen1, or Gen2 if the workspace is on Fabric capacity.

  2. Select Get data and choose a source, such as SQL Server, SharePoint or a REST API. On-premises sources need an on-premises data gateway.

  3. Build the queries in the online Power Query editor. The steps work the same way as in Desktop.

  4. For Gen2, optionally set a data destination for each query, such as a lakehouse table.

  5. Save or publish the dataflow, then set a refresh schedule in its settings.

A shared Customer query in a dataflow might look like this:

let
    Source = Sql.Database("myserver.database.windows.net", "CRM"),
    Customers = Source{[Schema = "dbo", Item = "Customer"]}[Data],
    KeptColumns = Table.SelectColumns(Customers, {"CustomerID", "CustomerName", "Country", "Segment"}),
    TrimmedNames = Table.TransformColumns(KeptColumns, {{"CustomerName", Text.Trim, type text}}),
    RemovedDuplicates = Table.Distinct(TrimmedNames, {"CustomerID"})
in
    RemovedDuplicates

Every model that needs customers now gets the same columns, the same trimmed names and one row per customer.

Use the dataflow in a report

In Power BI Desktop, select Get data, choose the Dataflows connector, sign in, and pick the workspace, dataflow and tables. The tables import into your model like any other source. Build relationships and measures in Desktop, then publish.

If a Gen2 dataflow writes to a lakehouse, you can also connect the model to the lakehouse tables instead of the dataflow.

Refresh order matters. Schedule the dataflow to finish before the semantic models that read from it refresh, or the models load the previous run's output.

Key features of Power BI dataflows

  • Reusable preparation. One set of queries feeds many semantic models.

  • Many sources. The same connectors as Power Query, including databases, files, SharePoint, SaaS apps and web APIs.

  • Scheduled refresh. Each dataflow has its own schedule, independent of the models that use it.

  • Incremental refresh. Reload only recent data on each run. On Gen1 this needs Premium or PPU; see incremental refresh in Power BI for how the policy works on semantic models.

  • Output destinations (Gen2). Write tables to a lakehouse or warehouse so SQL users and notebooks can use them too.

Dataflows vs Power Query in Power BI Desktop

Both use the same Power Query engine and the same M language. The difference is where the queries live and who can reuse them.


Power Query in Desktop

Dataflow

Where it runs

Inside one semantic model

In the service, independent of any model

Who can reuse the output

That model only

Any model with access to the workspace

Refresh

With the model

On its own schedule

Best for

Report-specific shaping

Shared tables, used by several models

If you already write Power Query in Desktop, moving a query into a dataflow is mostly copy and paste. For Power Query itself, see our Power Query guide.

A dataflow adds a second refresh to schedule and a second item to monitor. That cost is worth paying when several models share the logic. It is not worth it for one report.

When to use dataflows in Power BI

Situation

Use

One report, its own data shaping

Power Query in Desktop

Several models need the same cleaned tables, Pro licenses only

Dataflow Gen1

Several models need the same tables, Fabric capacity available

Dataflow Gen2

Output should land in a lakehouse or warehouse for other tools

Dataflow Gen2

Large joins, nested JSON, or transformations that cannot fold to the source are a sign to reach for a Fabric notebook instead of a dataflow.

Specific signs a dataflow will help:

  • The same query is copied into several .pbix files. A fix has to be made in each file, and they drift apart.

  • Several teams define Customer or Product differently. One dataflow gives one definition.

  • A slow source is queried by several models. One dataflow refresh replaces several source queries.

  • Report builders should not need source credentials. They connect to the dataflow, and one owner manages the source connection.

For choosing between Gen2 and Spark notebooks for heavier work, see Dataflows Gen2 vs notebooks in Microsoft Fabric. For how Fabric and Power BI fit together, see Microsoft Fabric and Power BI, and for which features each license includes, see our Power BI license guide.

FAQs

What are the different uses of Dataflows?

The main uses are sharing one cleaned table across several semantic models, centralizing source credentials with one owner, reducing load on a slow source by querying it once, and, with Gen2, landing prepared data in a lakehouse or warehouse for SQL and notebook users. Dataflows prepare data. They do not hold relationships or measures.

What is Dataflow vs. dataset in Power BI?

A dataflow produces tables. A dataset, now called a semantic model, holds tables plus relationships, measures and security, and it is what reports query. A common pattern is a dataflow that prepares Customer and Product, and a semantic model that imports those tables, relates them to sales, and adds the DAX.

Sources

A dataflow is a set of Power Query queries that runs in the Power BI or Fabric service instead of inside one report file. Use one when several semantic models need the same cleaned tables, such as Customer, Product or Date, so the cleaning logic is written once and every model reads the result. If only one report uses the data, keep the transformations in Power Query in Power BI Desktop.

There are two kinds. Dataflow Gen1 is the original Power BI dataflow and runs with Power BI Pro. Dataflow Gen2 is the Microsoft Fabric version, needs Fabric or Premium capacity, and can write its output to a lakehouse, warehouse or other destination. Microsoft recommends Gen2 for new work.

What dataflows are in Power BI

A dataflow holds queries written in Power Query (M), the same language as the Power Query Editor in Desktop. When the dataflow refreshes, the service runs the queries and stores the resulting tables.

The output is a set of tables, not a model. A dataflow has no relationships and no measures. A semantic model connects to the dataflow, imports its tables, and adds the relationships and DAX on top.

You build dataflows in the browser, in a workspace in the Power BI service or Fabric. You cannot create one in Power BI Desktop, but Desktop can read from one.

Dataflow Gen1 vs Gen2


Dataflow Gen1

Dataflow Gen2

Runs on

Pro, Premium Per User or Premium

Fabric capacity, Fabric trial or Premium capacity

Where output goes

Dataflow storage, read with the dataflow connector

Dataflow storage, or a destination such as a Fabric lakehouse, warehouse or Azure SQL database

Incremental refresh

Premium or PPU only

Yes

Linked and computed tables, enhanced compute engine

Premium or PPU only

Not applicable; Gen2 uses Fabric compute

DirectQuery from a semantic model to the dataflow

Yes, on Premium

No

Works in Fabric pipelines

No

Yes

Git integration and deployment

No

Yes, by default for new items

Two practical rules follow:

  • If your organization has only Power BI Pro licenses, Gen1 is what you can use.

  • If you have Fabric capacity, build new dataflows as Gen2. Existing Gen1 dataflows can be moved with Save As, or by exporting the queries as a template and importing them.

How to use dataflows in Power BI

Create the dataflow

  1. In the workspace, select New item and choose the dataflow type: Gen1, or Gen2 if the workspace is on Fabric capacity.

  2. Select Get data and choose a source, such as SQL Server, SharePoint or a REST API. On-premises sources need an on-premises data gateway.

  3. Build the queries in the online Power Query editor. The steps work the same way as in Desktop.

  4. For Gen2, optionally set a data destination for each query, such as a lakehouse table.

  5. Save or publish the dataflow, then set a refresh schedule in its settings.

A shared Customer query in a dataflow might look like this:

let
    Source = Sql.Database("myserver.database.windows.net", "CRM"),
    Customers = Source{[Schema = "dbo", Item = "Customer"]}[Data],
    KeptColumns = Table.SelectColumns(Customers, {"CustomerID", "CustomerName", "Country", "Segment"}),
    TrimmedNames = Table.TransformColumns(KeptColumns, {{"CustomerName", Text.Trim, type text}}),
    RemovedDuplicates = Table.Distinct(TrimmedNames, {"CustomerID"})
in
    RemovedDuplicates

Every model that needs customers now gets the same columns, the same trimmed names and one row per customer.

Use the dataflow in a report

In Power BI Desktop, select Get data, choose the Dataflows connector, sign in, and pick the workspace, dataflow and tables. The tables import into your model like any other source. Build relationships and measures in Desktop, then publish.

If a Gen2 dataflow writes to a lakehouse, you can also connect the model to the lakehouse tables instead of the dataflow.

Refresh order matters. Schedule the dataflow to finish before the semantic models that read from it refresh, or the models load the previous run's output.

Key features of Power BI dataflows

  • Reusable preparation. One set of queries feeds many semantic models.

  • Many sources. The same connectors as Power Query, including databases, files, SharePoint, SaaS apps and web APIs.

  • Scheduled refresh. Each dataflow has its own schedule, independent of the models that use it.

  • Incremental refresh. Reload only recent data on each run. On Gen1 this needs Premium or PPU; see incremental refresh in Power BI for how the policy works on semantic models.

  • Output destinations (Gen2). Write tables to a lakehouse or warehouse so SQL users and notebooks can use them too.

Dataflows vs Power Query in Power BI Desktop

Both use the same Power Query engine and the same M language. The difference is where the queries live and who can reuse them.


Power Query in Desktop

Dataflow

Where it runs

Inside one semantic model

In the service, independent of any model

Who can reuse the output

That model only

Any model with access to the workspace

Refresh

With the model

On its own schedule

Best for

Report-specific shaping

Shared tables, used by several models

If you already write Power Query in Desktop, moving a query into a dataflow is mostly copy and paste. For Power Query itself, see our Power Query guide.

A dataflow adds a second refresh to schedule and a second item to monitor. That cost is worth paying when several models share the logic. It is not worth it for one report.

When to use dataflows in Power BI

Situation

Use

One report, its own data shaping

Power Query in Desktop

Several models need the same cleaned tables, Pro licenses only

Dataflow Gen1

Several models need the same tables, Fabric capacity available

Dataflow Gen2

Output should land in a lakehouse or warehouse for other tools

Dataflow Gen2

Large joins, nested JSON, or transformations that cannot fold to the source are a sign to reach for a Fabric notebook instead of a dataflow.

Specific signs a dataflow will help:

  • The same query is copied into several .pbix files. A fix has to be made in each file, and they drift apart.

  • Several teams define Customer or Product differently. One dataflow gives one definition.

  • A slow source is queried by several models. One dataflow refresh replaces several source queries.

  • Report builders should not need source credentials. They connect to the dataflow, and one owner manages the source connection.

For choosing between Gen2 and Spark notebooks for heavier work, see Dataflows Gen2 vs notebooks in Microsoft Fabric. For how Fabric and Power BI fit together, see Microsoft Fabric and Power BI, and for which features each license includes, see our Power BI license guide.

FAQs

What are the different uses of Dataflows?

The main uses are sharing one cleaned table across several semantic models, centralizing source credentials with one owner, reducing load on a slow source by querying it once, and, with Gen2, landing prepared data in a lakehouse or warehouse for SQL and notebook users. Dataflows prepare data. They do not hold relationships or measures.

What is Dataflow vs. dataset in Power BI?

A dataflow produces tables. A dataset, now called a semantic model, holds tables plus relationships, measures and security, and it is what reports query. A common pattern is a dataflow that prepares Customer and Product, and a semantic model that imports those tables, relates them to sales, and adds the DAX.

Sources

Show us the report nobody trusts.

1 · A 30-minute call.

2 · We look at your current reports together.

3 · We tell you what we would do.

Show us the report nobody trusts.

1 · A 30-minute call.

2 · We look at your current reports together.

3 · We tell you what we would do.

Show us the report nobody trusts.

1 · A 30-minute call.

2 · We look at your current reports together.

3 · We tell you what we would do.

CaseWhen is a Berlin BI consultancy that builds reporting that leaders can trust, on the Microsoft stack: Power BI, Fabric and Azure.

Berlin, Germany

© CaseWhen Consulting GmbH

English

CaseWhen is a Berlin BI consultancy that builds reporting that leaders can trust, on the Microsoft stack: Power BI, Fabric and Azure.

Berlin, Germany

© CaseWhen Consulting GmbH

English

CaseWhen is a Berlin BI consultancy that builds reporting that leaders can trust, on the Microsoft stack: Power BI, Fabric and Azure.

Berlin, Germany

© CaseWhen Consulting GmbH

English