M365Con 2027 – free online conference · 11–22 Jan 2027 · 120 sessions m365con.net

Mapping data flows

Product

Visually designed, no-code data transformations in Azure Data Factory that run as activities inside pipelines on Apache Spark clusters managed by the service.

◆ PRODUCTMapping data flowsM365.FM
Title graphic: Mapping data flows (Product)

Also discussed

The episode about Mapping data flows also covers Integration runtime and Azure Data Factory.

Description

Mapping data flows are visually designed data transformations in Azure Data Factory. According to Microsoft Learn, the page also applies to Azure Synapse Analytics. They let developers build transformation logic without writing code. The service takes care of code translation, path optimization and the execution of the data flow jobs, which run on scaled-out Apache Spark clusters that the service manages.

A mapping data flow does not run on its own. It is executed as an activity inside a pipeline. In that pipeline activity, the integration runtime and the parameter values are chosen. Because of this, a data flow can use the scheduling, control flow and monitoring features that already exist for Azure Data Factory pipelines, rather than needing a separate mechanism for orchestration.

The authoring experience is built around a canvas with three main areas. A top bar offers functions for saving, validating and viewing the JSON or script of the data flow. A graph shows the stream from source to sink, so the path of the data through the transformations is visible at a glance. A configuration panel appears for whichever transformation is selected. Each transformation provides tabs for its own settings, for partition optimization, for a read-only metadata inspection and for a data preview.

There are a few practical constraints noted in the documentation. The data preview works only when debug mode is switched on. The inspect tab shows no metadata when the source has no defined schema, which is typical for schema drift. Anyone designing flows against sources with changing structures should expect this behavior when inspecting metadata during development.

In typical situations, mapping data flows are chosen when transformation logic should be expressed visually instead of in code, while the surrounding pipeline handles triggers, scheduling and monitoring. The decision is mostly about where transformation work should live: in a graphical stream of transformations executed on managed Spark, or elsewhere in the data platform. The documentation also points to Dataflow Gen2 in Microsoft Fabric as a comparable technology, which is relevant for teams weighing Azure Data Factory against Fabric.

The M365.FM episode "Azure Data Factory - Simply Explained" covers mapping data flows under the heading "Transforming Without Code". It places them alongside the other building blocks of the service: pipelines and activities, linked services and datasets, integration runtimes, and triggers and scheduling. The same episode also compares Azure Data Factory with Fabric pipelines and discusses costs and best practices.

The episode "What Microsoft Fabric Means for Power BI Professionals" mentions mapping data flows in a learning outline for data ingestion and integration in Fabric. There, learning mapping data flows and data transformation is listed together with exploring connectors and practicing ingestion into OneLake, using Data Factory capabilities in Fabric. This shows how the concept appears in both the Azure Data Factory context and in discussions of the Fabric data platform.

Overview compiled with AI from M365.FM episodes and the facts on this page. You can improve it with the edit button below.

TypeProduct

Key points about Mapping data flows

Key points about Mapping data flows from the episodes, each with its source episode:

Episodes about Mapping data flows

Episodes in which Mapping data flows is discussed:

Frequently asked questions

Questions marked with AI were written with AI from the facts on this page.

What are mapping data flows in Azure Data Factory? (AI)

They are visually designed data transformations that require no code. The service translates the design and runs it on scaled-out Apache Spark clusters that it manages.

How does a mapping data flow run? (AI)

It runs as an activity inside a pipeline, where the integration runtime and parameter values are selected. This lets it use the existing scheduling, control flow and monitoring features of pipelines.

Why is the data preview or metadata sometimes not shown? (AI)

The data preview works only when debug mode is switched on. The inspect tab shows no metadata when the source has no defined schema, which is typical for schema drift.

Glossary

Community wiki

Short info above, community knowledge below: add practical tips, examples or corrections. New contributions are reviewed first; trusted members publish directly.

Experts and companies for this topic

Nobody has listed this topic yet. Add it to your profile.

Sources

Maintained by Mirko Peters (Microsoft MVP) · Last updated: