OneLake - Simply Explained

Welcome to the M365 FM podcast, where we explore the people, ideas and technologies shaping the future of work. Opening the cost of copying data. Picture this one. Sales table lives in a warehouse, a lake house, a power BI import model, and three team folders. That first copy started as a convenience, but pretty soon the numbers disagree. Reports go stale and someone inherits another refreshed job. Storage isn't where the pain lives. Trust is. Welcome to the M365 FM podcast, where we explore the people, ideas and technologies shaping the future of work. We're looking at one lake and this idea of shared data, because one stored copy can still support every team's preferred way of working without the mess. The mess one lake was built to stop. Most organizations never planned to build that maze of data copies. It starts innocently enough. One team trying to solve a real problem fast finance needs a sales report, so they pull the data into their own workspace. Operations needs a daily view, so they spin up another pipeline product, wants to study customer behavior, and their team builds a lake for that work. Each of those decisions looks reasonable on its own, but when you step back, you get separate lakes, separate pipelines, separate permissions, and separate versions of the same business fact. Take a simple question how much did we sell last month? The SQL team might grab the number from a warehouse. A data engineer pulls from a spark table. After a transformation job, an analyst opens a power BI report that refreshed overnight. A business app reads from its own extracted data set. Those answers should match, but sometimes they don't. Picture this one pipeline runs at six in the morning, another fails and retries at noon. The warehouse excludes canceled orders, but the report uses an older rule. The sales team corrects a record in the source system, yet only one out of five downstream copies ever sees the change. Suddenly, the meeting isn't about sales anymore. It becomes a debate about which number you can trust. That debate costs more than storage space. People burn hours tracing columns through pipelines, comparing refresh timestamps, and hunting down the person who owns a table nobody remembers creating. Meanwhile, each extra copy demands access controls, monitoring backup plans, and someone on call. When the load breaks, the technical debt piles up quietly. A pipeline depends on a table. That table depends on another copy. Someone changes a column name upstream and a report breaks two systems away. The fix takes ten minutes. Finding the responsible person that can take days. You might think the answer is forcing everyone into one tool. Lock every team into SQL mandate, spark for every engineer, or enforce one model for all reports. But here's the head fake that doesn't solve the real problem. It just moves the frustration from data movement to people. A SQL developer needs a SQL experience because that's how they build and query data. A data engineer needs spark for large scale processing and notebooks. Analysts need reports that answer business questions without inspecting files or writing code. Applications need dependable access in the format they expect. The experience can differ without creating another physical copy and that separation matters. Storage answers the question where does the data live? The tool answers how does this person work with it? Once we separate those two questions, the mess starts looking fixable. But a single shared place can sound like a disaster waiting to happen. One giant folder. Unclear ownership. Everyone stepping on everyone else's work fabric addresses that concern with structure, so teams can share a foundation without giving up their boundaries. One lake, one logical lake. Many teams you might think the hardest part of data storage is just finding a place to put everything, but the real tension is that every team needs their own space while still sharing the same numbers. One lake solves that by giving your whole organization one logical data lake inside Microsoft fabric. The word logical matters here. You're not looking at one physical hard drive sitting in a single server room. Instead, you get one connected view of all your organizational data inside fabric spread across accounts and departments, but looking like a single lake. And here's the part that catches people off guard. When fabric is available in your tenant, one lake is already there. You don't provision a storage account. You don't create a lake, choose its infrastructure, or ask someone to maintain the base platform. Each fabric tenant has exactly one one lake. That simple rule removes a surprising amount of early architecture. Debate Teams spend less time asking where the next lake should live because the shared storage layer already exists. The question shifts from where do we put it to how do we organize and govern the data that enters it? Microsoft often compares one leg to OneDrive for data, and the comparison helps if you take it literally. OneDrive doesn't force every department into one document or one folder. It gives the organization a common service. While people still work in spaces they can understand and control. One lake follows that same file and folder model. Under the tenant boundary, you organize data through domains, then workspaces, then fabric items, then the files and tables inside those items. Each level gives people a clearer place to look and a clearer boundary for responsibility. A domain groups work around a business area like finance, operations, or product. Within that domain. Workspaces let teams create and manage the fabric items they need. Those items might include a lake house or a warehouse, and beneath them sit the actual files and tables. Think about what that changes for a finance team. Finance owns its workspace managers who can enter it and takes responsibility for the data products it publishes. Operations doesn't need to take over that workspace just to use the same overall storage foundation product, keeps its own workspace rules and day to day work as well. So this isn't centralized control pretending to be collaboration. It's distributed ownership inside a shared organizational boundary. The tenant administrator controls the outer boundary, but every team doesn't need to wait for one central data group to create every folder, table, or item that matters. Because data ownership works best close to the people who understand the business process behind it. Finance knows what an approved revenue number means. Operations knows how inventory status changes during the day. Product knows which customer events belong in a usage table. One Lake gives those teams room to own their part without isolating it from everyone else. Inside a workspace, fabric items provide the experience that fits the job. A lake house gives engineers a place to work with files and tables. A warehouse gives SQL focused teams, schemas and tables in a familiar environment. They feel different because the work is different. But here's the head fake underneath. Both connect to one lake. You aren't choosing between a lake lakehouse in one storage world and a warehouse somewhere else. You're choosing the way people interact with data stored in the same organizational lake. That's the part many people miss. One lake doesn't demand that every person work the same way. It creates a common storage layer, while fabric gives each team an interface that matches its skills and needs. But shared storage alone won't solve much. If each engine needs its own converted version before it can read the data. The next question is how one stored data set can work for SQL Spark and Power BI without starting another copy cycle. One copy multiple ways to work. One copy of data can sound restrictive. At first, you might hear it and think one team owns the table, one tool reads it, and everyone else has to wait in line. That isn't the model. One copy means one stored data set can support many kinds of work, while the teams using it keep the tools that fit their jobs. The shared table sits beneath fabric items in an open format called Delta parquet. Delta parquet lets different engines recognize the same table without each team exporting it, converting it, and storing another version somewhere else. The data stays in place, but the way people work with it can change. For a civil engineer, that means a warehouse can still feel like a warehouse. They use t-SQL. Work with schemas and tables, load data, and run the queries they already know behind that familiar experience. The table lives in one lake. The SQL team doesn't need to turn into a spark team just to take part in a shared data model. At the same time, a data engineer opens that same data through Apache Spark. Spark gives engineers and data scientists the tools they need for larger transformations, notebooks, and open source libraries. They read the Delta table, write changes back in the same format and hand the result to the next person without another export job sitting in the middle. That sounds small until you look at the usual alternative. A spark process creates a transform table. Someone copies it into a SQL store. Then an analyst imports it into power BI. Each handoff creates a delay and a new place where the data can drift away from the source. With one lake, the power BI side changes as well. A semantic model uses direct lake to access data in one lake without creating a separate import copy for the business user, the report still feels fast and familiar. They open a dashboard filter sales by region and ask why a number changed. They don't need to know whether spark or SQL prepared the table. The real tension here is that teams often mistake flexibility for duplication. They assume every engine needs its own stored version of the data, because that was often how platforms worked in the past. Fabric separates the engine from the stored table. You choose the engine for the task while preserving one approved source. Picture a sales data set arriving from a source system. It lands once in one lake. A data engineer uses spark to clean customer IDs, handle missing values, and prepare the table for analysis. The warehouse team then queries that same prepared table with t-SQL to answer detailed sales questions. Power BI reads it through a semantic model using direct Lake and leadership sees the results in a report. One data set different work no chain of copies passing from team to team. That doesn't mean every person can change the table whenever they want. One copy doesn't remove ownership. It doesn't remove data quality checks, approval steps, or access rules. In fact, those things become more important when more people depend on the same source. Someone still needs to define what a sales record means. someone needs to decide how canceled orders appear. Someone needs to check whether the load completed and whether the table is fit for reporting. Access still needs to match the job, so a person who can view a report doesn't automatically gain permission to alter the data behind it. One lake reduces duplication, it doesn't remove responsibility, and there's another limit worth keeping in view. One stored copy inside fabric helps a lot, but most organizations already keep data and other systems, clouds, and business platforms. Copying all of that data into fabric just to make it visible would create the same problem in a new place. Shortcuts data where it lives. A shortcut solves the problem of data that shouldn't move at all. Think about it. Data that belongs to sales should stay under sales ownership, but finance needs it to. The instinct says to copy it, creating another copy and another refresh schedule. But here's the head fake. You don't need a copy. A shortcut is a virtual reference to a file, folder or table in another location. It appears right where your team needs to work while the data stays at its source. Fabric just gives you a path to it. Take finance as an example. The sales team owns an approved sales table in its own workspace, with all the rules and checks that turn raw orders into accepted numbers. Finance needs that table for forecasting, but rebuilding the sales pipeline inside a finance workspace means another copy and another schedule to maintain. Instead, finance creates a shortcut to that approved table. From their point of view, the table appears in their workspace, ready to combine with budgets and forecasts. Sales still owns the source table, and nobody has to build a second pipeline just to move sales data across an internal boundary. That is an internal shortcut. It lets one workspace reuse a table or folder from another workspace. Even when teams sit in different business domains, the data stays under the ownership of the team that understands it, while other teams consume it where it fits their work. Shortcuts also reach beyond fabric. A team can create references to data in Azure Data Lake Storage, Amazon S3, or Dataverse. So if customer data already sits in Dataverse or an existing data platform stores files in ADLs or S3, you don't automatically need a copy in one lake before people can work with it. You might think that means one leg pulls every outside system into fabric. It doesn't. The source stays where it lives. The shortcut makes that source visible through one lake, which reduces the pressure to move data simply because another team needs it. That changes the conversation instead of when can we copy this table? The question becomes, can we safely use the approved source where it already lives? Fewer copies mean fewer scheduled loads, fewer failed refresh chains, and fewer moments where two teams look at two versions of the same sales number. But here's where the twist comes in. A shortcut isn't a magic permission pass. Creating one doesn't give someone blanket access to everything behind it. Access rules still matter, and people only see or use data where their permissions allow it. That boundary matters because shared access should expand the use of trusted data, not quietly open sensitive data to the entire organization. The shortcut also depends on the source. If the source data arrives late, changes structure without warning or contains poor quality records, the shortcut won't fix any of that. It removes unnecessary movement, but it can't replace an owner who defines the table, monitors its freshness, and tells consumers what they can trust, so use shortcuts to avoid building copies by default. Then confirm who owns the source who can access it, what happens when it changes, and how consumers know whether the data is current. A shortcut gives you a cleaner path to the data, but it doesn't remove the need to manage that path. Shared access only helps when people can find the right data, understand what it means, and control who can use it. How fabric makes the lake usable. A shared data lake only works if people can actually work with it. The big story here is that fabric handles that by giving different teams familiar ways to interact with the same storage layer, rather than asking everyone to become a data engineer for engineering work. A lakehouse gives you two areas files for raw material like CSV, Json images or logs, and tables for delta tables that spark can read and write. An engineer can land raw data, clean it, and turn it into tables without leaving the lakehouse experience. SQL teams see a different view. A fabric warehouse gives them schemas, tables, and t-SQL so they can query and manage data in the way they already understand. The warehouse feels familiar, but its data still sits in one lake underneath. You don't need a separate storage plan just because one team speaks SQL and another works in spark. Here's the head fake. A lake house and a warehouse aren't competing storage islands. They are different working surfaces over the same data. Choose the surface that fits the job. Some people won't start in fabric at all. One Lake File Explorer Let's permitted users browse workspaces and data files from Windows File Explorer. They can move through folders they have access to and work with files in a familiar interface that lowers the barrier for users who understand business data but don't want to begin with a notebook or command line. Fabric also stays open to tools outside its own interface. One lake supports the same ADLs Gen2 APIs and APFs paths used by compatible tools, including Azure Databricks. If your team already has a process that can read and write through those paths, it can connect to one leg rather than forcing another export into a separate storage location. Access doesn't mean everyone should see everything. Finding the right data set starts with structure. Workspaces. Group A teams work domains group work around business areas. Item permissions. Narrow access to the specific data, product or tool a person needs. Data catalog views help people search across available items and understand what they are looking at before they build something new. Keep access narrow by default. If an analyst only needs an approved table, grant access to that table or item. Rather than handing over broad workspace access. A broad role can feel faster in the moment, but it often creates clean up work later and makes it harder to see who can reach sensitive data. One more separation helps. One lake holds the data. Fabric capacities provide the compute that runs workloads against it a capacity, power, spark jobs, SQL activity reports, and other processing, storage, and Compute Connect. But they aren't the same decision. You can manage workload power without treating every data store as a separate infrastructure project. The technology can cut out many unnecessary copies. People still need to change the habit that created those copies in the first place, but that's where the real opportunity lies. When you stop treating every data need as a new copy project, you free your team to focus on what the data actually means. Putting one copy into practice. The real tension here isn't about migrating everything at once. It's about one high use data set where duplicate pipelines are already causing delays, failed refreshes, and arguments over which report is the real one. Sales, customer inventory, or finance data often exposes the problem quickly because multiple teams are fighting over who's version is current. You might think the solution is a massive migration plan, but that's the head fake. The actual first move is much simpler. Pick a single data set and start there. But before you build a shortcut, you need an owner. That owner has to publish the approved source, define what it means and tell consumers when they can trust it. Without that decision, a shared table just becomes a better connected source of confusion. Now here is where it gets interesting. Each team can keep the tool that fits their work while shared tables move toward delta parquet in one lake. The goal isn't to standardize everyone's workflow. The goal is to stop creating separate stored versions. Just because a different team needs a different interface. So before a new pipeline appears, ask whether a shortcut can connect the team to the approved source. Check the access rules, who owns the source and how freshness is monitored. If those answers aren't clear, don't hide the problem behind another copy. Track the practical results. Count the duplicate pipelines that disappear, the stored copies that no longer need refreshes, and how long it takes a team to reach approved data. Listen for fewer meetings spent defending competing numbers. Copying data can feel safe because each team controls its own version, but every new version becomes another thing someone has to refresh, secure, explain, and eventually fix. Conclusion. Stop building copies. So here's what it all comes down to. One Lake changes the question from where should this team copy the data to who owns the trusted source that every team can use? But here's the takeaway that shift is the key to stopping the cycle of technical debt. And that's exactly why you should subscribe to M365 FM for practical ways to use fabric, power, platform and AI without building more technical debt into the work your teams depend on.