How does MS Fabric change enterprise data analytics?

Data has been compared, ad nauseum, with all kinds of valuables, like oil and gold, and is part of many company strategies. With the explosion of AI, the need for a reliable and future-proof data platform is becoming a must-have for most companies. Still, many of our clients struggle to adopt data-driven practices at the root of their organizations. Often, the development of new real-time reports takes too much time. The reasons for this are twofold; IT lacks the resources to keep up with business user demands, and business users do not have the right tools or the skills to create these reports themselves. 

Furthermore, having the right tools is more complex than it sounds; balancing data access with the security of sensitive information requires a mature governance setup. This is generally not covered by a single tool. For example, Microsoft offers Azure Synapse for data transformation and visualization and Microsoft Purview for governance and data cataloging. Using multiple tools, all with different UIs, can be confusing, hindering collaboration and adoption, and comes with further challenges in other areas, such as access management and cost monitoring.

A recent study from Gartner (2023) on data analytics adoption shows that even though data is part of the roadmap of CIO offices, only 44% of data analytics leaders consider their teams effective and have a decent level of data maturity to deliver valuable information to the business. That number becomes even more interesting when compared to business investments in the data domains, as the average is now around 5 million per company per year. How can we ensure that the business users are provided with the right insights? How do we use our expensive data infrastructure to its full potential? How can we ensure a compliant but unlimited and intuitive flow of data within the company?

While there is no panacea for such complex questions, Microsoft recently took a shot at overcoming this complexity through Microsoft Fabric. The philosophy behind Fabric is to unify and extend the current Microsoft services used for all data-related work in a single tool. Fabric integrates extraction and transformation (Azure Synapse/Azure Data Factory, Databricks), visualization (Power BI), and Machine Learning (Azure Machine Learning) cloud products while simultaneously including data governance tools such as Microsoft Purview and IAM (Identity and Access Management). 

By unifying these tools into a single UI, business users have an easier time moving from being report consumers to self-service analysts. When it comes to Microsoft products, self-service analytics is further stimulated through Microsoft's Common Data Model, which reduces the complexity of very advanced data models and brings valuable data to the business faster. This Common Data Model is a standardized set of schemas and entities from different Microsoft products that represent business concepts and activities across the organizations, all wrapped up into tables that are easy to understand and query. Putting such data at the disposal of business users will entice them to explore options to create their own insights. Similar encouragement is given by introducing OneLake, an enterprise data lake that makes sharing data throughout your organization easier. 

From our experience, many firms with high data maturity are looking to implement self-service BI. Often, this is a shared objective from both the IT and business functions, as it tends to reduce the workload for IT while business can more quickly get the analytics that they need to run their operations and daily activities effectively. Still, even for companies that have invested in designing strong data models, the task of creating the right (Power BI) reports remains a challenge. When business users – especially the less technical ones – need additional calculations or complex visuals, companies tend to run into a shortage of report development capabilities, resulting in falling back on IT resources or missing targets.

Microsoft Fabric addresses these issues, as self-service analytics with Microsoft Fabric has been simplified by integrating Copilot into Power BI, offering help in creating insight from information. Another novelty in Fabric to increase the adoption of self-service analytics is the integration with Microsoft 365. For example, sharing (Power BI) visualizations in Microsoft Teams or data using Microsoft Excel can bridge the gap between data analytics and business users. 

A mature governance setup is crucial to manage the expected uptake in data sharing and collaboration securely. Creating such a setup sometimes seems elusive as it requires governance to be embedded in the organization and culture. Although this cannot be solved through tool choice, Microsoft Fabric does help by bringing governance closer to storage. It integrates Microsoft Purview options, such as data cataloging, lineage exploration, and sensitivity labeling, into the data storage. Access management for cloud resources and data has been united, too, simplifying overall access management.

While these are some nice features of Microsoft Fabric, they naturally have limitations. Whether it's the right tool for your organization depends on your cloud and data strategy: it requires some level of centralization of your data infrastructure. Also, while deciding about implementing Fabric, it is essential to ask yourself what challenges it addresses and how solving these ties into your strategy. 

To fully use Fabric's governance options, it is crucial to have an organizational setup with mature data management capabilities. Clarity on who guards data labeling discipline throughout the organization helps Fabric's data cataloging shine. Similarly, an organization needs to manage data access to unlock the full potential of OneLake’s data-sharing options without introducing vulnerabilities to data leaks. These are just two examples of how a mature governance setup is a prerequisite for the best use of Fabric.

Fabric is not generally available yet; it is still in public preview. This means that some features from cloud resources such as Synapse and Purview have yet to be available. At the same time, some other features are updated or not being migrated, such as the update from Synapse Dataflows to the Power Query-based Dataflow Gen 2. Because of these changes, implementing Microsoft Fabric in your cloud infrastructure might take some migration effort.

In conclusion, Fabric can provide many benefits, such as easier self-service BI and collaboration between teams through unified UIs, tighter integration between many cloud resources, and increased focus on data governance by embedding Purview in the data transformation and storage resources. Fabric can provide these benefits to larger organizations with some level of centralization in their IT setup. To profit from Fabric, some preparatory work on fitting it into your infrastructure and governance is often necessary. Too many organizations fail to implement a self-service BI setup due to a lack of strategic alignment. Luckily, as Fabric is still in public preview, you have time to anticipate its implementation, but you better get started soon!

Tom Janmaat, Expert, Data Engineering, and Daniel Sbai, Expert, Business Intelligence, contributed to this article.