Requiring teams to send their data to a central system administered by a single Data Management team is costly, time-consuming, and transfers data ownership away from domain specialists. Data Mesg gives the data domain ownership back to the business units, reducing the path between data sources and users.
Data is the lifeblood of any enterprise. And like blood, to be useful, it must flow through your business, not sit in a spreadsheet or database. It must be FAIR, that is:
- Findable - people who need the data must be able to locate it. Otherwise, vital information may be omitted from the decision flows, or, equally bad, multiple conflicting copies may circulate.
- Accessible - it"s not enough to just have data, it must be made available where it"s needed. Having the only copy locked in the boss"s desk may be secure, but is not very useful.
- Interoperable - data must be provided in formats and through interfaces useful to the users. (Just ask anyone who ever received a screenshot of a spreadsheet in an email.)
- Reusable - design your data flows with the whole operation in mind and find commonalities. The more you align your data spaces, the more coherent the information flows will be.
Can this be achieved through technology alone? Unfortunately, no. New systems can only take you part of the way. What you need is a socio-technical shift. You need Data Mesh.
What is Data Mesh?
Data Mesh is data ownership and governance decentralization paradigm designed to increase the business value extracted from the data within the business. Since the coining of the term in 2019, the methodology has been revolutionizing and disrupting the way companies deal with data.
Unlike centralized frameworks, Data Mesh does not require one team to take responsibility for all data within the firm. Within this framework, data is treated as a product, with responsibility for this product placed closest to its origin within the company. The governance framework is federated, bringing together representatives from among data producers and data users to strive towards a common goal of making data work for the future of the enterprise.
The key to achieving Data Mesh objectives is realizing that it's not just the systems that need to be updated. In practice, the systems often don't need to change that much. It's the mindset of the teams and management that needs to shift, to move from thinking about data as a by-product of the business to making data the critical ingredient in decision making.
What is wrong with the way data is currently handled?
Central governance creates bottlenecks
Tight coupling leads to extensive changes and testing downstream.
By closing the time and space gap between data collection and its use in decision making, the system is broken down into much simpler segments, which can operate independently, yet coherently. This paradigm responds well to scaling up and reduces the costs and complexity of keeping the business aligned with the markets and future objectives.
Conforming to data regulations is resource intensivene
Compliance with these regulations in central data management can be problematic and costly and may lead to sub-optimal solutions and delays. With Data Mesh, all data is governed within its rightful domain in alignment with a federated framework, lowering the cost of compliance and reducing risks.
Is Data Mesh for you?
Data Mesh is not a silver bullet.
Implemented unnecessarily or poorly may not meet the expectations.
How can we help?
Our Chief Data Science Officer is the co-author of the first practical guide to implementing Data Mesh, Data Mesh in Action published by Manning Publications.
Rather than just offering you the book, we can provide you with the expertise and resources to guide you in every step of your "Data Mesh journey". From defining the data domains, optimizing the flows, and implementing governance, we will work with your teams and your management to ensure your business uncovers and realizes the full benefits of your data.