Federated Governance: A Shared Responsibility
The concept of federation is an organization around data, capable of federating all of it. Because it comes from multiple sources, data must be federated by establishing an organization (governance) and systems (brick/framework) that enable or facilitate its implementation within the company. We discussed this in our previous article on the logical data fabric.
From an organizational perspective, federation involves fully implementing the data mesh logic by entrusting data owners with the responsibility for production on one hand, but more importantly, the responsibility for its quality on the other. Thus, business units have great freedom in handling their data and are given the fundamental mission of ensuring that the data is usable by everyone, for everything.
To best achieve this mission, it is evident that at a higher level, common governance rules must be established. This is the classic functioning of federal governance applied to the data mesh in a company. It provides the organization with both a functional and technical vision through the traceability of information, which has been lacking in the exploitation of the data heritage until now. It is also through this that we gain responsiveness and agility, as we respect the natural organization of the company while bringing a new perspective on data, which is no longer a by-product of IT but a product in its own right.
The fundamental principles of federated data governance
Two fundamental aspects allow for the implementation of federated data governance and data products: on one side, the technical organization prepares, models, and exposes the data and metadata, then offers a global vision of the ecosystem. On the other side, the human organization relies on the execution of a complete project that requires strong sponsorship from the business units, management, and the IT department, independent of the tools adopted.
The management of information requires a data-oriented organization within the structure to define the right processes. It involves asking what needs to be done within the company and who the audience will be around three themes:
- Knowledge: What are the business processes in place? From what consolidated knowledge of the company will we work? With what common language, shared throughout the company (glossary/semantics)?
- People: Who sponsors? With what organization? On what skills can we rely?
- Governance: This concerns the quality and consistency of the data, the data lineage (i.e., the repository of data flows and their metadata), security (data access policies), confidentiality, and more broadly, compliance. These are as many subjects as there are stakeholders and actors.
The technical solutions enabling the implementation of the data mesh do not go so far as to support companies in defining roles and management rules. This remains a process that will be refined as the company gains maturity and its data ecosystem evolves.
Federated Governance and Role Distribution
It becomes necessary to question and involve the business units to concretely determine the actors capable of participating in the definition of the common language, management rules, and processes (especially validation), in correlation with the tools in place. Several actors have a role to play, the essential thing being not to place all the responsibility for data quality on the shoulders of the IT department, at the risk of quickly hitting a wall.
It is a team effort, composed of those most interested in the question. A pool of data scientists, the role of the data security officer in the organization, access of data engineers to information, continuous involvement of data stewards with the data office, the pivot of the structure, everything remains to be determined.
First of all, it is important to define the data sets on which the quality improvement work will be carried out. After years of emphasizing the importance of qualitative data, companies still have difficulties knowing where to start. Experience shows that quality is not always there. While tools can intervene (and are of great help) to solve problems of duplicates or harmonization according to standards, they can do nothing about the final validation of the data. This is a mission that only humans can handle, starting with the implementation of data quality scoring to improve practices. Depending on the organizations, this task can be entrusted to the data steward (who has the business knowledge of the concerned domain), under the control of the data owner who will ensure its adequacy for the use case.
While it is relatively easy to implement the technical bricks allowing the establishment of a data mesh organization in the company, the creation of its federated governance requires the evaluation of its maturity level, mastery of its IT organization, knowledge of the needs of functional referents, as well as the analysis of data architecture and archiving solutions, among others.
Then, it is necessary to proceed perimeter by perimeter, if only to ensure sufficient data quality to produce valid models and then operational data products. The process will lead the company to question the relevance of its data stock, especially the oldest ones, kept at great cost (in more ways than one) without guarantee of exploitation. This may then be the first step towards its ability to balance the use of informational assets and environmental objectives, a major question indeed.
This article is part of a series dedicated to Data Mesh.
