La plateforme de Data Intelligence, le 3e pilier du Data Mesh

07 November 2023

The Hardware Approach of Data Mesh

Since the Data Mesh approach aims to give – or even return – control over their own data to the business units, it is necessary to provide it to them in a simple manner. This is the role of a data platformFederated and interoperable, it organizes the self-service of data and data products created by the business units (the domains according to the Data Mesh terminology).

Within the framework of Data Mesh, the platform goes beyond self-service data. It also provides the necessary resources for product design: application services, provisioning, hardware resources, etc. It is a true infrastructure.

The principle of a data platform is based on storing large volumes of data, with the aim of exploiting specific use cases, such as providing a near real-time response to demand or multiplying interactions between various types of data (consider user profiles rich in binary data and non-binary data). More broadly, a data platform addresses all the processes necessary for implementing projects such as: data qualityartificial intelligence or machine learningData science and other AI.

While the Cloud and virtualization offer the expected flexibility in data engineering, it should be noted that a data intelligence platform must remain agnostic. Depending on constraints, security requirements, and prior organization, the organization can perfectly adopt a hybrid operation, prefer a private Cloud, or rely on its Datalake if it already has one. The essential thing is to maintain control over the allocation of hardware resources and to preserve the agility required to industrialize all uses in the long term.

Data Intelligence

Based on a datalake initially implemented for the needs of the POST Luxembourg group, whose activities cover an extremely wide range of services, this data intelligence platform was first designed and conceived to ensure that all data is held, managed, and used in a rigorous and coherent manner to meet the needs of both the group's internal and external clients.

Now distributed by DEEP, this dataplatform is composed of several technological stacks, On premise or in the Cloud, which allow the ingestion of data, regardless of its origin, with different modes of articulation: ingestion, transformation, storage, historization, data discovery (data mining and machine learning) in particular. It is thus intended for all large organizations that have undertaken an initial big data approach or have detected specific needs related to significant data flows, such as the massive exploitation of IoT.

Data Intelligence manages data segregation, in order to operate the differential load to improve system latency times. Its multisite self-replication solutions ensure service continuity.

Just like the Logical data fabric, the Dataplatform is a modular (physical) framework, which is composed brick by brick according to its application needs and the development of data into information: real-time, archiving, data qualityAI, reporting, etc.

data platform does not replace the existing one. It consolidates and secures the numerous data management sources that a large organization generally holds.

A Concrete Example of Data Intelligence Possibilities

A use case that perfectly illustrates the possibilities of the Data Intelligence platform, and particularly its automated machine learning module, is currently underway on a European scale.

Data for road safety is a large-scale project aimed at securing European roads, based on the convergence and real-time analysis of multiple data sources from the POST Luxembourg platform. The contributors are numerous, including automotive manufacturers and equipment suppliers, traffic reports, GPS data, public data from authorities and European states, etc. This collaboration between the public and private sectors has, for example, helped reduce the detection times of critical safety situations by an average of 12 minutes in the Netherlands.

Support, Security, and Data Sovereignty

Available in the Cloud and On premise, POST Data Intelligence is a Data as a Service platform. Clients then have two options: develop their own products internally or entrust the task to the analytical department of POST Luxembourg.

Data enrichment can also be carried out by POST Luxembourg experts for the integration or provision of external data such as Open data and their exposure. Finally, dashboarding and visualization tools are directly implemented (as à la carte modules) on the datalake. In its Cloud version, POST Data Intelligence is hosted in a sovereign Cloud.

This is the third pillar of the Data Mesh. So, what remains to be implemented? Well, data governance and data productsFederated, of course, to not lose any advantage of the mesh enabled by the technology. It is both a management and a strategy. It orchestrates all interactions with quality and fluidity. Federated governance is the subject of the fourth and final article in our journey around the Data Mesh.

Continuing our Data Mesh journey. After the concept and its application within the Logical Data Fabric, it is time to address the physical tool, the engine of the Data Mesh: the self-service data infrastructure. The third pillar of the data mesh process, the data platform ensures the consolidation of data through its storage and ultimately enables the implementation of its governance.

Other articles in the category Data & AI

Federated Governance: A Key Pillar for Successful Data Mesh Implementation

Learn why federated governance is a critical organizational pillar in a Data Mesh architecture. A strategic issue for data-driven companies.

Read this article

Published on

12 December 2023

Top 10 Databases of 2020: Popularity Ranking

Explore the ranking of the top 10 most popular databases in 2020 according to DB-Engines, including Oracle, MySQL, and Microsoft SQL Server.

Read this article

Published on

14 November 2023

Our experts answer your questions

Do you have any questions about an article? Do you need help solving your IT issues?

Do you have any other questions? 

Call us free of charge on 8002 4000 or +352 2424 4000 for international calls, from Monday to Friday, 8 am to 6 pm.

About DEEP

Discover DEEP, your unique partner for your digital transformation.