Data Synchronization Strategies For Digital Transformation
Digital Platform Part 9 Of 10
Behind the scenes of your new Progressive Digital Platform, it is likely that you will be aggregating data from multiple legacy systems and issuing instructions across various systems. For example, retrieving complete information about a customer may require retrieving information from a CRM system, ERP system, web shop and others. All of this requires a strategy for bringing that information together and keeping it all up to date.
For the purposes of this article, we will focus on data retrieval, however, the same principles apply for updating data.
There are two primary data synchronization strategies: periodic synchronization of data and real-time switchboard.
A “real-time switchboard” system deals with requests for information (e.g. the customer information explained above) by providing a single point of contact that queries the sub-ordinate systems (CRM, ERP, etc) in real-time for each inbound request for information. This has the advantage of providing the latest, up to date information to the requester. This approach is only suitable where the systems being contacted have some kind of API access and are reasonably stable and quick to respond. In environments with a large number of legacy systems, this approach is unlikely to work well.
Where systems are unstable, slow to respond or lack a good API, a periodic synchronization of data may be a superior approach. In this approach at set intervals (once per hour / day etc) a synchronization Data Broker requests data about different data items (such as Orders) from all the different systems. The Data Broker collates all the results from the different systems and updates its own internal database containing a copy of the received data.
This means that users making requests do not have to wait for unreliable systems that may not respond (the Data Broker database will be cloud-based and very reliable). It also means that the data provided will always be out of date. If the synchronisation job runs once per day, then data could be 24 hours out of date.
In either approach Digital Adapters are typically built that front the legacy systems and provide modern access for the Data Broker. Typically these are required as the legacy systems typically do not play well with more modern systems!
In a more sophisticated platform a mix of both approaches may be suitable to accommodate the variance of reliability of the legacy systems that contain the data being request.
After learning about the various data synchronization strategies, the final stage in this series is to become familiar with access logging, legal issues and debugging.
For more information on any of the topics discussed in any of our digital platform blog series, please contact us.