How To Do Data Cleansing In Existing Legacy Systems For Your New Digital Platform

Digital Platform Part 5 Of 10

In previous articles, we have defined our goals, system architecture, data model and we have mapped out how to get access to the data we require. Now we should consider the data cleansing process within the legacy systems.

What Is Data Cleaning, Its Importance and Benefits?

Typically the data in legacy systems can be of poor quality and be inconsistent between systems. For example, a customer’s name may be Smith in one system and Jones in another. It is also possible that one system uses a number to uniquely identify each customer and another customer uses a completely different alphanumeric code.

The process of normalising and correcting data across your systems is known as cleansing.

Scoping And Cleansing

In article 1 of this series, we discussed the need to scope each iteration of your Digital Platform. This scoping will allow us to limit the amount of time-consuming and expensive data cleansing that we will need to undertake at any one time.

Although you should automate your data cleansing as much as possible, cleansing is frequently a semi-accurate manual task requiring time and money. As such you may choose to not cleanse some data and trust that your new Digital Platform (specifically the Data Broker component described in article 2) will work out the “most correct” data, when it is requested by the client Apps for your Digital Platform.

Ideally, one undergo the data cleansing process before incorporating it into a Digital Platform, but sometimes it is just too uneconomic to do so!

data cleansing process

Other Benefits Of The Data Cleansing Process

There are other benefits to cleansing your data. Especially in global organisations, cleansing for a Digital Platform is a great opportunity for agreeing on common coding and identification practices for key information in your organisation. This could be agreeing a global system for labelling products, identifying customers or coding projects. In itself, this will improve cross-border understanding and co-operation within your organisation.

Next Steps

Now that we have defined our goals, system architecture, data model, mapped out how to get access to the data we require and also learnt how to do data cleansing for this iteration of our Digital Platform, we are ready to define the API. External systems such as mobile Apps, web Apps etc will use the API to access our Digital Platform. For more information on the any of these subjects, please feel free to contact us.  

Nick McKenna
Since 2004, Nick McKenna, BSc, MBCS Biography has been the CEO of McKenna Consultants. McKenna Consultants is a bespoke software development based in North Yorkshire, specialising in Cloud development, mobile App development, progressive web App development, systems integration and the Internet of Things development. Nick also holds a First Class Degree in Computer Science (BSc) and wrote his first computer program at the age of nine, on a BBC Micro Model B computer. For the last 21 years, Nick has been a professional computer programmer and software architecture. Nick’s technical expertise includes; Net Core, C#, Microsoft Azure, Asp.Net, RESTful web services, eProcurement, Swift, iOS mobile development, Java, Android mobile development, C++, Internet Of Things and more. In addition, Nick is experienced in Agile coaching, training and consultancy, applying modern Agile management techniques to marketing and running McKenna Consultants, as well as the development of software for clients. Nick is a Certified Enterprise Coach (Scrum Alliance), SAFe Program Consultant (SAI), Certified LeSS Practitioner (LeSS) and Certified Scrum@Scale Practitioner. Outside the office, Nick is a professional scuba diver and he holds the rank of Black Belt 5th Dan in Karate.