Legacy System Migration: Moving to Modern Technology Without Pain
A lot of our clients have developed an extensive mesh of closely interdependent systems over the years. Typically these systems are based on old technologies that are no longer common. This makes recruitment for developers hard and a lack of vendor support for retired systems means that they have become inherently insecure. The web of interdependence between these systems means that it is hard to change any system without having unexpected (and sometimes disastrous) effects on the other, seemingly unrelated systems. Release cycles are long and real innovation is frequently impossible in these situations. System performance is often poor and maintenance costs are high. The difficulty and problems encountered in altering these systems means that they can no longer be kept up to date at the same speed at which the business wishes to innovate and change.
On the other hand, a great deal of investment has gone into these systems over the years and they often contain complex, poorly understood and critical business login; without which the operations of the business cannot run reliably. This means that there is a need for transforming legacy systems from the 1990s into modern, flexible, easy to update applications, with a minimum of interference in the ongoing business operations. Often the solution to this problem is seen as spending several years developing completely new systems with a complex data migration, with one “big bang” at the end, when old systems are turned off and new systems are turned on. This method of legacy system transformation usually goes very badly, with most of these projects being abandoned, as they run massively over budget and schedule. Simply put, the task of replacing all of the complex systems in one go is too hard to achieve.
The Digital Edge Platform
At McKenna Consultants we have developed completely open tools and techniques , to simplify the process of transforming legacy systems into the leaner more agile software of the future. We call this the “Digital Edge Platform”. The Digital Edge Platform is not actually a platform, we do not think that buying into a single, proprietary platform is generally a good solution for businesses. It simply replaces one aging monolith with another. Rather the Digital Edge Platform is a highly flexible set of tools and techniques made by providers such as Microsoft, Google, Apple etc that can all be used in an open systems approach, to make your legacy system transformation easier.
How Do You Successfully Transform Your Systems?
The key to a successful transformation is to find a way to slowly, piece by piece, replace your existing systems with newer systems, that closely match the business need and give you a competitive edge. We would typically recommend starting by identifying an existing system in need of replacement, or a new system that needs to logic or information that is only present in your legacy systems. In this article, it is proposed that you host your new system on a scalable, robust cloud platform such as Microsoft Azure or Amazon AWS, but it would be possible to implement this approach entirely in house.
The Enterprise Service Bus
The diagram above shows how a Service Bus can be used to organise asynchronous (or synchronous) communication between system components. In this example, three new business-focused services are exposed in the cloud platform. These could be services such as a Sales service, a Debt Management service, a Customer Information service etc. They are technology services defined in terms of the business that is using them rather than technology focused services that will never be valued or understood by business people. A “service” in this sense could be a RESTful web service that allows orders to be taken from web applications or mobile applications for example. There are other technical options available for service implementation, but this example suffices for illustration.
The services are accessed by web applications, mobile applications and whatever else needs access to them, all using modern, decoupled communication. Services can be re-used by a multitude of applications both internal and external to the business. Services are generally small and focused around satisfying a single business need. Keeping the services small makes them easier to change as the business changes, easier to test, lower in cost to maintain and carries many other benefits.
The services all communicate over the Enterprise Service Bus (in red in the diagram). The Enterprise Service Bus is critical to this model as it allows services and technology components to communicate with other services and components without having to be directly aware of them. For example, a Sales Service may process sales orders and save them into a database. On a successful order, the Sales Service could emit a “Sales Order Success” message on to the Enterprise Service Bus. This event can then be handled by one or more other services such as a Business Intelligence Service that generates reports on Sales and other aspects of the business. The Enterprise Service Bus provides a level of indirection between the Sales Service and the Business Intelligence Service so that they do not need to know about each other. If the Sales Service becomes unavailable due to a failure, the Business Intelligence Service carries on working. If the business decides to add a new Business Intelligence Service, the Sales Services does not need changing at all. The new Business Intelligence Service simply processes the same events from the Enterprise Service Bus as the old Business Intelligence service did!
Legacy Application Adapters
So far this is great for new applications, but how could you leverage the critical logic and data that remains locked up in your legacy applications? The solution is to create “Legacy Application Adapters” that communicate with new services using the Enterprise Service Bus. Every Legacy Application Adapter will be a specialist, unique piece of software that allows access to previously locked-in logic and data. The Legacy Application Adapter will receive messages on the Enterprise Service Bus, process them and send replies out on the Enterprise Service Bus. Thus new systems are built that integrate with the legacy software without being tightly bound up to the legacy software.
In the diagram above, it can be seen that one of the Legacy Applications has been removed and replaced with a Modern Web Application, but the Modern Web Application can still communicate with legacy systems for vital logic and data via the Enterprise Service Bus. The key to it all is the loose coupling between services and systems provided by the Enterprise Service Bus.
The Goal – Gradually Replace The Legacy Systems
Using the tools and techniques outlined above (which we call the Digital Edge Platform), the slow process of transforming legacy systems can be seen. Achieved by replacing every legacy system with a modern, new service whilst maintaining continual business availability and critical business functions.
As shown in the final diagram, eventually no legacy systems will remain.
The Advantages Of This Approach
The approach to legacy system transformation and the Services Oriented Architecture based around an Enterprise Service Bus presented here, has many advantages. Principally we are not throwing the baby out with the bath water! A large amount of money has been invested in the existing systems and we can leverage that for as long as needed, whilst building new systems. We do not require a huge one-off investment to get the legacy system migration going. All that is required is enough investment to build the first new service. Once that is successful, funding can be sought for the next small service, and then the next and so on. The piecemeal, business focused approach also means that development of new services can be prioritised according to business needs rather than technological concerns. The step-by-step process also means that we can release each service into production as soon as it is individually complete. This means that we generate a much more rapid return on investment for each new service that is constructed. The laser-like focus on each service also means that this is a very low risk approach. If something goes wrong with development of an individual service (which is less likely in this approach anyway), then the scope of the problem is limited to that service exclusively.
Having multiple, small services focused around business needs means that multiple teams can easily work on this platform without stepping on each other’s toes. Each team can focus on delivering real value by creating a high-priority business service with a minimum of cross-over into another team’s work. It also follows that with a minimum of effort, some of the legacy systems can be replaced with a more standard, off-the-shelf application. For example, a company could replace its own aging in-house CRM system with an off-the-shelf system such as Salesforce or Microsoft Dynamics. A simple adapter can be constructed to facilitate the integration with third party software, at a much lower cost than the roll-your-own approach.
Service resilience is much higher in the Digital Edge Platform as each service is completely self-contained and does not share resources with other services. For example, databases are not shared between services. This means that a system outage in one service does not impact the other services. They will continue with their normal service levels. The same separation of services also means that the services can be scaled separately. Should you be using a cloud platform (such as Microsoft Azure), it is simple to scale the resources of high usage services and reduce the resources of low usage services, thus reducing your costs and maximising the return on investment.
Small services are quicker and easier to change as the business rapidly changes in a modern, fast-paced environment. This means that your business will be more competitive as time goes on rather than being stuck in the legacy of the past. The Digital Edge Platform described here, is completely open and extensible and includes no proprietary frameworks and incurs no proprietary license fees! It is the platform without a platform. This means that you can endlessly add or remove components and services of any shape and size, to adjust to your changing business requirements, without ever being stuck with a single vendor.
The ability of the Digital Edge Platform to integrate both old and new technology means that progress will be made continually throughout the legacy system transformation process. New services can be created and deployed in days or weeks at regular intervals rather than in one big bang at the end. This agile continual delivery of value makes this approach unique and successful.
If you are interested in our legacy system migration service via a digital transformation, please fill in the form below for your free consultation. Or contact us for more information.