Software Architecture Challenges

As I near completion of the latest version of the mobile app I have been working on recently, I can take the time to reflect on the architectural challenges that I faced, and how I conquered them.

The mobile app was developed for the fleet management sector and was a complete re-write to the existing offering. There were many moving parts to the latest version of the mobile app. The mobile app allows users to send requests from their mobile devices so that the data they have submitted can be processed by the back-end line-of-business application. The mobile app allows users to send data that corresponds to booking an MOT or service, updating their mileage or completing an inspection for example. So the challenge was to devise an architecture that would guarantee this data would arrive at its destination, and that the architecture was capable of scaling to meet future demand, and that it was highly responsive. You don’t want to be able to guarantee delivery of data if this becomes a time consuming process and gives the user the impression of a sluggish application. Conversely you don’t want a highly responsive application which then cannot guarantee delivery of data or the data arrives corrupted.

Not an easy challenge by any stretch of the imagination.

To make matters even more difficult, the back-end line-of-business application is a legacy VB.NET application build around an equally legacy version of SQL Server. So I had to factor in these considerations from the outset as they are critical to the overall architecture.

The first decision was what technology to use to implement the services that would be required? Although I have used WCF (Windows Communication Foundation) extensively previously, we needed a technology that was built around HTTP and could easily consume JSON payloads. We also needed to be able to consume the services from the mobile apps which were implemented using Apache Cordova and Javascript. So the decision was made to go with ASP.NET WebAPI. This would allow us to build up the necessary suite of services using HTTP as the transport protocol (the clients would be mobile apps where HTTP is ubiquitous) and be able to exchange information using JSON. We used JSON rather than XML as the client application was implemented using Apache Cordova and Javascript. Naturally there is a far closer fit with JSON than XML when it comes to data exchange with Javascript.

All services required by the mobile app would be implemented using ASP.NET WebAPI and all data would be exchanged using JSON.

The next decision was where to host the WebAPI services? It was suggested (by yours truly) that we should look into using Azure for our hosting. Although we already had hosting with another supplier, it was agreed that we would use Azure for hosting as we were already looking into other areas of the Azure development platform. Although it is not strictly necessary to host your services on Azure to reap the benefits and have access to the many other services it has to offer, it’s fair to say that they just work better if you do.

The infrastructure offered by Azure would be vastly superior to any we had in-house or with our other hosting supplier. I added a separate deployment for Azure to our TFS 2015 build process. After some initial configuration to allow the build process to access the Azure hosting environment, you are then good to go. This build process doesn’t automatically deploy to Azure, as this is our production environment. Instead, deployments to Azure are triggered on an ad-hoc basis when needed.

The next challenge was how to guarantee that data sent from the mobile app would be received by the back-end line-of-business application? The levels of resilience needed by the app would require a service bus architecture. All messages sent from the mobile app would be added to an Azure Service Bus queue, where they could be subsequently picked up and processed. A service bus architecture has many advantages over traditional service delivery.

- Far higher degree of resilience
- The disconnected nature of a service bus means that you are not waiting for a response from the server (fire-and-forget)
- Able to process far higher loads
- Able to scale massively if neccesary
- You pay for what you use
- Azure Service Bus has excellent integration with the .NET ecosystem so can leverage it’s services from a .NET application with ease

Plus many more.

So I implemented a WebAPI service that was capable of adding messages to the Azure Service Bus. Each time data was submitted from a mobile app it would invoke this service.

I next needed to decide how I would retrieve the messages that were placed on the Azure Service Bus. Although it is perfectly possible to write an application that can listen to the Azure Service Bus for incoming messages, it seemed a far better idea to make use of an Azure Function and bind it to the Azure Service Bus. Each time a message was added to the queue it would invoke the Azure Function. By implementing the listener application using an Azure Function it reduces the burden on our local infrastructure and guarantees to be available at all times.

The next big challenge was how to ensure the data received from the mobile app was in a meaningful state and could be processed by the back-end line-of-business application. All data sent from the mobile app contained only a fraction of the data needed for it to be processed by the back-end line-of-business application. It became necessary therefore to supplement the data for it to be of any use to the back-end line-of-business application.

This required the addition of a separate service that would take the bare-bones incoming data from the mobile app and supplement it with further data before writing it into the back-end line-of-business application database. The development of such a large, enterprise architecture was far from straight forward and had more than its fair share of challenges. But each challenge was met with a steely determination to which a perfect solution was found and developed. It is not easy trying to mentally unpack and unpick such a large, unwieldy and difficult set of circumstances and problems. Many times I had a take a step back or step away to give them due consideration. Architecture is a difficult enterprise, made even harder with so many moving elements and difficult challenges.

This project was certainly one of the most enjoyable I have worked on for a long time. It’s given me great exposure to the Azure platform from a development perspective. It provided great exposure to service bus architecture and in particular Azure Service Bus. Getting to work on such a variety of problems, shiny technologies and architectural patterns was great fun and I enjoyed every minute of it.