This will be a slight departure from my usual posts about random bits of code and have more a focus on architecture.
In the last few years the world of development has seen quite a bit of upheaval as it adapts to new ways of working, the devops movement has, when implemented well, proven incredibly effective in increasing ‘throughput’ of code as well as offering opportunities for team empowerment and increased stability of the final product. Unfortunately, that ‘when implemented well’ is sometimes a sticking point. The idea behind devops is too integrate development and operations, to make operations (ie: deployments, infrastructure) more code driven and automated. What happens when it’s implemented badly though is that a separate ‘devops’ team is setup within an organisation, the majority of the members being sysadmins with maybe one guy who knows how to write bash scripts. I’ll cover devops in more depth in another post, but what I want to focus on in this post is the move towards software defined architectures, and how that movement can be harnessed effectively.
Software defined architecture
So what do I mean when I talk about software defined architecture? Well, traditionally applications have been clearly distinct from the archiecture that ran them. Organisationally you’d have a team of developers writing code, which would then be packaged up and deployed to the architecture, ie: the web servers, database servers etc that made up your infrastructure. The two teams would have some interaction when it came to setting up new database servers etc, since the code would have to take account of those new servers but, in general, they were completely separate functions.
Then came virtualisation. Initially the move from ‘big iron’ servers running an operating system with application servers on top to virtualised servers didn’t have much impact on developers since that separation between developers and operations was still present. However it soon became clear that once your code was running on virtualised servers, in effect, the server itself was a code entity. The introduction of vagrant (which opened up the building of virtual machines to developers) and finally docker (which in essence removed the ‘server’ from the equation all together) gave more and more options to developers when it came to deploying their code. Which brings us to the ‘software defined architecture’ I mentioned earlier, an architecture that is provisioned by code, maintained by code and is, in all respects, just another piece of code for your application.
New technology, old problem
The problem now is that, although the infrastructure is, as we’ve seen, ideally suited to being maintained and provisioned by developers, few organisations are taking advantage of that. I’ve seen organisations ‘move to the cloud’ by taking their current infrastructure, maybe adding some auto scaling if they’re feeling adventurous, and then deploying that to AWS or google and giving themselves a pat on the back for embracing the devops culture. That’s not ‘doing devops’, doing devops properly requires a complete paradigm shift (which I’ll cover in a later post) but it also requires a fundamental re-architecting of your applications to take advantage of the new stack. What I’m going to set out here is a high level overview of what your architecture may look like once you fully embrace this new way of working.
Now that we’ve seen how infrastructure has become code, it’s time to start applying some of the hard won lessons that software development has learned in the past decades, once of the most important of which is the ‘MVC‘ pattern. MVC is a pattern for breaking down your application into logically separated layers, each with their own area of responsibility, namely the ‘Model’, “View’ and ‘Controller’ layers. These layers can also be applied to our applications infrastructure, with the same benefits that come from applying it to your application.
The Model layer
In an application, the model layer is responsible for, well, modelling your data. It’s usually implemented as a database abstraction system, allowing mapping between the database and models within your application.
In an architecture sense, the ‘Model’ layer handles persistent storage for your application. It contains database servers, session storage, filesystems etc. The main point to note about this layer, as mentioned above, is that it is ‘persistent’. Any changes to your application, any deployments etc should have no effect on the data stored within this layer. This is crucial as we’ll see when we look at….
The View Layer
In an application, the view layer contains templates and other logic solely concerned with presenting your application to the end user. The UI basically lives entirely in this layer, and you should be able to change the entire presentation of the application without affecting its data or logic by making changes to this layer.
In our architecture the View layer is implemented with stateless, volatile servers that contain your application code. Your single page javascript application will live here, your PHP backend system etc. The one important fact about this layer is that anything living here can be thrown away / redeployed without any effect on the rest of the system. Immutable servers are a good thing to consider for this layer, you should never worry about upgrading the code that lives here, you should always be thinking in terms of ‘throw it away, build it again’.
The Controller Layer
Finally we come to the ‘controller’, which in an application would contain the majority of your code. The systems logic to get data from a model and push it to a view would live here for example.
For our architecture, the controller layer has a slightly different task, it’s the interface between the ‘real’ hardware and your application. For example, the controller layer might contain a dns system that resolves api endpoints to the docker container that can service those endpoints. Since the docker container lives in the ‘view’ layer, it has no persistence and doesn’t always have the same IP so some form of service discovery is more than likely required here.
Hopefully that gives some insight into how we can apply software development paradigms and methodologies to software defined architecture, in part 2 I’ll look more deeply at each ‘Layer’ and what it might look like in a real world example.
Some of the ideas in this article were inspired by this piece from thoughtworks