In an article earlier this year, I talked about the role of technical debt in cloud-native application complexity. However, there’s more to complexity than just technical debt.
In fact, the heart of complexity comes from a simple concept: Choice. Every choice added to a software system doesn’t just add one more pathway—it often adds multiple potential paths.
You see, the more choices you add to a system, the more complex that system becomes. Choices add options, options add alternatives and you lose clarity in the path users will take through your system. As soon as you give your users choices, you have multiple paths they can follow and the complexity of your system increases significantly.
If you want a great example of this phenomenon, look no further than the complexity of AWS. When AWS first started, it had a couple of basic services and nothing more. There was object storage (Amazon S3), server instances (Amazon EC2), a queue (Amazon SQS) and a basic database (Amazon SimpleDB). That was it. It was simple to understand and pretty simple to use. Of course, what you could do with AWS was more limited than it is today, but its offerings were easy to understand.
Compare that with AWS today. Now, there are over 200 distinct products and services with thousands of options and alternatives. Many of these services work well together, while others are independent and unrelated. Some even seem like competitors to each other (look at Amazon ElastiCache and Amazon MemoryDB).
AWS is no longer simple. In fact, to truly understand all of AWS’s offerings, you need a certified AWS expert who is trained by AWS to architect a system for you. Yet, with all this complexity, AWS can be used to solve almost any software application requirement.
Most cloud-native systems are somewhere between the simple, basic offerings of the early AWS and the huge complexity of the modern AWS. But as time goes on, most systems tend to increase the number of choices and options and hence increase their complexity. Uncontrolled complexity, like entropy, always increases.
Keeping control of your complexity and the growth of complexity in your cloud-native application is tough, but a good step in managing complexity is managing your application’s choices and options.
This can be quite challenging, especially in light of customers and their requests. Often, change requests from product management require adding additional options to a system. You want the ability to add a new feature without negatively impacting those customers who aren’t currently using that desired capability. One common approach is to add a capability, and rather than forcing everyone to move to the new capability, you give your customers a choice—stay with the old or move to the new—and this choice adds complexity.
In fact, in almost every case, individual enhancement requests always add complexity to your system. Inevitably, a request will add choices, and your system complexity spikes upwards.
Avoiding this complexity increase can’t be managed one feature at a time. Rather, you must take a holistic approach to improving your application in order to find methods of supporting new capabilities without simply adding additional new paths through the application. This holistic approach is needed to detect trends and patterns. These trends and patterns can be used to merge and simplify your codebase, reducing options without sacrificing customer capabilities. Ultimately, this leads to reduced complexity.
Sometimes, we don’t have the time to take a holistic approach to a feature request—sometimes, we need to do a quick and dirty implementation just to get something out to our customers. In fact, this is the very nature of the agile development process. However, if you don’t stop occasionally and look at the big picture of your application, your complexity will skyrocket.
During the agile development process, it’s important to take a step back occasionally, look at your application holistically and see if there are patterns and trends you can lean on to refactor your application to reduce your system complexity. It’s a critical aspect of all successful modern application development and absolutely essential to prevent your application from falling into the abyss of complexity that comes from one-off decisions.