Applanders

Based on my observations from the field, I regret to inform you that many, if not all, of your teams are woefully stuck in Appland.
Simply put, your teams are building applications when they should be building systems. But don’t blame them, it‘s not their fault. They’re simply proceeding as directed. It is the very latest in industry fashion to think in terms of apps and app technologies. In response, vendors of course all clamor to sell you and your teams this vision. A clean, simple story. You just need a website, an API and a mobile frontend. Keep applying the same app techniques across design, technology, coding and process that worked for you in the past; techniques you and your teams are comfortable with. Everyone of course is still secretly pining for that simple vertical assemblage of screens and a database. Click, click double-click. Done! Oh, those were the days.
Maintaining this app-centric mode of thinking would be just fine if it weren’t for the insane industry we’re in and all your capability crazed clients. This industry of ours, well you know this industry; it’s a tireless juggernaut beast ceaselessly churning, inventing alternate realities under the cover of darkness. And those pesky clients of yours, they’re certainly a fickle bunch. Unfortunately for you, even though they exclaim otherwise they’re in no way software experts. If you’re lucky, they may have an inkling of what they want, but they definitely do not know at all what they need. Providing them what they need is, as architect, the essence of your job.

Great Expectations

Against this backdrop of insane industry churn and crazed client expectations a point of no return is occurring. You and your teams are experiencing nothing short of a major industry inflection point that is driving expectations and capabilities formally reserved for the largest of systems into the laps of the smallest of teams. Consumer, commercial and even enterprise expectations of software capabilities are exploding in all directions.
At any moment, you and your team should expect to be hit with any or all of the following:

This list of course is by no means all-inclusive. Your market, your business leaders and the regulatory committees that haunt your dreams will not conveniently telegraph these demands to you in advance. Quite the contrary. These demands will sneak up on you in the night and hit you upside the head with a 2×4.
What this tells you is that distributed software evolution is inevitable. Quite against your will your apps will all conspire to become systems. All modern software, even the smallest of mobile apps, is actually a system.

Problem is…

Even though the industry inflection point is bombarding you and your teams with new demands for which existing techniques have no appropriate answer, due to prevailing industry and vendor sentiment your teams are often thinking, “Yeah? So what? I don’t need all this crap. We have our website and our web API and that’s enough!”
Yes, sadly it is true. This is an actual quote. I’ve since heard many teams resoundingly echo the same. With bravado even. Unfortunately, this is an egregious oversimplification of the new problems at hand. Even so, your teams will try to do it. They will try to stuff a system into an app.
By Occam’s Razor, when you mistake simplicity for correctness you end up with a solution that is no longer accurate. This is what happens when your teams apply application techniques to system problems. The solution becomes over-wrung. There is a point when KISS (keep it simple stupid) is no longer appropriate. In any problem space, there is an inherent level of complexity. Trying to squeeze too much complexity out leaves you with a solution that is no longer accurate or even viable. There will come a time when your requirements shift so significantly that they then delineate a new problem space. This new problem space brings with it a new, inherent level of complexity. It is a mistake to try to get rid of it. You cannot. My second corollary of computing states, “In any adequately designed system, complexity is neither created nor destroyed, it just gets moved around.”
Apps are but one piece of the puzzle. They’re not even the central part. In the Wild, Wild West that is the Multiplicity of Clients with its endless churn and continual stack attrition it is wise to treat your apps as commodities. It’s the classic iceberg analogy. 90% of your software assets should be below the water line of your apps. This is how you carry the greater majority of your asset value forward in time. This is where you should invest.

Stonehenge No Longer Works

A great example of how the industry at large is waking to a new level of system awareness is the long anticipated emergence of the microservice movement. While the concepts are certainly not new, the driving need for this new breed of service-oriented architecture is very real; monoliths no longer work. To deliver on modern market expectations you must make your software malleable. Provide it expansion joints. Allow it to reside in multiple locations. Enable its morphing into previously unforeseen hybrids. And most of all promote composition and reuse at multiple levels of granularity. All seamlessly.
Just because you have a website, an API or a mobile app does not mean you are immune. They too can and often do fall victim to monolithology. It’s still a fat website. It’s still a fat API. It’s still a fat mobile app. Stuffing your dirty laundry into your controllers does not a system make. Neither does bolting your app on top of a fat service or for that matter a fat database. Unrepentant coupling abounds, making your resulting solution very brittle and not in the least bit malleable. Every little change reverberates across the entire solution. Enlightened architects are now realizing they cannot deliver Modern Software Architecture using app design techniques alone.

Still Stuck

Even with these new insights, I find many system inspired recommendations in the industry today still cling to outmoded app techniques and technologies. This is particularly true when it comes to interaction, integration and decomposition. Many sense and see there’s a problem. But they’re unsure what it is, why it is happening or how to fix it. In our workforce currently, there is little expertise in the building of systems. Couple this with the realization that there’s a palpable global talent drought and you may come to understand as I have that we have a true crisis on our hands.
As a practicing architect this should be one of your chief concerns, where will you find the appropriate talent to build what your market demands? Of course, the answer is simple. You won’t find them. You must instead grow them. But grow them well.

What the Big Boys Do

As a start to helping your teams form a more system-centric mindset, I suggest seeking insight beyond industry fashion and vendor product vision. Instead, study as I have those who really know how to build modern software systems; The Cloud Builders. These guys and gals know systems. They are the true masters of service orientation.
Of course, I’m not suggesting you need to build your own Cloud. What I am suggesting is that you study those who are shaping our industry with their bare hands, steal their techniques and most of all, their rationale and apply it to the new distributed system challenges your teams now face. Well, perhaps ‘steal’ is too strong a word, particularly when this knowledge is out in the open for all to see. You’ve unfortunately never heard much of their wisdom though, because their story is one contrary to the vendor’s vison of selling you products and a click, click, double-click, done monolithology.
It turns out; the experts are building systems in very similar ways. Led by the problems they must solve, they have gravitated to consensus. The Cloud Builders for example use very similar architectures and techniques. Regardless of platform. The same goes for all the brand name websites you know and love to use. There’s a notion of beyond the firewall vs. behind the firewall. And they gear everything to go beyond the website. Essentially this means, it’s all API (read: raw HTTP) beyond the firewall to maximize reach and opinionated sockets, pipes and queues behind the firewall to maximize efficiency, throughput, robustness and elasticity. And in the Cloud, these techniques also reduce cost.
These master builders realized you cannot build modern systems simply with a single transport particularly when that transport is not very efficient, robust nor reliable. They also long ago realized the value of microservices.
In this milieu, app techniques are only one relatively small part of the whole story. Why opt to maximize for public interoperability when you’re behind your own firewall? You are robbing yourself of powerful alternatives, efficiencies and a simplified programming model for your Devs. The browser interoperability stack is a perfect match for those who code beyond the firewall, but it is a complete mismatch for those who work behind it.
In the end, it all starts to look something like Figure 1 below. You might call this the contemporary building codes of Modern Software Architecture. Black dashed arrows are queue-based interactions. Gray dashed arrows are asynchronous, but non-queued interactions. Solid arrows are synchronous interactions. Red boxes represent deployment boundaries.
Of note is that the diagram suggests a microservice taxonomy that extends the microservice metaphor beyond the system level. In the abstract, the taxonomy defines subsystem microservices (S1-Sn) and one or more component microservice types (C1-Cn), each playing a different, but consistent role of encapsulation within the taxonomy. Each microservice represents a unique service-oriented isolation boundary through which you can apply service specific aspects, as well as, enforce service specific policies. Applying a microservice taxonomy can help you solve one of the most vexing riddles of microservice-based systems, deriving a consistent and appropriate service granularity for your system.

Many of the Cloud Builders have devised their own service-oriented taxonomies opinionated for their environments. Another example of a well-conceived microservice taxonomy broadly applicable to business systems is IDesign’s system design and construction methodology; The IDesign Method.
To remedy the significant problems associated with the proliferation of endpoints in microservice-based systems, component microservice endpoints are private to their subsystem. Subsystems only ever consume component microservices internally using efficient techniques that require no configuration. It is also important to recognize that applying a microservice taxonomy not only helps you more clearly convey the design of your system to your teams, it also informs your teams on how you expect them to construct the system as well. Subsequent diagrams detailing microservice relationships expressed using taxonomy rules become the blueprints of system construction.
In no way should you misconstrue the central queuing infrastructure in Figure 1 with that dirty word, ESB. This is all lightweight, modern queueing technology with only the necessary messaging patterns baked in. The emergence of lightweight connectivity infrastructure is also signaling the death of the monolithic connectivity server as we have known it all these many years.
With the lightweight technologies currently available, you could also do something like Figure 2, what you might call a microsystem. This means you host your entire connectivity infrastructure within a single deployment boundary or even within a single process boundary. While this definitely opens up a number of interesting options, right now at least I wouldn’t do this to my poor DevOps guys. They would clearly expire trying to comply with such a granular level of attack hardening. This is clearly one of the challenges of the burgeoning IoT.

Trend-Worthy

As with the trend toward lightweight, small footprint hosting, not all trends in the industry are questionable. Some are commendable. One technique of The Cloud Builders is making its way into nearly every connectivity stack in use today; support for multiple interaction modes. An interaction mode is a connectivity design reduction that goes well beyond the transport to normalize binding, behavior, configuration, extensibility and policy enforcement into a consistent configuration-free programming model for each type of interaction within your system.
Most stacks started out supporting only HTTP variants. Now, there is a confluence of thought across all stacks. Nearly all them support some flavor of message queuing. Many also support classic TCP sockets. Some support WebSockets. And an enlightened few also provide some form of intra-process (InProc) communication mechanism. If designed correctly, these stacks already do some level of normalizing their connectivity offerings, which greatly aides you in formalizing your own interaction modes. It’s important to note that while these additional interaction modes are not based on HTTP, they are not RPC either. RPC is a service-oriented design problem; it is no longer one of technology. Circa 2015 no one should be doing chatty contracts anymore. No one.
One interaction mode of particular interest is the WebSocket. Its emergence signals the end of the Internet as you know it. The rationale behind its existence, as well as, its popularity is centered firmly in a system-centric view of the universe. You must be able to push unsolicited from the system to the app. The system drives the app. Not the other way around.
As a logical evolution of this need, AMQP (Advanced Message Queueing Protocol) is quickly becoming the new standard for interoperability. A queue-based standard for public interoperability allows you to promote reliability and elasticity between disparate systems. It also supports broad reach to the multiplicity of clients. This means you can now provide a queue-based API beyond the firewall that is very compatible, if not identical, to the system composition techniques you apply behind the firewall as shown in Figure 3:

It is no accident that modern connectivity is moving in this direction. It is the more accurate solution to the construction of Modern Software Architecture. It is also the shape of things to come.
Perhaps the future holds something more like Figure 4:

The Fall of Stonehenge

In response to all the great expectations that the industry inflection point has suddenly thrust upon them, business leaders are demanding new levels of agility from their software assets. In practical terms, this new level of software asset agility takes shape in the form of system flexibility. Flexibility in turn infers composability, reusability, extensibility and elasticity. All of which requires a certain modularity in design. A desire for modularity means the end of monolithology. And somewhat counterintuitively, agility also infers maintainability. You cannot be agile by efficiently designing, building and deploying new things if the old things are a pain to maintain.
To rid your shop of monolithology, you must guide your teams and educate your execs to recognize that systems are different, fundamentally. They come together altogether differently than apps. Systems are all about integration, many levels of integration. This shift affects everything. From the way you approach design to the technologies you select to how your Devs code and test and most of all the process by which you ensure successful delivery. You must maintain a system-centric mindset with each.
There shall also be no ivory tower or astronaut architects. You must be technically capable of validating that your design can even be built. You do this by creating initial models of what you’re proposing in sufficient detail to convey in concrete terms to your teams the technical mechanisms behind your design. As I recommend below, you must roll up your sleeves and get dirty. What dirties your hands as architect though is different than your teams. You create the consistent foundational canvas of connectivity infrastructure upon which you teams can unleash their service-oriented creativity.

System Design

To deliver useful system designs for your teams, you need to apply a design approach that helps you arrive at an appropriate and cohesive system decomposition. This has little to do with screens, pages or databases. And I find partitioning by domain or model alone is not enough. When applied to service-oriented systems, these application-centric techniques often recreate new monoliths or produce rampant object-as-service proliferation. Both result in high degrees of coupling or lack the necessary cohesion that is the hallmark of good design.
In practice, these missteps mean that when your teams go to change the system they have to change more pieces than should be necessary. Routine evolutionary changes to the system begin to broaden in ever increasing scope. You can see this most often in the dreaded cascading accumulation of sideways calls between modules, subsystems or bounded-contexts. As service-oriented systems decay, their designs lose their structure as teams lose their discipline. The net result is a system with random interactions occurring at every layer in every which way; up, down, sideways, end-around and round-about. Just as OO gone bad gave us moldering balls of mud, diagramming the relationships of such a decaying service-oriented system would reveal that you have a painful ball of spikes in your hands. Any way you come at it you’re going to get hurt. If your systems develop this malady as they age, you have not insulated solution aspects that frequently change or change at different rates from each other. Nor have you provided a clear and encapsulated design approach for reuse.
As first suggested by Dr. David Parnas and later refined by Juval Löwy, preferably you would use a design methodology that does not ignore change, but instead attacks it head on. One that helps you better encapsulate frequent change in the system aspects or business requirements of your solution so that you localize the scope of change for your teams. The resulting design often looks quite different than a straight cut by domain alone. IDesign’s system design methodology The IDesign Method is the only design approach I know of that promotes and addresses directly the encapsulation of common areas of change within service-oriented systems.

System Technology

The demands placed on modern systems mandate elasticity. It is no longer enough to scale out. Your system must also shrink as well. The easiest way to address this need is to use a queuing technology. Queues normalize system load in a very reliable, predictable and efficient way. They absorb unexpected spikes and cyclical rises in load at lower cost. Queues are also the recognized building code for high throughput. And most importantly, queues provide the essential temporal decoupling you will need between your subsystem level microservices to extend technical empathy to DevOps.
The rise of numerous lightweight, mature queue-based messaging technologies clearly indicates the value and need for queuing. And it’s no mistake that all of the queuing technologies that matter now also support AMQP. All the modern software systems that you know and love to use employ some type of queuing technology as the backbone of their system architecture. And queuing is of course the cornerstone of the Cloud. This suggests that queuing, not HTTP should be the central interaction mode within the modern systems you design and build as well.
With this in mind and taking cues from both The Cloud Builders and the stack masters you will inevitably need multiple, efficient interaction modes. You can then match each interaction mode to where within your system its capabilities and its programming model are best suited. It is better to plan for this inevitability sooner than later. Prefer stacks that provide a clear path toward helping you to normalize your system’s connectivity programming model. Neither a plumber, nor a message wrangler be. Circa 2015, there are stacks that already do this for you. In addition, any stack worth its salt is built upon an extensible, interception-based pipeline. This pipeline often obviates the need for hexagonal-styled port and adapter patterns. Never do something that you can get the framework to do for you. You have many more important things to do.
Along these lines, enabling the automation of your system’s deployment and availability management is of course a top priority. Your designs must give love to DevOps in the form of clear diagramming that calls out directly the many isolation boundaries in play for each microservice in your system. You must clearly convey important aspects such as identities, security and process to groups outside of Dev. Though, you must beware of putting your cart before your horse. The industry has a serious preoccupation with deployment technologies and again the vendors want to sell you their vision here as well. Somehow in many conversations, deployment of microservices takes precedence over even having a solid system design in the first place. No amount of shiny deployment technology will remedy a poor design.
To ease your team’s adoption of new technologies and their understanding of the increased design constraints you plan to impose, consider doing a team pilot. Architects often chose as their first pilot to introduce queuing into their system landscape. To be real, your pilot project must have a trajectory toward production. To mitigate technology as a risk and promote absorption, adoption and advocacy across your teams select for your pilot a task ancillary to your business’s core processes. Teams often choose more granular tracing, higher levels of auditing or one or more BI insights like health monitoring, user telemetry or inflight ETL that supports more efficient analytics. And depending on your environment, introducing one or more hybrid Cloud platform capabilities is also an option.
One final note on system technology, avoid if you can allowing a heterogeneous technology landscape creep into your systems. Anyone who promotes multiple ways of doing the same thing has never owned a heterogeneous backend for more than one version. Of course you must always explore better ways of doing things, but you must introduce them into your architecture with discipline. It should be a discussion across the organization, not just one internal to Dev. Encouraging technical heterogeneity simply in the name of convenience or for lack of rigor only serves to reinforce silo boundaries, which are the exact antithesis of modularity. And heterogeneous back ends are a true pain for Ops.

System Coding

As architect, it is your job to make a Dev community of a broad spectrum of acumen productive. In all my years of building microservice-based systems (this is all I do), the only approach I have found that works and promotes successful delivery of modern distributed systems is to first normalize the system’s connectivity programming model. This means that as much as possible you must endeavor to make the programming model your Dev’s will use day in and day out the same for API services (aka raw HTTP), intranet services (aka socket-based), queue-based services, bus or broker backed services and in-process services. This is particularly important for the interaction modes you need behind your firewall, where 90% of your software assets will reside. And yes, to build Modern Software Architecture effectively, I believe you will eventually need all 5 interaction modes.
Your programming model must also enforce consistent policy across your system’s configuration, connectivity, behavioral and hosting aspects, as well as, all the necessary aspects of service orientation now in play such as isolation boundaries, message-based security, identity propagation, authentication, authorization, fault isolation, timeouts, instancing, concurrency, throttling, state management and most of all context flow. In essence, you must craft a Service-Oriented Programming Model for your teams that promotes consistency, repeatability and sustainability across your entire distributed system effort.
Ultimately, it is this service-oriented programming model that will allow you to efficiently and effectively support multiple layers of service granularity. This is what I believe is missing from the current microservice movement; consistently applying the microservice metaphor at multiple levels of abstraction as shown in Figure 1. The current recommendations simply do not go deep enough.
Whatever you do, do not dismiss the notion of crafting a service-oriented programming model for your teams as too hard or advanced an endeavor. The value for the investment is clearly a net positive. I can assure you it will pay you handsome dividends over the long haul of your system’s lifetime. Currently in the industry today, there a few mature, very well-designed connectivity frameworks that already provide you all the necessary ingredients for the recipe. And every other connectivity framework is trending this direction as well. For example with .NET’s Windows Communication Foundation (WCF), you can establish a working foundation for your service-oriented programming model in as little as 400 lines of code. You can start by providing an opinionated, convention-based veneer over the framework’s existing capabilities and let the framework do the heavy lifting.
It should go without saying that as architect you must code. Not only should you code, you must command an architectural understanding of a broad swath of technical knowledge relevant to the problem at hand. You must also be able to display an unequivocal technical leadership in the one aspect that matters most to your modern system; connectivity. You will need to educate management that what you code is different. You must own the connectivity infrastructure that facilitates your service-oriented programming model.
With these goals in mind, you must choose your stack wisely. The last thing you want to do is to recreate for your system Devs the endlessly churning and deprecating patch quilt that is now the state of your UI stack. Platforms like .NET, Node.js and Java all provide the necessary base ingredients to craft a service-oriented programming model as a single stack. You will find some stacks are more expressive than others when you assess their level of support for the many, all-important aspects of service orientation. But assess you must because none of the vendors to this point have delivered on what Juval Löwy first proposed back in 2007; a consistent, cohesive Service-Oriented Platform.

System Delivery

Since integration is such a central theme of a system-centric approach, it has dramatic effect on how you successfully deliver software. Integration implies dependencies. And dependencies imply there’s an ordering to your system’s construction. Features will emerge only as dependencies integrate and as points of integration aggregate. Coming at this ad hoc will only lead you to great amounts of cyclical refactoring as your teams build things out of order, attempting to divine the system’s appropriate structure through repeated experimentation. Instead, your design must indicate the necessary relationships and dependencies between system components. You can then use these relationships to inform your teams of your system’s structure, as well as, aide you in planning.
A system-centric approach also changes significantly the way you test. Unit tests are only the beginning and ultimately a small part of your system testing strategy. Given that integration is so predominate, this is where you should invest your testing dollar. You should develop techniques that allow you to test each and every integration point as your unit under test grows in scope. I call this the spiral of test. At the smallest scope of the spiral is your traditional unit test, which tests your service-based logic without any of the service-oriented aspects in play. Then you increase the scope of the spiral to include those aspects and test again. Then, on each successive test iteration the spiral expands to include one more integration point. Methods integrate into service operations. Operations integrate into services. Service-based components aggregate into one or more internal integration points. Component interactions integrate into use cases. Use cases integrate into subsystems and so on. As soon as you reach critical mass at the use case level with your testing efforts, system load testing should kick in. As much as possible you should use scenario-based load testing to model how clients or other systems actually consume your system.
Integration aggregation eventually reaches the app, where your UI guys should have applied the same techniques to test their model, view and controller components and to mock your system API. If your organization is seriously committed to quality, this is the investment required. By the time your system reaches production, every integration point has been tested so many times you can actually sleep peacefully at night.

Proper Posture

As you craft your new system-centric architectural style and grow your teams in the Brave New World you find yourself in, do not follow fashion or vendor vision. Model your approach after what the big boys do. Adopt a system-centric mindset from the outset for all phases of your endeavor. Your methodology for design, technology selection, coding practices and most of all your development planning and process must all be system-centric. Seek out industry leaders whom like IDesign are experts in system design and construction. You will know these experts because they will provide clear methodologies for all software phases including delivery that are system-centric, not solely application-centric.
Train and prepare your teams to build and test systems that leverage multiple interaction modes. And if you believe like me that Modern Software Architecture is actually a software fractal, you must also endeavor to enable multiple levels of service granularity. Effectively, you must be able to apply the microservice metaphor at multiple levels of abstraction, not simply at the system level or through use of only a single interaction mode.
Taking cues from The Cloud Builders, as well as, seasoned systems experts has helped me bring a system-centric posture to my architect practice across design, technology, coding and process. And leveraging the cutting-edge techniques of a service-oriented programming model has helped me empower many diverse teams to achieve escape velocity and leave Appland forever behind.
View this raw, rollicking article as a flashpoint for action. I’ve just scratched the surface of the many system-centric ideas and concepts you will need in your tool box. I hope this introduction inspires you to dig deeper into how The Cloud Builders and systems experts conceive, design and deliver Modern Software Architecture. Studying these techniques will change forever the way you think about software and bring a more system-centric swagger to your architect practice. Empower your teams to build for the future and finally escape Appland once and for all.

About the Author

Michael ‘Monty’ Montgomery is a master software architect and trainer with the software architectural firm IDesign (idesign.net), where he helps clients conceive, design and deliver microservice-based systems. Monty will reveal in detail how he delivers microservice-based systems at the upcoming O’Reilly Software Architecture Conference March 16-19, Boston. Learn more or contact Monty at idesign.net.