Working Page – All Materials are DRAFT. 

Description

From a grammatical perspective, DevOps is a combination of the words ‘Development’ (Dev) and ‘Operations’ (Ops). It is not a mythology, or software, but a ‘way of doing’.  DevOps is a set of practices that attempts to bridge the gap between two practitioner work approaches defined by the Development and Operations community. In bridging the ‘gap’, the combined effort promises to be ‘better together’, delivering speedy, optimized and high quality software.  DevOps pulls heavily from Agile thinking and incorporates these principles into the software delivery cycle.  DevOps practices include the use of existing disciplines, e.g. governance, quality assurance (QA), testing, security and release management, but extends this work-effort to include disciplines involving people, culture, processes, tools and methodologies that reduce risk and cost, enables technology to change at the speed of the business, and improve overall quality.  This aims to solve a fundamental business problem: reduce the time-to-market for new products and features based on ever evolving customer needs; quickly resolve operational and support issues and; show greater responsiveness to changing (internal and external) environmental situations.

As for DevOps Architecture: Architecture is the process of development of methodical information technology specifications, models and guidelines. DevOps is commonly understood to encompass related technical goals under one umbrella of practice. The architect’s responsibility is therefore to describe these related technical goals (using Architecture Descriptive Language) under the DevOps umbrella, specific to people, culture, processes, tools and methodologies.

Perspective

DevOps from a process architecture perspective

Market pressure demands that organizations continuously respond to ongoing changes and evolving environmental factors. The interplay between different market forces requires enterprises to react and adapt to change more quickly than ever before. This gave rise to what we call ‘chaordic organizations’ and created a stronger reliance on software for the development and delivery of appropriate products and services. Thus, software processes are becoming an integral part of enterprise processes. Current methods of modelling software process reconfigurations are limited in their ability to consider multiple enterprise perspectives and inform selections from alternate configurations. For the rapid and frequent delivery of new software product features and service innovation, the DevOps approach overcomes some of these challenges by enabling;

  • The promotion of a culture of collaboration and information sharing between multiple teams—the traditional approach of having organization silos with defined boundaries and handover points is discouraged, and team members are expected to collectively collaborate towards the attainment of enterprise objectives.
  • The automation of activities in the overall software development process, through the introduction of software tools and custom development of scripts, thus shortening the time required for new feature development and bug fixes through reduction of manual effort—this enables software teams to deliver more frequent releases to customers and the user base.
  • The usage of feedback loops to continuously improve software development processes and the development of product features, through the monitoring and measurement of various software process and technical metrics—these metrics are then interpreted and utilized for overall process improvement.

The framework and the enablers align as shown figure below: A culture of collaboration and information sharing between multiple teams is depicted by Agile disciplines. Automating activities in the overall software development process is depicted by Continuous Integration. Using feedback loops for continuously improving software development processes is depicted by Continuous Process Improvement.  [1].

devops-framework

Figure 1 DevOps framework – Based on work performed by Intel

The framework of figure 1 depicts the DevOps process design, systems and tools development and deployment, and social and organizational issues. The outcomes of DevOps are Continuous Integration (CI) and Continuous Deployment (CD) of product functionality and infrastructure setup[1]. These characteristics are not unique to DevOps, and indeed, are generally applicable to enterprises with respect to enterprise agility and enterprise digital transformation.

DevOps from a practice perspective

CI and CD fit into the overall DevOps process. The simplified DevOps practice consists of Production Management, Development and Automation Engineering streams. Development consist of DevOps Engineering, QA Engineering and Software Engineering. At the centre of DevOps is the development stream. This stream is depicted in yellow in the framework and labelled ‘Agile’. The Automation Engineering is labelled Continuous Integration (inclusive of Continuous Delivery). The process is underpinned by the Continuous Process Improvement stream.

Quality Assurance Testing

Once a feature has been developed, it should be tested (functionally and/or end-user testing) before it goes through the Continuous Delivery process. This testing can be carried out by Quality Assurance engineers in at least two ways:

  • they can retrieve the committed code from the code repository and test it in a controlled environment
  • they can collaborate with the software engineer to quickly validate the functionality before the codebase is committed to the code repository.

Consideration: The degree to which Trunk Based Development is implemented, serves as a best practice guide. Trunk Based Development (TBD) is a development process in which all developers of a deployable unit commit to one shared branch under source-control.

 

Release Planning

The enterprise is assumed to have both periodic and fixed release cycles of appropriate duration. A release planning activity is carried out at release initiation, which results in a release backlog; this artefact is then used to plan out individual sprint iterations. Two of the possible alternatives are;

  • The release backlog is produced once and remains static throughout the release duration
  • The release backlog is revisited at the beginning of every sprint and “groomed” (i.e. reordered and re-estimated), based on on-going change in circumstances and priorities.

Consideration: Agile implementation in the organization

 

Automated Testing

To reduce product delivery duration, some product testing can be automated by developing test plans that are then scripted for execution as part of the Continuous Integration process. The test scripts can be developed once and reused for subsequent CI activities, or they can be developed every time to serve specific testing needs based on the product feature being tested.

Consideration: Consider both Functional and Non-Functional testing using test scripts. Non-Functional aspects can be tested with a more generic test scripting language. Functional testing is very dependent on the complexity and variability of the code.

Tool Usage for Automation

DevOps is characterized by the usage of third-party tools for CI and CD, server configuration, infrastructure provisioning, deployment management, etc. These tools are configured for repeated use without requiring the knowledge of their inner workings. This is not made part of the DevOps engineering stream since the decision criteria will be organization wide.

Consider: Which tools will be used for automation and who will own the decision.

Roles in the DevOps process

There are three high-level processing communities, namely: Production Management; Development and; Automation Engineer. Each of these communities function independently.

Production Management

A number of articles refer to Global Software Development as a (rapid) production process. Software Development should be viewed as an extension of manufacturing.

The key in production management is recognizing that agile manufacturing is a necessary condition for competitiveness. The agile manufacturing, as discussed in the early 90s, was proposed to solve:

  • Products and services with high information and value-adding content
  • Mobilization of core competencies
  • Responsiveness to social and environmental issues
  • Synthesis of diverse technologies
  • Response to change and uncertainty
  • Intra- and inter-enterprise integration

DevOps adopted the Agile approach, but speed and agility where not the only reasons. Goldman and Nagel argue that; “agile manufacturing assimilates the full range of flexible production technologies, along with the lessons learned from total quality management, & just-in-time production and lean production.”  How would we define agility then? Bruce, Daly et al states: “A manufacturing system with extraordinary capabilities (Internal capabilities: hard and soft technologies, human resources, educated management, information) to meet the rapidly changing needs of the marketplace (speed, flexibility, customers, competitors, suppliers, infrastructure, responsiveness). A system that shifts quickly (speed, and responsiveness) among product models or between product lines (flexibility), ideally in real-time response to customer demand (customer needs and wants)”.

From these statements, we can deduce that the Agile approach brings about higher quality products, a reduced delivery time, increased communication between teams, and better morale. These statements align with the agile manifesto principles:

  • the software product is the essential focus of the development process
  • human behavior and the quality of interactions are essential enablers and success factors
  • incremental and spiral approaches based on frequent releases and strict collaboration with the customer, are the quintessential traits of modern software development initiatives
  • in general, it is vital to promptly and quickly react to requirement changes at any stage of the development process.

Thus, the launch pad for DevOps adoption is implementing the Agile principles in the software development production process, but the success of an organization goes beyond Agile implementation in ‘IT’ projects. Fitzgerald and Stol states, “Enterprise Agile concept has emerged as a recognition that the benefits of agile software development will be suboptimal if not complemented by an agile approach in related organizational function such as finance and HR.”

Development

The process above describes continuity between Production Management, Development and Automation Engineers.  The process clearly shows continuous integration as a practice that emerges to eliminate discontinuities between development and deployment. In the development, community development is represented by the Software Engineering role and deployment is represented by the DevOps Engineering role. The glue that brings together development and deployment and renders it continuous, is the QA Engineering role. In figure 1, a combination of Agile Manufacturing and Continuous Integration results in Continuous Delivery, as depicted by the process. The emphasis is on Continuous Delivery.  “We believe that rather than focusing on agile methods per se, a useful concept for assessing continuous concepts in software development comes for the lean approach, namely that of ‘flow’”—Fitzgerald and Stol. Therefore, Agile approach starts the journey of DevOps, but is itself not DevOps. “Rather than a sequence of discrete activities, performed by clearly distinct teams or departments, the argument for continuous software engineering is to establish a continuous movement, which we argue closely resembles the concept of flow found in lean manufacturing and product development, a school of thought that is called ‘Lean Thinking’”. A fundamental focus in lean thinking is that of ‘value,’ and ‘reducing waste’. Any product feature or development step that does not add value is considered to be waste (deploying infrastructure by hand is waste, unwanted, such as unused product features is waste)” – Fitzgerald and Stol.

(Note: From a process architecture perspective, we can identify waste in existing development processes by conducting a value stream mapping exercise, whereby the current processes are identified and visualized to be able to gauge where improvements can be made.)

In this process, the Production Management ‘flows’ into Software engineering. Unlike ‘batch-and-queue’ thinking where actions are done on batches of products, after which they are queued for the next processing step, ‘flow’ refers to a connected set of value-creating actions that occur once a product feature is identified. The Backlog is immediately designed, implemented, integrated, tested, and deployed.  This is the essence of the DevOps vehicle. This principle of continuous flow not only refers to a software development function in isolation, but should be leveraged as an end-to-end concept that considers other functions within an organization, such as planning, deployment, maintenance and operation.

(Note: Implementing a purely Agile approach could disguise the software development function as flowing to some degree, but the planning and deployment of features is still done in batches, and not in a continuous flowing movement.)

The proposed ‘flow’ is a continuous process of improvement. The development community will rely on improvements in Application Performance Management, Software Development Processes and Tools for improved implementation of the continuous process and measure the success of the improvements (control) using key process indicators (Metric).

Automation Engineering

Traditionally, the QA team’s responsibilities consisted largely of mimicking customers’ usage patterns and minimizing the discovery of unpleasant surprises. In the Waterfall model, a QA team’s badge of honour was its ability to anticipate, find, and deliver reproducible scenarios that could result in problems for users. When dealing primarily with client-side code, the variability of client permutations and a lack of accurate behaviour measures make QA input far less objective and verifiable. Today, the more we understand about the code and platform for an application, the greater the chance for building and fixing that application timeously. Therefore, QA Engineering focuses not only on Software testing, but rather on the combination of ‘Operational Acceptance Testing’, ‘Functional Testing’ and ‘Software Testing’.

QA Engineering focuses on software projects that can be labelled as complex adaptive systems (CAS). CAS’s are composed of agents. CAS agents can be molecules, neurons, web servers, fish, starlings, and people – always forming new emergent structures with new emergent behaviors. CAS’s are able to adapt to their environments: an infant learning to walk; a strain of bacteria becoming resistant to an antibiotic; car drivers evading a traffic jam; an ant colony learning about the location of spilled honey; a software team adapting to what their customer really wants. When applying complex systems theory to software development and management, we are treating the organization as a system.  From a QA perspective, Quality Assurance Engineering has to focus on both outcome-oriented and process-oriented views to effectively evaluate performance from a user/service satisfaction perspective.

(Note: QA Engineering will rely on new models, e.g. EFQM Excellence Model)

DevOps Engineering is largely based on release engineering[1], but extends the task by adding automation using a high degree of manage scripting.  Thus, DevOps can be defined as, “a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality.”

Note: The System Testing should not be confused with the QA Engineer tasks. This testing is merely ‘verification for readiness to proceed based on the QA Engineering documentation’.

Automation Engineering is placed in its own community. Automation Engineering is an enterprise function— “Automation engineers design, program, simulate and test automated machinery and processes in order to complete exact tasks”. This enterprise function focuses on the tools that are needed by the Development community, as well as the Test Plan and test artifacts (scripts, etc.). This community’s role should not be confused with the tasks of a DevOps engineer. The automated machinery in this discussion is the end-to-end DevOps pipeline. The Automation Engineering group is responsible for putting in place the Enterprise DevOps pipeline. This pipeline is based on organization and industry standards.

Best Practices

DevOps is a cultural and professional movement. It’s not a tool (thing). It’s not a title (person). It’s not just dev & ops. –Rockwood

DevOps as a practice focuses on the actual application or use of an idea, belief, or method, as opposed to the theory of application or use.  This idea is built around a group of people who believe that the application of a combination of appropriate technology and attitude can revolutionize the world of software development and delivery.

Culture

DevOps is a cultural movement combined with several software development practices that enable rapid development.  A number of technical processes and tools have emerged, but these are not enough to create the collaborative environment that many of us refer to today as DevOps.  But what exactly is meant by culture? As a simple definition, we can say: “Groups of people create a culture through shared values and behaviors”. DevOps is as much about culture as it is about tools, and culture is all about people. “Implementing a prescribed toolchain won’t magically turn your team into a DevOps team. Using DevOps-friendly tools and workflows can help your team work in a more DevOps manner, but creating a culture that is supportive of the ideals of the DevOps movement is crucial”—M Walls.

 

DevOps Culture

On one hand, the tools used in the DevOps practice are relative easy to identify and use. On the other hand, the use of people as tools is complex. Friction between teams thought of as “development” and teams thought of as “operations” boils down to a few key cultural characteristics.

At the heart of the DevOps culture is lots of open discussion and debate. Siloed technical teams interact through complex ticketing systems, whilst ritualistic request procedures obscure communications and mask underlying friction. A team taking a more DevOps approach would talk about the product throughout its lifecycle, discussing requirements, features, schedules, resources, and whatever else might come up. The focus is on the product, not building fiefdoms and amassing political power.

People who are empowered to make a direct improvement are the ones who are alerted to the defect. Development isn’t rewarded for writing lots of code. Operations isn’t punished when all that code doesn’t run as expected in production. The team is rewarded when the product is awesome, and shares in the improvement process when the product could be more awesome—M Walls.

Respect

Team members should respect all the other team members. No team member should be afraid to speak for fear of being abused or rudely dismissed. Trust is a massive component of achieving a good DevOps culture. Operations must trust that Development is doing what they are because it’s the best plan for the success of the product. The cultural shift is acknowledging that everyone has valuable input and communicates well with each other. We can call this shift a ‘DevOps cultural shift’. It is a change in the value of some new behaviors, versus the old ones. This shift will be the primary barrier to overcome. This cultural shift needs to originate from the desire to know the goals of your business (strategy) and successfully relate this into daily activities (tactics). DevOps cultural change should be aligned to the successful execution of the overarching business goals (Vision, Mission and translated functional tactics). These business drivers for competitive advantage became transparent, and also the primary goal to succeed in.

Automation

Beside cultural and organizational changes that are necessary to implement DevOps in practice, tooling is required to implement end-to-end automation of deployment processes. Automation is the key to efficient collaboration and tight integration between development and operations.

The traditional split between developers and operations found in many organizations today is a major obstacle for the fast and frequent release of software. Using automation to overcome this obstacle is a competitive advantage that will enable this.

DevOps Artifacts

Reusable DevOps artifacts such as scripts, modules, and templates are publicly available to be used for deployment automation, but are usually bound to certain tools. Thus, there is a variety of solutions and orchestrating the best of them requires to integrate the corresponding tools, e.g. writing workflows or scripts that handle the individual invocations, the parameter passing, etc. One attempt to seamlessly integrate different kinds of DevOps artifacts, is the emergence of OASIS standard TOSCA (Topology and Orchestration Specification for Cloud Applications).

DevOps artifacts are based on two major classes:

  1. Node-centric artifacts (NCAs) are scripts, images, modules, declarative configuration definitions, etc. that are executed on a single node such as a physical server, a virtual machine (VM), or a container. Cross-node relations, such as an application component connecting to a database running on another node, are not explicitly expressed and implemented. Consequently, NCAs are not meant to be used to deploy complete application topologies.
  2. Environment-centric artifacts (ECAs) are scripts, bundles, templates, etc. that are executed in an environment, potentially consisting of multiple nodes. Cross-node relations are explicitly expressed and implemented. Consequently, ECAs can be used to deploy complete application topologies.

Unix shell scripts, Chef cookbooks, Puppet modules, SaltStack modules, and Docker images are a few prominent examples for NCAs. In terms of ECAs, Juju charms and bundles, Amazon Cloud Formation templates, and OpenStack Heat templates are representatives of this class of artifacts.

Automation in DevOps will be dependent on the artifacts chosen and the maturity of Orchestration Specification. Care should be taken to not develop point solutions based on any specific artifacts or set of artifacts.

Measurement

DevOps is a journey of discovery, not a destination—Rockwood.

But how do we measure success on that journey? DevOps needs to measure key performance indicators (KPIs) to demonstrate success and drive further transformation. DevOps grew out of the Agile movement (where the goal was for development teams to speed up software delivery) to an enterprise movement that includes every part of the software delivery process (planning all the way through securing, monitoring, and updating an application that’s already in production).

Today, DevOps is widely accepted as the way that businesses of all kinds will build, test, and deliver software in the future. However, speed cannot come at the expense of quality. Delivery of a poor-quality product will be remembered long after a late delivery will.

Unlike traditional software deployment, where an increase in speed (velocity) often resulted in a decrease in quality, DevOps overcomes these constraints by releasing in small batches, yet maintaining quality and security.

Although an argument can be put forward that an increase in velocity and quality will cost more, DevOps initiatives claim to drive down all three. They key ingredient here is adopting a metrics framework that indicates where you are succeeding and where you still need to improve.

Why have metrics?

Simply put: To generate executive support for DevOps. Since DevOps has no formal framework, a few standard ways to measure progress will be crucial to demonstrate value. Most organization will follow a gradual adoption strategy; early wins should be clearly articulated based on fact. DevOps is an evolutionary journey that will follow an iterative process of constant refinement.

 

Monitor

The inclusion of automated benchmarks into continuous integration setups enables early detection and repair of performance issues before such issues are propagated into a release. In DevOps, the software engineering discipline needs to address the operation of continuously running software services. Besides functional acceptance tests, automated tests of non-functional quality attributes, such as performance, are required to ensure seamless operation of the software. An important requirement for DevOps thus, emerges: Continuous monitoring of software runtime behaviour. In contrast to profiling for construction activities, monitoring of operational services should only impose a small performance overhead. The Kieker monitoring framework is an example of providing these means with a small performance overhead.

Unlike current practices where benchmark and monitoring instrumentation is often only integrated into the software when problems occur after release (in the production environment of IT operation), DevOps includes benchmarks into Continuous Integration early in the development process.

As a DevOps best practice, measurement instrumentation, i.e. benchmarks as well as monitoring, should be integrated into software development right from the start. Automated benchmarks are then able to detect performance issues that may have been introduced between versions before these issues are propagated via the deployment pipeline. Monitoring can aid in the early detection of further performance issues that only manifest in the actual deployment of the software.

Capabilities

DevOps is evolutionary in its thinking with regards how IT services are delivered and supported. The schools of thought and practices are largely based on preceding work in the area of Continuous Integration and Application Life Cycle Management (ALM) and is therefore rooted in the agile philosophy, which also attempts to bridge the traditional organizational process divide between development and operations teams. DevOps’s value lies behind bridging the current gap between the different technical roles who work in silos within the same team.

“The DevOps approach is thus, built around those who believe that the application of a combination of appropriate technology and attitude can revolutionize the world of software development and delivery especially these different roles share the same objective which is the delivery of a successful products under a stressful market conditions” – S. I. Mohamed.

In Conclusion

The obstacles facing DevOps adoption may be marginalized as merely “technology problems”—however, business leaders who do so put themselves in grave risk. Transforming the IT value stream by adopting DevOps methods will likely result in a productivity surge as large, or larger, than the manufacturing revolution 30 years ago, making this one of the most important and urgent business problems of our age.”[1]

DevOps is not simply a ‘putting together’ of developer and operations teams, but a number of key practices, organizational changes and a carefully planned journey. DevOps is an integral part of the organization and not a separate team doing other work away from everyone else. Although tooling is an integral part of DevOps, it is not ‘The DevOps’, rather DevOps is about adopting automation, increasing collaboration and culture, to name but a few.

DevOps is not only an IT problem concerned with software developers to system administrators, but rather, at its core, DevOps is a business problem. The DevOps movement is about facilitating a cultural and operational shift, managing people and processes to create a well-oiled machine that can develop, deploy, and respond to changes in the internal or client requirements quickly and efficiently.

To conclude: “The business case couldn’t be clearer: just as in the manufacturing revolution, there will be winners and losers. As Dr. W. Edwards Deming said, “Learning is not compulsory, but neither is survival.” – Gene Kim, author of ‘Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win’

[1] http://blogs.wsj.com/cio/2014/05/22/enterprise-devops-adoption-isnt-mandatory-but-neither-is-survival/

Resources

[1].  Kruchten, Philippe, Architectural Blueprints — The “4+1” View Model of Software Architecture. IEEE Software 12 (6), pp. 42-50, November 1995.

Babar, Z., et al. (2015). Modeling DevOps Deployment Choices Using Process Architecture Design Dimensions. The Practice of Enterprise Modeling, Springer: 322-337.

 

Bruce, M., et al. (2004). “Lean or agile: a solution for supply chain management in the textiles and clothing industry?” International journal of operations & production management 24(2): 151-170.

 

Economou, F., et al. (2014). “Your data is your dogfood: DevOps in the astronomical observatory.” arXiv preprint arXiv:1407.6463.

 

Ellis, B. and S. Herbert (2011). “Complex adaptive systems (CAS): an overview of key elements, characteristics and application to management theory.” Journal of Innovation in Health Informatics 19(1): 33-37.

 

Fitzgerald, B. and K.-J. Stol (2014). Continuous software engineering and beyond: trends and challenges. Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering, ACM.

 

Fowler, M. and M. Foemmel (2006). “Continuous integration.” Thought-Works) http://www. thoughtworks. com/Continuous Integration. pdf: 122.

 

Fuggetta, A. and E. Di Nitto (2014). Software process. Proceedings of the on Future of Software Engineering, ACM.

 

Herbsleb, J. D. and D. Moitra (2001). “Global software development.” Software, IEEE 18(2): 16-20.

 

HP (2015). “Measuring DevOps success.”

 

Kerzazi, N. and B. Adams “Who Needs Release and DevOps Engineers, and Why?”.

 

Mohamed, S. I. “DevOps shifting software engineering strategy Value based perspective.”

 

Myburgh, A. B. (2015). “Complex Adaptive Responses of Organizational Teams.” Int. J. Comput. Sci. Theor. App 4(1): 23-39.

 

Nagpal, S. and A. Shadab “Literature Review: Promises and Challenges of DevOps.”

 

Rockwood, B. “The DevOps Transformation.” keynote at the USENIX LISA 11.

 

Waller, J., et al. (2015). “Including performance benchmarks into continuous integration to enable DevOps.” ACM SIGSOFT Software Engineering Notes 40(2): 1-4.

 

Walls, M. (2013). Building a DevOps culture, ” O’Reilly Media, Inc.”.

 

Wettinger, J., et al. (2014). Standards-based devops automation and integration using tosca. Proceedings of the 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing, IEEE Computer Society.

 

Mohamed, S. I. “DevOps shifting software engineering strategy Value based perspective.”

Author

Evan E.