By now, DevOps is well-established within web companies, unicorns, and product companies—and especially among companies targeting the cloud. To spare you the lengthy introduction, DevOps brings “development” and “operations” together as a moniker for company-wide collaboration that will improve business agility. The key DevOps traits are:
- Involving Ops teams in early stages of development
- Focus on automating all aspects of the IT life cycle
- Continuous improvement
- Maturity of self-service model
Enterprise DevOps Challenges
Despite its success within smaller companies, implementing DevOps in large enterprises has proven to be more difficult. Rachael Shannon-Solomon writes in The Wall Street Journal that DevOps is perhaps better suited for startups at the current time than for enterprise IT. Regardless of whether you agree with her article, it does raise some important points related to siloed structures, organizational change and affecting cultural change on a large scale.
The issue is not that enterprises aren’t adopting DevOps (just look at the latest State of DevOps report for evidence to the contrary); it is the unique set of challenges that large enterprises face that make it harder for DevOps to succeed. Let’s take a closer look:
Large Application Portfolios: Large enterprises—which have thousands of employees, billions in revenue, numerous cycles of mergers and acquisitions—manage dozens of applications. Not only are these applications a mix of built and bought, they are based on a wide spectrum of technology stacks. Each application can potentially have multiple environments (aka “swim lanes” e.g., Dev1-n, Test 1-n, Training 1-n, Load Testing), as well as staging and production. Oftentimes, there are multiple Continuous Integration (CI)/Continuous Deployment (CD) pipelines, monitoring mechanisms, and health dashboards.
DevOps Challenge #1: Develop a unified DevOps model that is capable of dealing with a large and diverse application portfolio in a consistent manner. This includes consistent monitoring, incident handling and logging.
Multiple Frameworks: A good indicator of the IT complexity in large enterprises is the use of multiple reference frameworks like ITIL, CMMI, and COBIT. These reference models are key to managing complexity, and have served large enterprises well over the years.
DevOps Challenge #2: Find an approach that bridges the application-focused frameworks such as CMMI with more operations-centric frameworks like ITIL and COBIT.
Siloed Structures: Enterprises have to deal with calcified silos resulting from the inherent legal, geographical and financial constraints they face. This makes them culturally resistant to change and less agile.
DevOps Challenge #3: How do enterprises break down these silos and adopt DevOps without embarking on a “boil the ocean” culture change or completely redrawing the organizational boundaries?
Ever–growing weight of expectations: Further exacerbating the agility concerns are the increasing expectations from the business for capabilities like self-service and on-demand provisioning of resources, growing impatience with trouble-ticket culture, and a desire to experiment with rapidly-evolving productive platforms (often cloud-based) that can accelerate the time to market.
DevOps Challenge #4: How do enterprises extend their existing infrastructure to a hybrid environment that allows them to take advantage of the elasticity and scale offered by the public cloud?
Getting Started: Establish a DevOps Toolchain
Now that we’ve laid out the key challenges, let’s look at how an enterprise can get started. As you might expect, there is lot of great advice out there on starting with a small team, picking the first mover app, setting up clear metrics and automating error-prone and tedious tasks. But alongside these suggested steps, it is equally important to have “something working early” – no different from any other agile project.
But what does “something working early” mean in the context of DevOps? To answer this question, let’s return to the challenges of a large application portfolio. It is more than likely that enterprises already have “islands of automation” comprised of multiple sets of competing tools, representing areas such as configuration management, CI/CD, monitoring, etc. Consider this example: one team may be building a web application on ASP.NET, another team may be building a headless batch process on Linux…and yet another team may be building out hardened databases on demand.
It is likely that each of these teams have their own distinct automation steps pertaining to CI/CD, provisioning, deployment logic and even the monitoring process – what we referred to previously as “islands of automation.” Turns out that the “something working early” may be to start connecting these “islands of automation” and bring consistency to aspects like provisioning and deployment.
This is where a DevOps toolchain comes in. A traditional toolchain is a distinct set of software tools that connect together by various stages. Likewise, a DevOps toolchain seeks to connect these “islands of automation”. Please refer to the diagram below for a conceptual model of a DevOps toolchain. A DevOps toolchain can be made of one or more of these building blocks. Furthermore, each building block can be realized using a variety of well-established tools depicted in this diagram.
But how does one author these DevOps toolchains in a consistent manner, given that the automation constructs, exposed at each step, are very different – not to mention an enterprise will likely have more than one tool for each step?
This is where the notion of DSL (domain specific language) comes in. DSL can bring consistency to the DevOps toolchain. DSL is used to define the order of the tools in the chain, as well as the expression of policies, such as security and configuration. Treating each instance of a toolchain as code has many benefits, including the ability to repeat, the ability to version and the ability to execute. An instance of a toolchain can include creating a Virtual Machine, securing and deploying a build output, as well as multi-tier applications—complete with provisioning and deployment steps with policies for security and management. Examples of DSL include Azure ARM Templates, Chef Recipes and AWS Cloud Formation.
Service Catalog as a DevOps Toolchain Enabler
Defining a DevOps toolchain using a DSL by itself is not enough. You need a consistent way to author and “execute” the toolchains. This brings us to the notion of a Service Catalog. A Service Catalog allows organizations to create, centrally manage and “execute” DevOps toolchains. Note that each toolchain instance (or collection of toolchain instances) maps to a type of application within the enterprise. For example, if you need an instance of application A, “execute” tool chain A. In this manner, a Service Catalog represents information about all applications that are available and operated within the enterprise.
Let’s review the key features of the Service Catalog:
Consistent authoring experience – Service Catalog provides the ability to author a toolchain based on a DSL. Often the Service Catalog will provide a graphical authoring experience. Authoring experience is key to ensuring that only “approved” steps are included in the toolchain. For example, the deployment step can enforce a constraint on the geo-location of a production instance of the database. This is why it is important to involve the Ops and security team early on in the process. Jointly developing the toolchain can minimize unexpected surprises at deployment time.
Self-Service – Once the toolchain for a given application type has been authored and made available within the Service Catalog, users can self-service provision an application via a central UI Portal. This capability is available without creating service tickets or waiting for long periods of time (days or weeks). Users can also access the status of their submitted requests and obtain reporting and charge-back information via the UI portal.
Workflow – A toolchain may involve a workflow of steps for approval, escalation if milestones are not met and notification upon successful completion. This workflow capability is a way to bridge application process guidance (i.e. CMMI) and operational process guidance (i.e. ITIL).
Extensibility – The Service Catalog DSL is extensible to include the new tools and services being made available at a rapid clip.
The following diagram depicts the conceptual model for Service Catalog. Service Catalog contains a set of toolchains – each representing a distinct application (or more generally, a service offered by the IT department to its users).
To complete the full circle, I also mapped the key features of the Service Catalog, described above, to the specific challenges faced by enterprises adopting DevOps.
Summary
Enterprise DevOps faces many challenges given the sheer size and complexity of the IT portfolio. A good way to get started and produce early results is to select a DSL to define instances of the DevOps toolchain. A toolchain instance can connect disparate “islands of automation” and express policies (such as security and configurability) as code. Having a common DSL can help achieve consistency and compliance across a heterogeneous mix of technology stacks and tools. In this manner, a Service Catalog serves as a key enabler for the DevOps toolchain and a good starting point for your DevOps journey. All that said, tooling alone will not make your DevOps journey successful. A more immersive effort, in the form of process and culture improvements, is needed along the way.
Further Reading:
