Azure DevOps provides a suite of tools that your development team needs to plan, create, and ship products. It comes in two flavors:
- Azure DevOps Services – the SaaS option hosted by Microsoft.
- Azure DevOps Server – the IaaS option hosted by you.
When comparing the two to decide which option enables your team to deliver the most value in the least amount of time, Azure DevOps Services is the clear winner, but velocity alone is not the only consideration for most government teams. The services you use must also be compliant with standardized government-wide security, authorization, and monitoring requirements.
Azure DevOps Services are hosted by Microsoft in Azure regions. At the time of this writing, you do not yet have the option to host Azure DevOps in an Azure Government region so you must use one of the available Azure Commercial regions. As a public service, it has also has not yet achieved compliance with any FedRAMP or DoD CC SRG audit scopes. This may sound like a non-starter, but as it states on the FedRAMP website, it depends on your context and how you use the product.
Depending on the services being offered, the third-party vendor does not necessarily have to be FedRAMP compliant, but there are security controls you must make sure they adhere to. If there is a connection to the third-party vendor, they should be listed in the System Security Plan in the Interconnection Table.
This is the first of two blog posts that will share solutions to common Azure DevOps Services concerns:
- In “Azure DevOps Services for Government: Access Control” (this post), I will cover common access control concerns.
- In “Azure DevOps Services for Government: Information Storage”, I will address common concerns with storing information in the commercial data centers that host Azure DevOps Services.
Managing User Access
All Azure DevOps Organizations support cloud authentication through either Microsoft accounts (MSA) or Azure Active Directory (AAD) accounts. If you’re using an MSA backed Azure DevOps organization, users will create their own accounts and will self-manage security settings, like multi-factor authentication. It is common for government projects to require a more centralized oversight of account management and policies.
The solution is to back your Azure DevOps Services Organization with an AAD tenant. An AAD backed Azure DevOps organization meets the following common authentication and user management requirements:
- Administrators control the lifecycle of user accounts, not the users themselves. Administrators can centrally create, disable, or delete user accounts.
- With AAD Conditional Access, administrators can create policies that allow or deny access to Azure DevOps based on conditions such as user IP location.
- AAD can be configured to federate with an internal identity provider, which could be used to enable CAC authentication.
Controlling Production Deployment Pipelines
The value added by Continuous Delivery (CD) pipelines includes increasing team velocity and reducing the risks associated with introducing a change. To fully realize these benefits, you’ll want to design your pipeline to begin at the source code and end in production so that you can eliminate bottlenecks and ensure a predictable deployment outcome across your test, staging, and production environments.
It is common for teams to grant limited production access to privileged administrators, so some customers raise valid concerns with the idea of a production deployment pipeline when they realize that an action triggered by a non-privileged developer could initiate a process that ultimately changes production resources (ex: deployment pipeline triggered from a code change).
The solution is to properly configure your Azure DevOps pipeline Environments. Each environment in Azure DevOps represents a target environment of a deployment pipeline. Environment management is separate from pipeline configuration which creates a separation of duties:
- Team members who define what a deployment needs to do to deploy an application.
- Team members who control the flow of changes into environments.
Example
A developer team member has configured a pipeline that deploys a Human Resource application. The pipeline is called “HR Service”. You can see in the YAML code below, the developer intends to run the Deploy.ps1 scripts on the Production pipeline Environment.
If we review the Production environment configuration, we can see that an approval check has been configured. When the deployment reaches the stage that will attempt to run against the production environment, a member of the “Privileged Administrators” AAD group will be notified that deployment is awaiting their approval.
Only the Privileged Administrators group has been given access to administer the pipeline environment, so the team member awaiting approval would not be able to bypass the approval step by disabling it in the environment configuration or in the pipelines YAML definition.
By layering environment configuration with other strategies you’ll establish the boundaries needed to protect your environment but will also empower your development team to work autonomously and without unnecessary bottlenecks. Other strategies to layer include:
- Governance enforces with Azure Policies
- Deployment gates based on environment monitoring
- Automated quality and security scans into your pipeline
It is also important for all team members involved with a release to be aware of what is changing so that you are not just automating a release over the wall of confusion. This is an example of how a product alone, such as Azure DevOps, is not enough to fully adopt DevOps. You must also address the process and people.
Deployment Pipeline Access to Private Infrastructure
How will Azure DevOps, a service on a commercial Azure region, be able to “see” my private infrastructure hosted in Azure Government? That’s usually one of the first questions I hear when discussing production deployment pipelines. Microsoft has a straight-forward answer to this scenario: self-hosted pipeline agents. Azure DevOps will have no line of sight to your infrastructure. The high-level process to follow when deploying an agent into your environment looks like this:
- Deploy a virtual machine or container into your private network.
- Apply any baseline configuration to the machine, such as those defined in the DISA’s Security Technical Implementation Guides (STIG).
- Install the pipeline agent software on the machine and register the agent with an agent pool in Azure DevOps.
- Authorize pipelines to use the agent pool to run deployments.
With this configuration, a pipeline job queued in Azure DevOps will now be retrieved by the pipeline agent over 443, pulled into the private network, and then executed.
Conclusion
In this post, I’ve introduced you to several practices you can use to develop applications faster without sacrificing security. Stay tuned for the next post in this series where we will discuss common concerns from Government clients around the storage of data within Azure DevOps.
