(Don’t miss Part Two and Part Three of this series!)

Early in my web development career, I always tried to avoid deployment work. It made me uneasy to watch other developers constantly bang their heads against their desks, frustrated with getting our app deployed to whatever cloud service we were using at the time. Deployment work became the “short straw” assignment because it was always a long, unpredictable and thankless task. It wasn’t until I advanced in my tech career that I realized why I felt this way.

My experience with deployment activities, up to this point, always involved a manual process. I thought that the time it took to set up an automated deployment mechanism was a lot of unnecessary overhead – I’d much rather spend my time developing the actual application and spend just a few hours every so often on a manual deployment process when I was ready. However, as I got to work with more and more experienced developers, I began to understand that a manual deployment process is slow, unreliable, unrepeatable, and rarely ever consistent across environments. A manual deployment process also requires detailed documentation that can be hard to follow and in constant need of updating.

As a result, the deployment process becomes this mysterious beast that only a few experts on your development team can tame. This will ultimately isolate the members of your development team, who could be spending more time working on features or fixing bugs related to your application or software. Although there is some initial overhead involved when creating a fully automated deployment pipeline, subsequent deployments of the same infrastructure can be done in a matter of seconds. And since validation is also baked into the automated process, your developers will only have to devote time to application deployment if something fails or goes wrong.

This three-part blog series will serve to provide a general set of instructions on how to build an automated deployment pipeline using Azure cloud services and Octopus Deploy, a user-friendly automation tool that integrates well with Azure. It might not detail out every step you need, but it will point you in the right direction, and show you the value of utilizing automated deployment mechanisms. Let’s get started.

An Overview of Azure Resource Manager Templates (ARM)  

Microsoft Azure is an enterprise-grade cloud computing platform that provides a vast collection of integrated services related to analytics, computing, networking, data storage and much more. Azure also offers a web interface called the Azure Portal, where you can easily create and manage individual resources like virtual machines, virtual networks, application gateways, load balancers, etc. with relatively little effort.

Despite being easy to use and navigate, deploying a complex cloud infrastructure in the Azure Portal is still a manual process, and one prone to human error which can lead to inconsistent behavior and unexpected results.

To alleviate the burdens that come along with these potential inconsistencies, Azure offers a feature where you can deploy cloud resources with a JSON template which functions as a deployment configuration file. With nothing but a terminal or a PowerShell session, you can spin up an entire cloud infrastructure with a few commands. Even though you might spend some time creating the initial JSON template, subsequent deployments of that same infrastructure will be much faster and more consistent than deploying with Azure Portal alone. In this first part of the series, I’ll go over how to create an Azure ARM template that can be used to provision an entire custom cloud infrastructure and networking system.

Creating an ARM Template

To understand how ARM templates are constructed, take a look at some of the examples that Azure provides for a variety of different cloud infrastructure patterns. You can check them out here. Notice that most of the folders contain two files:

  • json – this will function as your main template or configuration file; this should expose a set of parameters that you can use to configure your deployment. Parameters might be things like a virtual machine size, the name of a storage account, or even an administrative password and username.
  • parameters.json – this is a parameter configuration file that you can deploy along with your main template; this file will contain the specific values for your configuration related to each parameter in the main template (azuredeploy.json)

There is a lot of material available online regarding how to properly create these templates. The links below will help guide you:

Let’s take a look at this specific example of a template that can be used to create a simple Linux virtual machine with a public IP Address inside a virtual network.

By looking at the example above, we can see that Azure ARM templates are divided into four main sections. Each section corresponds with a top level property on the JSON object represented in the file from the link above:

    • parameters (Object) – This is where you want to expose any configuration values that can configured by the user who is deploying the template. You can also validate parameters by listing accepted values as well as providing the parameter type (string, boolean, array, integer etc.) and default values. If you do not provide a default value, the parameter will function as a mandatory parameter that must be passed into the template for the deployment to work. Here are some examples:
  {
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
"skuName": {
  "type": "string",
  "allowedValues": [
    "Standard_Small",
    "Standard_Medium",
    "Standard_Large"
  ],
  "defaultValue": "Standard_Medium",
  "metadata": {
    "description": "Sku Name"
  }
},
"capacity": {
  "type": "int",
  "minValue": 1,
  "maxValue": 10,
  "defaultValue": 2
},
"IPAddresses": {
  "type": "array",
  "defaultValue": [{"IpAddress":"10.0.0.4"}, {"IpAddress":"10.0.0.5"}]
}
    },
   “variables”: {
	…
   },
“resources”: [{
	…
 }]
}
  • variables (Object) – This is where you will put any constants that will be used for the deployment process.
  • resources (Array of objects) – Here you will create a list of resources to be deployed. Each resource can be configured using references to the parameters and variables sections. A resource can be a variety of things including (but not limited to) a virtual machine, virtual network, storage account, virtual machine extension, public IP address, or a load balancer. You can even have a resource that acts as a sub-deployment and provisions several other resources from another template file. (I will touch more on this later.)
  • outputs (Array of objects) – Place references to parameters and variables here; when the deployment is complete, these parameters and variables will be available to use for subsequent deployments.

Study the template in the link provided above and understand how it fits together. It’s important to take note of the following things:

  • Referencing items in the parameter and variable section within the resources and outputs section.
  • Using built-in template functions. Some important functions that I use quite frequently are concat(), uniqueString(), resourceGroup.id(), and copyIndex(). Visit this link for more information. copyIndex() is very useful when you want to dynamically deploy different instances of the same resource type.
  • Understanding how to use the dependsOn parameter within the resources section. With this feature, you can run sub-deployments in parallel or in series. This is extremely important because some resources need to be deployed before others in order for the deployment process to work.

Alternatively, you can create a cloud infrastructure in Azure Portal and export a template that you can use in the future. However, since there is not one specific way to organize a template for a specific environment, I tend to avoid this route, especially with more complicated infrastructure patterns. I like knowing how my template fits together and what parameters/variables my resources are being derived from. If you create the template yourself, you will have an easier time debugging down the road.

A More Complicated Example: Nesting and Linking Templates Together

The example in the previous section is a template for a relatively simple cloud infrastructure. As you can see, an ARM template can still get pretty long even when the respective infrastructure is relatively simple. I was recently assigned a task to create an ARM template for a cloud infrastructure environment that included:

  • Two SQL Databases with Always On configured
  • An Active Directory with two domain controllers
  • One application gateway or Layer 7 load balancer that handles routing to two scalable application servers

If you don’t know what some of the things are above, don’t worry – it’s not really important for this post. Still, you can probably imagine how large the template would be to create this model.

In order to make a template readable and easier to reason about, you can create a main template made up of smaller, nested templates instead of cloud resources (virtual networks, virtual machines, public IP addresses etc.). You can then organize specific cloud resources together within these nested templates. As a result, your top level or main template will contain all the necessary parameters for each sub-deployment template. Each sub-deployment template will only contain parameters related to its own deployment.

If we use nesting for the example above, we could have one nested template to deploy base cloud resources like virtual networks and network security groups, one template to deploy and configure our Active directory, one template to deploy and configure the SQL Always on databases, one template to deploy our application resources, and one template to configure and deploy our application gateway. This is only one way to do it – there are plenty of options to configure your deployment to best fit your needs.

You will need to host your nested template files somewhere online – this could either be on Github (publicly available), an Azure storage account (publicly or privately available), or any other similar service. After you do that, you can link to each template within your main deployment template. I won’t go into detail on how to do this, but definitely check out this resource on the topic. You can also use this method of linking to external resources when you want to use a script for a virtual machine extension or desired state configuration (DSC). I will touch more on this later. A lot of the more complicated infrastructure patterns in the Azure quick start Github project use nesting as a best practice.

Shared Access Tokens

If you are privately storing your templates on an Azure Storage Blob, you will need to generate a shared access token that you can append to the URL of you template files. Once you generate a shared access token, you can use it to deploy your main template hosted in your Azure Storage Blob. You also have to pass in the shared access token as a parameter to your main template so you can use it to construct your nested template URLs. There are no built in ARM template functions to create these, so you must generate them using PowerShell. See lines 50-53 of this script for an example on how to do this. I will go over this script in more detail in the section about Azure Automation and Runbooks.

Now you know how to create a custom template for your networking infrastructure with only two JSON files. In the next part of this series, we will go over how to actually deploy your template to provision physical resources on Azure Portal.

(Don’t miss Part Two and Part Three of this series!)