Also in this series:

Terragrunt Example

Let’s consider the situation where we want to maintain the infrastructure for a system with two major components: an API and a database solution. You must also deploy dev, test, stage, and production environments for this system. Dev and Test environments are deployed to one region, while stage and production environments are deployed to two regions.

We’ve created a preconfigured sample repository to demonstrate how we might handle something like this with Terragrunt. Now, although the requirements and scenario described above may not pertain to you, the preconfigured sample repository should give you a good idea of what you can accomplish with Terragrunt and the benefits it provides in the context of keeping your Terraform code organized. Also, keep in mind that Terragrunt is unopinionated and allows you to configure it in several ways to accomplish similar results; we will only cover a few of the benefits Terragrunt provides, but be sure to check out their documentation site for more information.

To get the most out of the code sample, you should have the following:

Run through the setup steps if you need to. This will involve running a mini terraform project to provision a few resource groups in addition to a storage account to store your terraform state files.
The sample repo contains several top-level directories:

  • /_base_modules
  • /bootstrap
  • /dev
  • /test
  • /stage
  • /prod
  • _base_modules folder – contains the top-level terraform modules that your application will use. There are subfolders for each application type, the API, and the storage solution (/api and /sql). For example, there is a subfolder for the API, which contains the terraform code for your API application, and one for SQL, which will include the terraform code for your storage/database solution; take note of the main.tf, variables.tf, and outputs.tf files in each subfolder. Each application type folder will also contain a .hcl file that contains global configuration values for all environments that consume that respective application type
  • [dev/test/stage/prod] – environment folders that contain subfolders for each application type. Each subfolder for each application type will contain Terragrunt configuration files that contain variables and inputs specific to that environment
  • Bootstrap – a small isolated terraform project that will spin up placeholder resource groups in addition to a storage account that can be used to maintain remote terraform state files

As mentioned above, there are several .hcl files in a few different places within this folder structure. These are Terragrunt configuration files. You will see one within each sub folder inside the _base_modules directory and one in every subfolder within each environment folder. These files are how Terragrunt knows what terraform commands to use, where to store each application’s remote state, and what variable files and input values to use for your terraform modules defined in the _base_modules directory. Read more about how this file is structured on Gruntwork’s website. With this sample repository, global configurations are maintained in the /_base_modules folder and consumed by configurations in the environment folders.

Let’s go over some of the basic features that Terragrunt offers.

Keeping your Remote State Configuration DRY

I immediately noticed when writing my first bits of Terraform code that I couldn’t use variables, expressions, or functions within the terraform configuration block. You can override specific parts of this configuration through the command line, but there was no way to do this from code.

Terragrunt allows you to keep your backend and remote state configuration DRY by allowing you to share the code for backend configuration across multiple environments. Look at the /_base_modules/global.hcl file in conjunction with the /dev/Terragrunt.hcl file.

/_base_modules/global.hcl:

remote_state {
  backend = "azurerm"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }
  config = {
    resource_group_name  = "shared"
    storage_account_name = "4a16aa0287e60d48tf"
    container_name       = "example"
    key            = "example/${path_relative_to_include()}.tfstate"
  }
}

This file defines the remote state that will be used for all environments that utilize the api module. Take special note of the ${path_relative_to_include} expression – more on this later.
A remote state Terragrunt block that looks like this:

remote_state {
    backend = "s3"
    config = {
      bucket = "mybucket"
      key    = "path/for/my/key"
      region = "us-east-1"
    }
  }

Is equivalent to terraform block that looks like this:

  terraform {
    backend "s3" {
      bucket = "mybucket"
      key    = "path/to/my/key"
      region = "us-east-1"
    }
  }

To inherit this configuration into a child sub folder or environment folder you can do this:

/dev/api/terragrunt.hcl

include "global" {
  path = "${get_terragrunt_dir()}/../../_base_modules/global.hcl"
  expose = true
  merge_strategy = "deep"
}

The included statement above tells Terragrunt to merge the configuration file found at _base_modules/global.hcl with its local configuration. The ${path_relative_to_include} in the global.hcl file is a predefined variable that will return the relative path of the calling .hcl file, in this case,/dev/api/terragrunt.hcl. Therefore, the resulting state file for this module would be in the example container at dev/api.tfstate. For the sql application in the dev environment, the resulting state file would be dev/sql.tfstate; look at the _base_modules/sql/sql.hcl file. For the api application in the test environment, the resulting state file would be, test/api.tfstate. Be sure to check out all of the built-in functions Terragrunt offers out of the box.

Using the feature just mentioned, we only define the details of the remote state once, allowing us to cut down on code repetition. Read more about the remote_state and include blocks and how you can configure them by visiting the Terragrunt documentation. Pay special attention to merge strategy options, how you can override includes in child modules, and the specific limitations of configuration inheritance in Terragrunt.

Keeping your Terraform Configuration DRY

Merging of configuration files do not only apply to remote state configurations – you can also apply them to the sources and inputs of your modules.
In Terragrunt, you can define the source of your module (main.tf or top level terraform module) within the terraform block. Let’s consider the api application:

/_base_modules/api/api.hcl

terraform {
  source = "${get_terragrunt_dir()}/../../_base_modules/api"

  extra_arguments "common_vars" {
    commands = get_terraform_commands_that_need_vars()

    required_var_files = [
      
    ]
  }
}

You’ll notice this is referencing a local path; alternatively, you can also set this to use a module from a remote git repo or terraform registry.
The api.hcl configuration is then imported as a configuration into each environment folder for the api application type:

Ex. /dev/api/terragrunt.hcl

include "env" {
  path = "${get_terragrunt_dir()}/../../_base_modules/api/api.hcl"
  expose = true
  merge_strategy = "deep"
}

Include statements with specific merge strategies can also be overwritten by configurations in child modules, allowing you to configure each environment separately if needed.
Merging inputs before they are applied to your terraform module is also extremely helpful if you need to share variables across environments. For example, all the names of your resources in your project might be prefixed with a specific character set. You can define any global inputs in the inputs section of the _base_modules/global.hcl file. Because Terragrunt configuration files are written in the HCL language, you can also utilize all the expressions and functions you use in Terraform to modify or restructure input values before they are applied. Look at how we are defining the identifier input variable found in both sql and api modules:

Here is the terraform variable:

/_base_modules/api/variables.tf and /_base_modules/sql/variables.tf

variable "identifier" {
  type = object({
      primary = string
      secondary = string
      type = string
  })
}

Here is the primary property being assigned from the global env:

/_base_modules/global.hcl

... 
inputs = {
    identifier = {
        primary = "EXAMPLE"
    }
}
...

Here is the secondary property being assigned from the dev/dev.hcl file:

/dev/dev.hcl

inputs = {
 identifier = {
     secondary = "DEV"
 }
}

And here is the type property being applied in the module folders

/_base_modules/sql/sql.env.tf

...
inputs = {
    identifier = {
        type = "SQL"
    }
}

/_base_modules/api/api.hcl

...
inputs = {
    identifier = {
        type = "API"
    }
}

All configurations are included in the environment configuration files with:

include "global" {
  path = "${get_terragrunt_dir()}/../../_base_modules/global.hcl"
  expose = true
  merge_strategy = "deep"
}

include "api" {
  path = "${get_terragrunt_dir()}/../../_base_modules/api/api.hcl"
  expose = true
  merge_strategy = "deep"
}

include "dev" {
  path = "../dev.hcl"
  expose = true
  merge_strategy = "deep"
}

would result in something like:

inputs = {
    identifier = {
      primary = "EXAMPLE"
secondary = "DEV"  
type = "API"
    }
}

We utilize this pattern to share variables across all environments and applications within a specific environment without having to declare them multiple times.
It is also important to note that because Terragrunt configuration files are written in the HCL language, you can access all of Terraform’s functions and expressions. As a result, because you can inherit Terragrunt configuration files into a specific environment, you can restructure, merge, or alter input variables before they are sent to terraform to be processed.

Running Multiple Modules at once

You can also run multiple terraform modules with one command using Terragrunt. For example, if you wanted to provision dev, test, stage, and prod with one command, you could run the following command in the root directory:

terragrunt run-all [init|plan|apply]

If you wanted to provision the infrastructure for a specific tier, you could run the same command inside an environment folder (dev, test, stage etc.).
This allows you to neatly organize your environments instead of maintaining everything in one state file or trying to remember what variable, backend, and provider configurations to pass in your CLI commands when you want to target a specific environment.

It is important to note that you can maintain dependencies between application types within an environment (between the sql and api application) and pass outputs from one application to another. Look at the dev/api environment configuration file.

/dev/api/terragrunt.hcl

dependency "sql" {
  config_path = "../sql"
    mock_outputs = {
    database_id = "temporary-dummy-id"
  }
}

locals {
}
inputs = {
    database_id = dependency.sql.outputs.database_id
    ...
}

Notice that it references the dev/sql environment as a dependency. The dev/sql environment uses the _base_modules/sql application so look at that module, specifically the outputs.tf file.

/_base_modules/sql/outputs.tf

output "database_id" {
  value = azurerm_mssql_database.test.id
}

Notice that this output is being referenced in the /dev/api/terragrunt.hcl file as a dependency.

The client requirements described earlier in this post proved to be especially difficult to maintain without the benefit of being able to configure separate modules that depend on one another. With the ability to isolate different components of each environment and share their code and dependencies across environments, we could maintain multiple environments effectively and efficiently with different configurations.

Conclusion

Terraform as an infrastructure as code tool has helped us reliably develop, maintain, and scale our infrastructure demands. However, because our client work involved maintaining multiple environments and projects simultaneously, we needed specific declarative design patterns to organize our infrastructure development. Terragrunt offered us a simple way to develop multiple environments and components of a given application in a way that was repeatable and distributable to other project pipelines.

There are several features of Terragrunt we did not discuss in this post:
– Before, After, and Error Hooks
– Maintaining CLI flags

We would like to see some of the functionality Terragrunt offers baked into Terraform by default. However, we do not feel like Terragrunt is a final solution; Terraform is rather unopinionated and less concerned with how you set up your project structure, while Terragrunt is only slightly more opinionated in your setup. Terragrunt claims to be DRY, but there is still a lot of code duplication involved when creating multiple environments or trying to duplicate infrastructure across regions. For example, creating the folder structure for an environment is cumbersome, especially when you want to add another tier.