Building reusable Terraform Modules

Terraform has become an industry standard when it comes to creating, changing, improving or deleting infrastructure.

In order to build reusable Terraform modules that can be easily leveraged to achieve almost any architecture, I believe that at least the following best practices should be put in place:

Each Terraform module should have its own repository
Use for_each and map variables
Use dynamic blocks
Use ternary operators and take advantage of terraform built-in functions, especially lookup, merge, try, can
Build outputs
Optional: Use pre-commit

Terraform module in its own repository

You may wonder why I am suggesting that every terraform module should be in its own repository.

Well, doing so, will most definitely help you, when you are developing new features to that module, because you won’t need to do big changes in a lot of places.

In addition to this, creating / updating repositories with new features of that module, will occur really fast, and you will still have the possibility to keep the older versions in place for other automations that you are building.

You can easily reference a git source / registry source when you want to use a module, but tagging is essential.

E.G. Referencing a Github Terraform Module:

module "example" {    
  source = "git@github.com:organisation(user)/repo.git?ref=tag/branch"  
}

When it comes to tagging, my recommendation is to use the following tag format: vMajor.Minor.Patch.

Use for_each and map variables

I didn’t specify count, did I?

Using count on your resources, will cause more problems than offer help in the end. I’m not talking about checking if a variable is set to true and if it is, set the count to 1, else set it to 0 (not a big fan of this approach, either), I’m talking about creating multiple resources based on a list variable.

Let’s suppose you want to create 3 ec2 instances in AWS using Terraform, each of them with different images, different types and different azs, similar to the code below.

locals {  
  vm_instances = [  
    {  
      ami           = "ami-1"  
      az            = "eu-west-1a"  
      instance_type = "t2.micro"  
    },  
    {  
      ami           = "ami-2"  
      az            = "eu-west-1b"  
      instance_type = "t3.micro"  
    },  
    {  
      ami           = "ami-3"  
      az            = "eu-west-1c"  
      instance_type = "t3.small"  
    }  
  ]  
}

resource "aws_instance" "this" {  
  count             = length(local.vm_instances)  
  ami               = local.vm_instances[count.index].ami  
  availability_zone = local.vm_instances[count.index].az  
  instance_type     = local.vm_instances[count.index].instance_type  
}

The code is working properly, all of your 3 instances are created and everything is looking smooth after you apply it.

Let’s suppose, for some reason, you want to delete the second instance. What is going to happen to the third one? It will be recreated, because that’s how a list variable works. When you are removing the second instance (index 1), the third instance (index 2) will change its index to 1.

This is just a simple example, but how about a case in which you had 20 instances and for whatever reason you needed the remove the first one in the list? All the other 19 instances would be recreated and that downtime could really affect you.

By leveraging for_each you can avoid this pain, and due to the fact that it exposes an each.key and each.value when you are using maps, this will greatly help you achieve almost any architecture you desire.

locals {  
  vm_instances = {  
    vm1 = {  
      ami           = "ami-1"  
      az            = "eu-west-1a"  
      instance_type = "t2.micro"  
    }  
    vm2 = {  
      ami           = "ami-2"  
      az            = "eu-west-1b"  
      instance_type = "t3.micro"  
    }  
    vm3 = {  
      ami           = "ami-3"  
      az            = "eu-west-1c"  
      instance_type = "t3.small"  
    }  
  }  
}

resource "aws_instance" "this" {  
  for_each          = local.vm_instances  
  ami               = each.value.ami  
  availability_zone = each.value.az  
  instance_type     = each.value.instance_type  
}

The code is pretty similar but it is now using a map variable instead of a list one, and by removing an instance now, the others are not going to be affected in any way. In the above example, the keys are vm1, vm2 and vm3 and the values are the ones that are after the equal sign.

By using the for_each approach, you have the possibility of creating the resources in any way you want, reaching a more generic state and accommodating many use cases.

Keep in mind that both examples from above are not actual representation of modules, they are just used to point out the differences between for_each and count.

Use dynamic blocks

I remember when dynamic blocks weren’t a thing in Terraform, and I had to build a module for security lists in Oracle Cloud.

The problem was that security list rules were blocks inside of the security list resource itself, so whatever we would’ve done, that module, in the end, wasn’t very reusable.

One can argue that we could’ve prepared some jinja2 templates, had a script in place that would render those based on an input, but still, in my opinion, that isn’t a reusable terraform module.

Using dynamic blocks, you can easily repeat a block inside of a resource how many times you want based on the input variable and that most certainly was a game changer at the time of the release.

The good part is that you can have a dynamic block in place, even when you don’t want to create that block at all. In some architectures you will need a feature enabled many times, but in others you won’t need that feature at all. Why build two separate modules for that when you can have only one, right?

Use Ternary operators and Terraform functions

Terraform functions complicate the code. That is most certainly a fact and if somebody doesn’t have that much experience with Terraform they will have a hard time reading the code.

Nevertheless, when you are building reusable modules, you will most likely encounter the need of having some conditions inside of your code. You will see there are some arguments that cannot exist with other ones, so for your module to accommodate both use cases, you will most likely need to leverage a ternary operator with a lookup on the variable.

Build Outputs

Inside of a module, an output is literally what that module is exposing for other modules to use.

Make sure that whenever you are building a module, you are exposing the component that can be reused by other components (e.g. subnet ids, because they may be used by vms).

By using for_each, exposing an output will become harder, but don’t worry, you will get the hang of it pretty fast.

output "kube_params" {  
  value = { for kube in azurerm_kubernetes_cluster.this : kube.name => { "id" : kube.id, "fqdn" : kube.fqdn } }  
}

In the above example, I am exposing some azure kubernetes cluster outputs, that I consider relevant. Due to the fact that my resource has a for_each on it, I have to cycle through all of the resources of that kind to be able to access the arguments. I am creating a map variable with a name key that underneath will have an id and fqdn as part of its values.

Terraform documentation helps you a lot in knowing what a resource can export so you will only need to go the documentation page of that particular resource.

Using Pre-commit

Pre-commit can easily facilitate your terraform development and can make sure that your code respects the standard imposed by your team before you are pushing the code to the repository.

By using pre-commit, you can easily make sure that:

your terraform code is valid
linting was done properly
documentation for all your parameters has been written

There are other things that can be done, so you should check the documentation for the following:

In order to take advantage of pre-commit, you should define a file called .pre-commit-config.yaml inside of your repository with content similar to this:

repos:  
  - repo: https://github.com/pre-commit/pre-commit-hooks  
    rev: v4.3.0  
    hooks:  
      - id: end-of-file-fixer  
      - id: trailing-whitespace  
  - repo: https://github.com/antonbabenko/pre-commit-terraform  
    rev: v1.72.2  
    hooks:  
      - id: terraform_fmt  
      - id: terraform_tflint  
      - id: terraform_docs  
      - id: terraform_validate

Of course, many other hooks can be added, but in the example from above, we are using just the essentials.

You should have pre-commit installed locally, because as its name states, you should run it prior to making a commit. When you are doing so, all of the problems related to your code will be mentioned and some of them will even be fixed.

Bonus

I am a big fan of Github and Github Actions, so I like to add some simple pipelines to my modules. Also, as I don’t like to repeat myself and copy paste code all over the place, I like to use templates.

You can check out some of my repositories to find out how I am using those.

Building reusable Terraform Modules

Comments

More from this blog

Terraform Optional Object Type Attributes

🚀 Spacelift to the moon 🚀

[Github Actions] Pipelines for Terraform

Hashicorp Terraform Associate Certification Tips

Reasons to consider Terraform Cloud

Terraform module in its own repository

Use for_each and map variables

Use dynamic blocks

Use Ternary operators and Terraform functions

Build Outputs

Using Pre-commit

Bonus

Command Palette

Comments

More from this blog

Terraform module in its own repository

Use for_each and map variables

Use dynamic blocks

Use Ternary operators and Terraform functions

Build Outputs

Using Pre-commit

Bonus