Manage AWS Route 53 DNS records in bulk with Terraform

DNS records in AWS Route 53 does not support metadata, like tags or a description, to keep track of their use which is not always clear. I tried managing DNS records in bulk with Terraform variables and auto.tfvars files, but it quickly became messy. Terraform maps allow duplicate keys (but last definition wins) and the map is not (easily) sortable - so it’s hard to keep an overview.

DNS records for your apps and services are an integral part of your cloud infrastructure, and are typically deployed as part of the same infrastructure code. But you likely also have MX records for email routing, TXT for domain verification (e.g. for certificates) and other records. It’s nice to have a structured way to store this and infrastructure code to manage these.

Data format

Plain Terraform variables

Relying on plain Terraform variables, you might define a variable something like this (at least I did):

variable "route53_records" {
  type = list(object({
    name    = string
    type    = string
    ttl     = optional(number, 300)
    records = list(string)
    comment = string
  }))
  default = []
}

Records for setting up a custom domain for GitHub pages would look something like:

route53_records = [
  {
    name    = ""
    type    = "A"
    records = [
      "185.199.108.153",
      "185.199.109.153",
      "185.199.110.153",
      "185.199.111.153",
    ]
    comment = "GitHub Pages IP addresses"
  },
  {
    name    = "www"
    type    = "A"
    records = ["abstrask.github.io."]
    comment = "GitHub Pages 'www' subdomain"
  }

However, this approach has a major problem: It’s non-sortable so it doesn’t take many records before you lose overview. And as a consequence, it’s easy to add duplicate records my mistake.

Using YAML instead

rask.dk:
  /A:
    comment: GitHub Pages IP addresses
    values:
      - 185.199.108.153
      - 185.199.109.153
      - 185.199.110.153
      - 185.199.111.153
  www/CNAME:
    comment: GitHub Pages 'www' subdomain
    values:
      - abstrask.github.io.

This solves the previous problems: The YAML format requires keys to be unique, so your linter will detect conflicting records.

Even if you were to refactor the previous example to use a map with similar keys (e.g. www/CNAME), with HCL you get no indication of conflicting keys in a map - the last definition silently wins.

The YAML format allows records to easily be sorted by running:

yq -i -P 'sort_keys(..)' route53_records.yaml

This could also be automated through pre-commit hooks, or GitHub Action.

As a plus, the updated format also lets you managed records across multiple DNS zones (domains).

Terraform code

With Terraform you can define detailed data types for variables, ensuring input data complies with schema and any validation rules.

As our data is stored in YAML, we cannot directly use variables and catch malformed input this way. We can fix this by creating separate modules for reading and decoding the YAML file, and actually creating the records.

Route53 records module

First, we’ll create a module for managing the records, allowing us to define the requirements and constraints for each record through the records variable.

At least in AWS, DNS records don’t have tags, or descriptions, to remind you what they are there for or who requested it. Requiring a comment to be supplied, even though it’s not currently stored anyhwere, nudges you to specify some context to each record.

Currently this only validates that a valid record type was supplied, but can be expanded to all sorts of validation. Maybe you don’t want TTLs lower than 5 minutes? Maybe your DNS provider requires more (or less) information to be passed?

variable "records" {
  type = map(object({
    comment = string
    domain  = string
    name    = string
    ttl     = string
    type    = string
    values  = list(string)
    zone_id = string
  }))
  default = {}

  validation {
    condition     = alltrue([for _, v in var.records : contains(["A", "AAAA", "CAA", "CNAME", "DS", "HTTPS", "MX", "NAPTR", "NS", "PTR", "SOA", "SPF", "SRV", "SSHFP", "SVCB", "TLSA", "TXT"], v.type)])
    error_message = <<-EOF
    Valid record type values are A, AAAA, CAA, CNAME, DS, HTTPS, MX, NAPTR, NS, PTR, SOA, SPF, SRV, SSHFP, SVCB, TLSA, and TXT.
    The following record(s) specifies an invalid type:
    ${join(", ", compact([for k, v in var.records : !contains(["A", "AAAA", "CAA", "CNAME", "DS", "HTTPS", "MX", "NAPTR", "NS", "PTR", "SOA", "SPF", "SRV", "SSHFP", "SVCB", "TLSA", "TXT"], v.type) ? "${v.domain}\t${v.name}\t${v.type}\t${join(", ", v.values)}" : null]))}
    EOF
  }
}

resource "aws_route53_record" "this" {
  for_each = var.records
  name     = each.value.name
  records  = each.value.values
  ttl      = each.value.ttl
  type     = each.value.type
  zone_id  = each.value.zone_id
}

Root module

In the root module, we need to perform a few tasks, before we can call the sub-module.

# Read and decode the YAML file
locals {
  route53_records_raw = yamldecode(file("./route53_records.yaml"))
}

# Lookup up the ID of each Route53 zone we specify records for
data "aws_route53_zone" "this" {
  for_each = local.route53_records_raw
  name     = each.key
}

# Construct a map for all records, with unique keys and all the properties needed, for sub-module to create records
locals {
  route53_records = merge(flatten([for domain, records in local.route53_records_raw : {
    for id, props in records : "${domain}/${id}" => {
      comment = props.comment
      domain  = domain
      zone_id = data.aws_route53_zone.this[domain].id
      name    = split("/", id)[0]   # the key of each record consist of name and type separated by "/"
      ttl     = try(props.ttl, 300) # default to 5 minutes if not specified
      type    = split("/", id)[1]   # the key of each record consist of name and type separated by "/"
      values  = props.values
    }
  }])...)
}

module "route53_records" {
  source  = "./modules/route53-records"
  records = local.route53_records
}

Supporting other DNS providers

The YAML format and the local variable route53_records proposed here, is a pretty universal abstraction. This allows adaptation to other DNS providers with minor changes.

You could replace the aws_route53_zone data source in the root module with the equivalent in the Terraform provider for your DNS host, and likewise for the aws_route53_record resource in the “records” module.