Overview

Tagging can be a powerful tool in Azure. You can use it to organize and categorize your resources making them easier to find. Tags can also help with other things like automating processes like locking virtual networks or destroying test resources which can protect and reduce the cost of your Azure environment.

Tagging can be an important asset for large companies with cloud platforms that service multiple internal business streams and external clients all around the world. I would recommend that you take some time and plan out your tagging strategy. Think through what information you want to capture for each resource and how that data will be used.

Here are some common use cases for tagging:

• Logically organizing resources into a taxonomy
• Automating operations
• Reducing costs
• Maintaining compliance
• Managing cloud usage costs

In order to create a consistent metadata model using tag key value pairs, you need a way of ensuring your tagging is consistent. This article will provide details on how you can use Terraform to:

• Create policy definitions
• Create a policy initiative
• Assign a policy to a scope with an identity
• Create a runbook for maintaining compliance at the resource level based on the deployed policy

Here is a high-level overview of what we want to achieve. Terraform will be used to create resources that will:

• Enforce tags be added to resource groups and resources
• Validate tags that are added against a set of pre-defined values
• Inherit resource tags from resource groups

And a runbook that will allow you to:

• Trigger a remediation task on the initiative to keep your resource tagging evergreen

Here is the general flow:

Creating Policies

Azure policies help technology teams to enforce organizational standards and compliance requirements at scale. The policies we are creating will help us to enforce, validate, and inherit tags.

• Tag enforcement ensures that any required tag keys are included as part of a resource or resource group.
• Tag validation checks that a given value for a tag key is appropriate by checking against a pre-defined list of valid values.
• Tag inheritance ensures that all resources are tagged. It also makes reduces burden on team members deploying new resources in an existing resource group.

Tag enforcement and validation helps to ensure we have a consistent metadata model while tag inheritance helps to ensure we have a complete metadata model.

In this example, we will be creating policies that enforce a single tag called “Environment” with valid values of “Development”, “QA”, “UAT”, and “Production”.

You can find the full code base on my GitHub at https://github.com/Jsoconno/terraform-azure-tagging-standards/tree/main/tagging-policy.

Prerequisites

The Policy deployment will require that you have the appropriate user permissions to manage policies and assignments at the tenant root of your Azure tenant or that you have an app registration with these permissions with a generated client secret. You will also need to have created the initial Tenant Root Group management group.

Tag Enforcement Policies

The first policies we want to make are policies that enforce a certain list of tags to be added to resource groups and resources.

Here is the Terraform code for resource group tag enforcement:

resource "azurerm_policy_definition" "enforce_resource_group_tags" {
    name         = "apd_enforce_rg_tags"  
    management_group_name = data.azurerm_management_group.tenant_root.name
    policy_type  = "Custom"
    mode         = "All"
    display_name = "Enforce Resource Group Tags"

    lifecycle {
        ignore_changes = [
            metadata
        ]
    }

    metadata = <<METADATA
    {
        "category": "Tags",
        "createdBy": "",
        "createdOn": "",
        "updatedBy": "",
        "updatedOn": ""    
    }
    METADATA

    parameters = <<PARAMETERS
    {
        "tagName": {
            "type": "String",
            "metadata": {
                "displayName": "Tag Name",
                "description": "Name of the tag, such as costCenter"
            }
        },
        "policy-effect": {
        "type": "String",
        "metadata": {
            "displayName": "Policy Effect",
            "description": "The available options for the Policy Effect"
        },
        "allowedValues": [
            "audit",
            "deny"
        ],
        "defaultValue": "audit"
        }
    }
    PARAMETERS

    policy_rule = <<POLICY_RULE
    {
        "if": {
            "allOf": [
                {
                    "field": "type",
                    "equals": "Microsoft.Resources/subscriptions/resourceGroups"
                },
                {
                    "field": "[concat('tags[', parameters('tagName'), ']')]",
                    "exists": "false"
                }
            ]
        },
        "then": {
            "effect": "[parameters('policy-effect')]"
        }
    }
    POLICY_RULE
}

Here is the Terraform code for resource tag enforcement:

resource "azurerm_policy_definition" "enforce_resource_tags" {
    name         = "apd_enforce_r_tags"
    management_group_name = data.azurerm_management_group.tenant_root.name
    policy_type  = "Custom"
    mode         = "Indexed"
    display_name = "Enforce Resource Tags"
    description  = "Policy to enforce that a specific tag exists for a resource.  Excludes metric alerts."

    lifecycle {
        ignore_changes = [
            metadata
        ]
    }

    metadata = <<METADATA
    {
        "category": "Tags",
        "createdBy": "",
        "createdOn": "",
        "updatedBy": "",
        "updatedOn": ""    
    }
    METADATA

    parameters = <<PARAMETERS
    {
        "tagName": {
            "type": "String",
            "metadata": {
                "displayName": "Tag Name",
                "description": "Name of the tag, such as 'environment'"
            }
        },
        "policy-effect": {
            "type": "String",
            "metadata": {
                "displayName": "Policy Effect",
                "description": "The available options for the Policy Effect"
            },
            "allowedValues": [
                "audit",
                "deny"
            ],
            "defaultValue": "audit"
        }
    }
    PARAMETERS

    policy_rule = <<POLICY_RULE
    {
        "if": {
            "allOf": [
                {
                    "field": "[concat('tags[', parameters('tagName'), ']')]",
                    "exists": "false"
                }
            ]
        },
        "then": {
            "effect": "[parameters('policy-effect')]"
        }
    }
    POLICY_RULE
}

Tag Validation Policies

Now that we can enforce tags, we want to be able to validate those tags. You will notice in the code that there are some exclusions. That is because, at the time of writing this, those resources cannot be tagged in Azure and result in inaccurate tagging results.

Here is the Terraform code for resource group tag validation:

resource "azurerm_policy_definition" "validate_resource_group_tags" {
    name         = "apd_validate_rg_tags"
    management_group_name = data.azurerm_management_group.tenant_root.name
    policy_type  = "Custom"
    mode         = "All"
    display_name = "Validate Resource Group Tags"
    description  = "Policy to validate the value supplied for a tag key based on a parameterized array for a resource group."

    lifecycle {
        ignore_changes = [
            metadata
        ]
    }

    metadata = <<METADATA
    {
        "category": "Tags",
        "createdBy": "",
        "createdOn": "",
        "updatedBy": "",
        "updatedOn": ""    
    }
    METADATA

    parameters = <<PARAMETERS
    {
        "tagName": {
        "type": "String",
        "metadata": {
            "displayName": "Tag Name",
            "description": "Name of the tag, such as 'environment'"
        }
        },
        "policy-effect": {
            "type": "String",
            "metadata": {
                "displayName": "Policy Effect",
                "description": "The available options for the Policy Effect"
        },
        "allowedValues": [
            "audit",
            "deny"
        ],
        "defaultValue": "audit"
        },
        "options": {
            "type": "Array",
            "metadata": {
                "displayName": "Options",
                "description": "List of available options for validation."
            }
        }
    }
    PARAMETERS

    policy_rule = <<POLICY_RULE
    {
        "if": {
            "allOf": [
                {
                    "not": {
                        "field": "[concat('tags[', parameters('tagName'), ']')]",
                        "in": "[parameters('options')]"
                    }
                },
                {
                    "field": "type",
                    "equals": "Microsoft.Resources/subscriptions/resourceGroups"
                }
            ]
            },
        "then": {
            "effect": "[parameters('policy-effect')]"
        }
    }
    POLICY_RULE
}

Here is the Terraform code for resource tag validation:

resource "azurerm_policy_definition" "validate_resource_tags" {
    name         = "apd_validate_r_tags"
    management_group_name = data.azurerm_management_group.tenant_root.name
    policy_type  = "Custom"
    mode         = "Indexed"
    display_name = "Validate Resource Tags"
    description  = "Policy to validate the value supplied for a tag key based on a parameterized array for a resource."

    lifecycle {
            ignore_changes = [
                metadata
        ]
    }

    metadata = <<METADATA
    {
        "category": "Tags",
        "createdBy": "",
        "createdOn": "",
        "updatedBy": "",
        "updatedOn": ""    
    }
    METADATA

    parameters = <<PARAMETERS
    {
        "tagName":{
            "type":"String",
            "metadata":{
                "displayName":"Tag Name",
                "description":"Name of the tag, such as 'environment'"
            }
        },
        "policy-effect":{
            "type":"String",
            "metadata":{
                "displayName":"Policy Effect",
                "description":"The available options for the Policy Effect"
            },
            "allowedValues":[
                "audit",
                "deny"
            ],
            "defaultValue":"audit"
        },
        "options":{
            "type":"Array",
            "metadata":{
                "displayName":"Options",
                "description":"List of available options for validation."
            }
        }
    }
    PARAMETERS

    policy_rule = <<POLICY_RULE
    {
        "if": {
            "allOf": [
                {
                    "not": {
                        "field": "[concat('tags[', parameters('tagName'), ']')]",
                        "in": "[parameters('options')]"
                    }
                }
            ]
        },
        "then": {
            "effect": "[parameters('policy-effect')]"
        }
    }
    POLICY_RULE
}

Tag Inheritance Policy

Now that we can enforce and validate tags at the resource group and resource levels, we want create a policy that will serve as a mechanism for appending tags from a resource group to a resource deployed in that resource group if it does not have the required tags.

Here is the Terraform code for resource tag inheritance from the parent resource group:

resource "azurerm_policy_definition" "append_resource_group_tags" {
    name         = "apd_append_rg_tags"  
    management_group_name = data.azurerm_management_group.tenant_root.name
    policy_type  = "Custom"
    mode         = "Indexed"
    display_name = "Append Resource Group Tags"

    lifecycle {
        ignore_changes = [
            metadata
        ]
    }

    metadata = <<METADATA
    {
        "category": "Tags",
        "createdBy": "",
        "createdOn": "",
        "updatedBy": "",
        "updatedOn": ""    
    }
    METADATA

    parameters = <<PARAMETERS
    {
        "tagName": {
            "type": "String",
            "metadata": {
                "displayName": "Tag Name",
                "description": "Name of the tag, such as costCenter"
            }
        }
    }
    PARAMETERS

    # By using the modify effect, we are able to create a remediation task for this policy
    # An Azure Automation Runbook can be used to accomplish this on a regular basis so tags are evergreen
    policy_rule = <<POLICY_RULE
    {
        "if": {
            "allOf": [
                {
                    "field": "[concat('tags[', parameters('tagName'), ']')]",
                    "exists": "false"
                },
                {
                    "value": "[resourceGroup().tags[parameters('tagName')]]",
                    "exists": "true"
                },
                {
                    "value": "[resourceGroup().tags[parameters('tagName')]]",
                    "notEquals": ""
                }
            ]
        },
        "then": {
            "effect": "modify",
            "details": {
                "roleDefinitionIds": [
                    "/providers/microsoft.authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
                ],
                "operations": [
                    {
                        "operation": "add",
                        "field": "[concat('tags[', parameters('tagName'), ']')]",
                        "value": "[resourceGroup().tags[parameters('tagName')]]"
                    }
                ]
            }
        }
    }
  POLICY_RULE
}

Creating a Policy Initiative

Azure Policy Initiatives, also known as Policy Sets, are a collection of policy definitions that help you to achieve an overall goal. In this case, our overall goal is to ensure our tagging standards are met. In particular, that we are prescribe that an “Environment” tag be added to all resources and that a particular set of values be required.

Here is the Terraform code to create a policy set that can be used to manage all of our policy definitions together in a single initiative:

locals {
    environment_tags = [
        "Development",
        "QA",
        "UAT",
        "Production"
    ]
}

resource "azurerm_policy_set_definition" "tagging_standards" {
    name                  = "api_tagging_standards"
    policy_type           = "Custom"
    display_name          = "Tagging Standards"
    management_group_name = data.azurerm_management_group.tenant_root.name
    description           = "Tagging Standards to be applied to the Azure environment."

    lifecycle {
        ignore_changes = [
            metadata
        ]
    }

    parameters = <<PARAMETERS
    {
    "policy-effect":
        {
            "allowedValues":["audit","deny"],
            "metadata":{
                "description":"The effect options for the initiative.",
                "displayName":"policy-effect"
            },
            "type":"String",
            "defaultValue": "audit"
        }
    }
    PARAMETERS

    # Enforces the Environment tag on any resource group that is created.
    policy_definition_reference {
        policy_definition_id = azurerm_policy_definition.enforce_resource_group_tags.id
        parameter_values = jsonencode({
            policy-effect = {value = "[parameters('policy-effect')]"},
            tagName = {value = "Environment"}
        })
    }

    # Validates the Environment tag on any resource group that is created.
    policy_definition_reference {
        policy_definition_id = azurerm_policy_definition.validate_resource_group_tags.id
        parameter_values = jsonencode({
        policy-effect = {value = "[parameters('policy-effect')]"},
            options = {value = local.environment_tags},
            tagName = {value = "Environment"}
        })
    }

    # Appends the Environment tag on a resource based on the parent resource group if one is not provided.
    policy_definition_reference {
        policy_definition_id = azurerm_policy_definition.append_resource_group_tags.id
        parameter_values = jsonencode({
            tagName = {value = "Environment"}
        })
    }

    # Enforces the Environment tag on any resource that is created.
    policy_definition_reference {
        policy_definition_id = azurerm_policy_definition.enforce_resource_tags.id
        parameter_values = jsonencode({
            policy-effect = {value = "[parameters('policy-effect')]"},
            tagName = {value = "Environment"}
        })
    }

    # Validates the Environment tag on any resource that is created.
    policy_definition_reference {
        policy_definition_id = azurerm_policy_definition.validate_resource_tags.id
        parameter_values = jsonencode({
            policy-effect = {value = "[parameters('policy-effect')]"},
            options = {value = local.environment_tags},
            tagName = {value = "Environment"}
        })
    }
}

Creating A Policy Assignment

In order for the policy initiative to take effect, we need to assign it to a scope. This can be a management group, subscription, resource group, or a specific resource. In this case, we will want to assign this to our root management group so all resources are covered.

Here is the Terraform code for the policy assignment:

resource "azurerm_policy_assignment" "apa_tagging_standards" {
    name                 = "apa_tagging_standards"
    scope                = data.azurerm_management_group.tenant_root.id
    policy_definition_id = azurerm_policy_set_definition.tagging_standards.id
    description          = ""
    display_name         = "Example Tagging Standards"
    location             = "eastus"

    parameters = <<PARAMETERS
        {
            "policy-effect": {
                "value": "deny"
            }
        }
    PARAMETERS

    identity {
        type          = "SystemAssigned"
    }
}

resource "azurerm_role_assignment" "apa_tagging_standards" {
    scope                 = "/providers/Microsoft.Management/managementGroups/${data.azurerm_management_group.tenant_root.name}"
    role_definition_name  = "Contributor"
    principal_id          = azurerm_policy_assignment.apa_tagging_standards.identity[0].principal_id
}

Creating Runbook

Azure Automation Runbooks provide a means for automating and scheduling activities in Azure. In this case, we are going to create a runbook that can trigger the remediation task in our append resource group tags policy and have it run every four hours. This will ensure that any resources created for any reason that are not tagged in Azure do have the tags of the parent resource group.

You can find the full code base on my GitHub at https://github.com/Jsoconno/terraform-azure-tagging-standards/tree/main/tagging-runbook.

Below is the PowerShell script we will use to remediate tags:

##
# tagging-remediation.ps1
#
# Description: Used to give child resource the tags required from the parent resouce group
##

# Login to Azure
$connectionName = "AzureRunAsConnection"
try
{
    # Get the connection "AzureRunAsConnection"
    $servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
 
    "Logging in to Azure..."
    Add-AzAccount `
        -ServicePrincipal `
        -TenantId $servicePrincipalConnection.TenantId `
        -ApplicationId $servicePrincipalConnection.ApplicationId `
        -CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
        # Add the below if you are using Azure Government to ensure authentication works as expected
        # -Environment "AzureUSGovernment"
}
catch
{
    if (!$servicePrincipalConnection)
    {
        $ErrorMessage = "Connection $connectionName not found."
        throw $ErrorMessage
    }
    else
    {
        Write-Error -Message $_.Exception
        throw $_.Exception
    }
}

$policySetName = "api_tagging_standards"
$policyDefinitionName = "apd_append_rg_tags"
$policyAssignmentScope = Get-AzManagementGroup | Where {$_.DisplayName -eq "Tenant Root Group"}
$policyDefinitionId = "$($policyAssignmentScope.Id)/providers/Microsoft.Authorization/policyDefinitions/$($policyDefinitionName)"

# Fetch Policy and Assignment Information
try
{
    # Get the policy set definition based on name
    $policySet = Get-AzPolicySetDefinition | where { $_.Name -eq $policySetName }

    # Get the policy definition that is responsible for appending tags from the resource group level
    $policies = $policySet.Properties.PolicyDefinitions | where { $_.policyDefinitionId -eq $policyDefinitionId }

    # Get the assigments for the policy definition
    $assignment = Get-AzPolicyAssignment -Scope $policyAssignmentScope.Id | where { $_.Properties.PolicyDefinitionId -eq $($policySet.PolicySetDefinitionId)}

}
catch
{
    Write-Error -Message $_.Exception
    throw $_.Exception
}

# Create Remediation Task
try
{
    foreach ( $policy in $policies ) {
        $job = Start-AzPolicyRemediation -PolicyAssignmentId $($assignment.PolicyAssignmentId) -PolicyDefinitionReferenceId $($policy.policyDefinitionReferenceId) -Name "remediate-$($policy.parameters.tagName.value)-tag" -ManagementGroupName $policyAssignmentScope.Name -AsJob
        $job | Wait-Job
        $remediation = $job | Receive-Job
        Write-Output "$($remediation.Name):"
        Write-Output $($remediation.DeploymentSummary)
        Start-Sleep -Seconds 60
    }
}
catch
{
    Write-Error -Message $_.Exception
    throw $_.Exception
}

Now that we have the script, we need to create a runbook that uses it. Here is the Terraform code for the runbook:

resource "azurerm_automation_runbook" "tagging_remediation" {
  name                    = "TaggingRemediation"
  location                = data.azurerm_resource_group.main.location
  resource_group_name     = data.azurerm_resource_group.main.name
  automation_account_name = data.azurerm_automation_account.main.name
  log_verbose             = "true"
  log_progress            = "true"
  description             = "Used to give child resource the tags required from the parent resource group"
  runbook_type            = "PowerShell"

  # must have a content link, but will use custom content from local file
  publish_content_link {
    uri = "https://www.microsoft.com/en-us/"
  }

  content = data.local_file.tagging_remediation.content
}

And here is the code to put the runbook on a four-hour schedule:

resource "azurerm_automation_schedule" "every_four_hours" {
  name                    = "Every Four Hours"
  resource_group_name     = data.azurerm_resource_group.main.name
  automation_account_name = data.azurerm_automation_account.main.name
  frequency               = "Hour"
  interval                = 4
  timezone                = "America/New_York"
  description             = "Runs Every 4 Hours"
}

resource "azurerm_automation_job_schedule" "tagging_remediation" {
  resource_group_name     = data.azurerm_resource_group.main.name
  automation_account_name = data.azurerm_automation_account.main.name
  schedule_name           = "Every Four Hours"
  runbook_name            = azurerm_automation_runbook.tagging_remediation.name
}

In order for the Run As Account to have the permissions it needs, we also have to make it Contributor at the management group scope:

resource "azurerm_role_assignment" "automation_run_as_account" {
  scope                = data.azurerm_management_group.tenant_root.id
  role_definition_name = "Contributor"
  principal_id         = data.azuread_service_principal.automation_run_as_account.object_id
}

Conclusion

Be above code can serve as the baseline for building a flexible, reliable, complete, and consistent tagging strategy in Azure. Hopefully you find it helpful as you explore ways to build a robust tagging system in your own Azure environments.

For the full code including the variables files, data sources, and other items, check out the repository on GitHub:

https://github.com/Jsoconno/terraform-azure-tagging-standards