CloudFormation Template: Tag AWS Volumes for Lifecycle Manager Backups

Feb 12, 2019- aws cloudformation

If you wan’t a simple AWS Backup solution you can use AWS Lifecycle Manager to create snapshots from your AWS EC2 volumes.

Lifecycle Manager is easy to use and even gives you some retention rules, no scripting needed for your Backups at all.

You can easily define which target volumes Lifecycle Manager should snapshot through tags on your volumes.

Lifecycle Manager - Snapshot Lifecycle Policy

In following example we will take snapshots all 24h of all volumes which are tagged backupid: AUT01 between 09 and 10 UTC and will retain 7 snapshots.

Usually we use CloudFormation to create our AWS environments and our EC2 instances. Unfortunately the tags you use for your EC2 instances are not automatically added to the according volumes of your instance. Bummer!

This means we have to find a way to tag the instance volumes right after creation and of course easiest way to do this is using some magic in a User data script.

Needed User data script

Following script may be used as User data script:

aws ec2 create-tags --resources $(aws ec2 describe-volumes --filters Name=attachment.instance-id,Values=$(curl -s http://169.254.169.254/latest/meta-data/instance-id) --query 'Volumes[*].[VolumeId]' --region=eu-central-1 --out text | cut -f 1) --tags Key=$Key,Value=$Value --region eu-central-1

There are two parts in this script:

Getting the VolumeIds of the volumes with the help of the local server metadata

aws ec2 describe-volumes --filters Name=attachment.instance-id,Values=$(curl -s http://169.254.169.254/latest/meta-data/instance-id) --query 'Volumes[*].[VolumeId]' --region=eu-central-1 --out text

Tip	You can add a filter for device names as well, i.e. you only want to tag/backup your data volume which is mounted as /dev/xvdb then add following right before the "--query" statement

Name=attachment.device,Values=/dev/xvdb

Tag these Volumes with the provided key and value

aws ec2 create-tags --resources VOLUMEIDS --tags Key=$Key,Value=$Value --region eu-central-1

As you can see we are using an EC2 instance in the eu-central-1 region, you have to change this to the region you are using.

The EC2 instance needs an IAM role with sufficient rights to get the volume id’s and to tag the volumes. We will add following policy to this role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "ec2:Describe*",
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": "ec2:CreateTags",
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

Lifecycle Policy

Final step is to add the Snapshot Lifecycle Policy with the needed parameters (TargetTags…)

  BasicLifecyclePolicy:
    Type: "AWS::DLM::LifecyclePolicy"
    Properties:
      Description: "Lifecycle Policy using CloudFormation"
      State: "ENABLED"
      ExecutionRoleArn: !GetAtt
        - lifecycleRole
        - Arn
      PolicyDetails:
        ResourceTypes:
          - "VOLUME"
        TargetTags:
          -
            Key: "backupid"
            Value: "AUT01"
        Schedules:
          -
            Name: "Daily Snapshots"
            TagsToAdd:
              -
                Key: "type"
                Value: "DailySnapshot"
            CreateRule:
              Interval: 24
              IntervalUnit: "HOURS"
              Times:
                - "09:00"
            RetainRule:
              Count: 7
            CopyTags: true

As you can see an execution role is needed as well (with proper policy attached). You will find this role and all additional needed resources in the full CloudFormation template on Github.

Feedback is always welcome!