Automating Migration AWS EC2 Instance Metadata Service (IMDSv2) using Ansible

Published in

Appsecco

5 min readDec 31, 2019

TL;DR If you are interested in getting started about this feature, head to our previous blog post — Getting started with Version 2 of AWS EC2 Instance Metadata Service (IMDSv2)

This post is a continuation to the one about AWS EC2 instance metadata service (IMDSv2), how to get started, how to enable, monitor and disable IMDSv2 in your EC2 instances. In this post you will learn how to automate the migration to IMDSv2 for a large scale of EC2 instances.

EC2 configuration automation using Ansible

How to migrate a bulk of EC2 instances to AWS EC2 Instance Metadata Service(IMDSv2)

In order to test it out, let’s create four instances — two each in ap-south-1 and us-east-1 regions. All the four instances by default use IMDSv1.

Instances in us-east-1 — North Virginia Region

At the end of the post, our goal is to migrate all these instances to IMDSv2. We plan to use Ansible playbook to achieve this goal.

Ansible

Ansible an open source tool owned by Redhat now is what is commonly known as a software provisioning and automation tool. The thing that we love about Ansible is the fact that it is agentless which means no new ports to secure. Works over SSH for linux hosts.

Prerequisites

awscli (version >= 1.16.287 with Python/3.6.8)
ansible (version >= 2.9 — ec2_instance)

We have created instances in only two regions, but if a company has like instances in all of the regions, they can define the regions as variable in Ansible like below,

regions: ["eu-north-1", "ap-south-1", "eu-west-3", "eu-west-2", "eu-west-1", "ap-northeast-2", "ap-northeast-1", "sa-east-1", "ca-central-1", "ap-southeast-1", "ap-southeast-2", "eu-central-1", "us-east-1", "us-east-2", "us-west-1", "us-west-2"]

The list of available regions can be got from a simple AWS CLI call like below,

aws ec2 describe-regions --output text | cut -f4

Once we are set with the regions, we need to know the instances that are present in all the regions, in our case we need to know all the four instances that are created. We can use a default EC2 module with Ansible — ec2_instance_info for getting this information. We iterate this module for all regions and store the result in a variable instance_info.

- name: get instance info
  local_action:
    module: ec2_instance_info
    region: "{{ item }}"
 
  register: instance_info
  with_items:
    - "{{ regions }}"

Now we do not really need all the information about instances in all regions, we would just need the instance_id's belonging to each region. So we will query instance_id from the result and also iterate with the regions list that we have. As a result we get instance_id's for each region which we can use for running migrate command.

- name: Enable IMDSv2 for instances in each region
    command: "sh {{ shell_file }}  {{ item | join(' ') }}"

    with_together :
        - "{{ regions }}"
        - "{{ instance_info | json_query('results[*].instances[*].instance_id') }}"

At the time of writing this post, option to modify-instance-metadata-options was not available in the AWS module for Ansible. So we chose to quickly write a shell script that internally uses aws-cli for modifying metadata options.

#!/bin/bash

export AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY>

region=$1
# We remove region from the arguments list for iterating only the instances
shift
for instance_id in "$@"
do
    result=`aws ec2 modify-instance-metadata-options --region ${region} --instance-id ${instance_id} --http-endpoint enabled --http-token required`
    echo $result
done

<YOUR_AWS_ACCESS_KEY_ID> and <YOUR_AWS_SECRET_ACCESS_KEY> should be substituted with an IAM role that has EC2FullAccess permission to modify instance metadata

Our script gets called like below by the task in our Ansible playbook,

sh modify_instance_metadata_options.sh <region> <instance_id_1> <instance_id_2> <instance_id_n>

Below is the complete code of our playbook (Github Gist),

---
  - name: To enable IMDSv2 in AWS EC2 instances
    hosts: localhost
    vars:
      regions: ["eu-north-1", "ap-south-1", "eu-west-3", "eu-west-2", "eu-west-1", "ap-northeast-2", "ap-northeast-1", "sa-east-1", "ca-central-1", "ap-southeast-1", "ap-southeast-2", "eu-central-1", "us-east-1", "us-east-2", "us-west-1", "us-west-2"]
      shell_file: "modify_instance_metadata_options.sh"
    tasks:

      - name: get instance info
        local_action:
          module: ec2_instance_info
          region: "{{ item }}"

        register: instance_info
        with_items:
          - "{{ regions }}"

      - name: Enable IMDSv2 for instances in each region
        command: "sh {{ shell_file }}  {{ item | join(' ') }}"

        with_together :
          - "{{ regions }}"
          - "{{ instance_info | json_query('results[*].instances[*].instance_id') }}"

Note: We assume the shell file is in home directory and if you would like to run in another ansible node, you should configure it as needed. For running in your local machine, you will have to add your SSH public key to the authorized_keys file and add your private key with ssh-add id_rsa before running the playbook.

Video Demo on YouTube

Video demo of automating IMDSv2 migration

This method also works for stopped EC2 instances, so that we need not worry about starting an instance at a later point and do migration for that separately.

However, this is only for currently present instances. This does not prevent use of IMDSv1 for instances that are created newly. AWS has introduced a way to enforce the use of IMDSv2 on all new instances using IAM — you can read about it here.

References

At Appsecco we provide advice, testing and training around software, infra, web and mobile apps, especially that are cloud hosted. We also specialise in auditing AWS environments as per the AWS CIS Foundations Benchmark to create a picture of the current state of security in your AWS environment. Our experience has led us to creating multiple hands on training courses like the very popular “Breaking and Pwning Apps and Servers on AWS and Azure” and “Automated Defence using Cloud Services for AWS, Azure and GCP”.

Drop us an email, contact@appsecco.com if you would like us to assess the security of your AWS infrastructure or if you would like your security team trained in advanced pentesting techniques against AWS.