Dynamic DNS hostnames with Route53 and EC2
I keep a lot of one-off EC2 instances for specialty tasks that I do once in a while. I have a FreeBSD instance, a phpMyAdmin instance, a forensic analysis instance, and so on. It's a good practice with AWS EC2 to stop these instances when they aren't needed. It means that I don't pay for instance hours that I'm not using. I start them, do what I need to do, and then stop them when I'm done. I assign public IP addresses to these instances so that I can reach them, but the instance gets a new IP public address each time it starts up. Rather than looking up the new public IP address each time the instance starts, I want to use a DNS name to reach it. I need an A record in DNS to be updated each time the instance boots, so that I can reach it by name, instead of looking up its new public IP address every time I start it.
In this blog post I provide a script that runs at boot time to update DNS each time an instance boots with a public IP. The script figures out the DNS record that needs to be created by looking at the instance metadata, and then it updates records in the right zone in Route53. Because the script pulls all the important information from the instance metadata, there's no unique code or configuration that has to be installed on the instance. I just install this one script and it will figure out the rest from EC2 metadata. If you launch an instance from an AMI, and you tag that instance correctly and you put the instance into the right role, this will work without any changes specific to the individual instance itself.
To make this work, you need a few things before you start. I am not going to describe how to do them.
- You need a public hosted zone in Route53
- You need python and boto3 installed in your instance
- You need to create your own AMI, or your own way of customising instances after launch.
Public hosted zones are described in the Route53 documentation, and installing python and boto3 are described in the boto3 documentation.
The process looks like this.
- Create a dynamic subzone for these host names
- Create a policy that allows editing the subzone
- Create a role and attach the policy to it (or attach the policy to an existing role)
- Load the script onto the instance and enable it at boot time
- Launch an instance with the role and correct tags
Create a Hosted DNS Zone
In the interest of security, we want a dedicated zone for these dynamic host names. We don't want to grant EC2 instances
the privilege of writing into our top-level zone. For example, if your organisation is “example.com”, you don't want EC2
instances launching that have rights to modify DNS records in the example.com domain. In this example, I'll use a
subzone of example.com,
ec2.example.com, for all my dynamic host names. You can imagine uses like
sandbox.example.com, and so on. We can grant permissions to modify one specific hosted domain without
worrying about unauthorised changes to
example.com itself. This also works if
example.com is not running in Route53, but
ec2.example.com to be dynamic and hosted in Route53. It requires cooperation with the
example.com domain owner,
but just once.
Create the Subzone
In the AWS Console, create a zone called ec2.example.com. This is not a record set in example.com whose name is “ec2” (we'll create one of those in a second). First you create a new hosted zone.
Capture Some Details
When you create the hosted zone, you immediately see some NS records. It's a list of DNS server names. Copy these NS records. You will also see a Hosted Zone ID. It looks like this: ZZ123ABCD123. Make note of that ID for future use.
Create the Glue Records
Go to your example.com hosted zone and create a record set.
- For the name, choose whatever subzone you're creating. In my case, I type
ec2so that I am creating a record set
- For the record set type, choose “NS – Name Server”.
- In the Value box, paste the name servers that you just copied. This is called creating a “glue record” in DNS. It indicates that
ec2.example.comis a separate DNS zone (as opposed to an individual host) and it designates the name servers who can answer queries about that zone.
Create an IAM Policy to Allow Changing the Zone
Before our host can dynamically update a zone, we have to grant it permissions. It is important to grant the instance just enough privileges to do this, and nothing more. It's also poor practice to embed AWS IAM access keys or secrets in code. To avoid coding secrets, we are going to use an EC2 Instance Role. That role will need a policy, so that's why we create the policy first.
To create a policy¹ that allows an entity to edit records in your
ec2.example.com subzone, go to IAM and choose Create
Policy. Choose “Create Your Own Policy”. Give it a descriptive title. I chose r53-edit-ec2-example-com to indicate that
it allows editing the
ec2.example.com zone in Route53. Enter the following policy. Note that where it says [ZONEID]
that's where you put enter the subzone Hosted Zone ID that you recorded earlier.
This policy is in the dns-update-policy.json file here in the repository. It grants the minimum privileges needed by the script. If any of these are restricted, the script will not be able to do its work.
¹ This part of the solution was inspired by this blog post at cantina.co.
Attaching the Policy to an EC2 Instance Role
If you already have a role that you apply to your EC2 Instances, you don't need to create another role. Skip this next step. If you don't have an instance role, then you should create one so that you can attach policies to it.
Create the Role
Go to IAM and create a new role. Give it a descriptive name. I chose
dynamic-ec2 to express the idea that these
instances are dynamic and in my ec2 DNS zone. Under “AWS Service Roles” click “Select” next to “Amazon EC2”. From the
policy list, find the
r53-edit-ec2-example-com policy and attach it to the role you're creating. Click “Create Role” and
the role will be created.
Attach the Policy
If you already have an instance role that your instances use, find that role and edit it. Attach this policy to that instance role. Now those instances will also have the privilege to edit this Route53 zone, in addition to whatever privileges their existing role grants them.
Launch Instances with the Role
Note that you can't assign a role to an existing instance; you can only specify a role when you launch a new instance. Be sure to specify add this role in Step 3: Configure Instance Details.
From a command line or launching instances via API calls, it is a two-step process. First you create an instance profile for your instances. Then you attach the role to the instance profile. Finally, you launch instances with that instance profile.
See the documentation on Working with Roles for information about attaching roles to instance profiles. For more information about instance profiles, see the documentation About Instance Profiles.
Create The Tag
You need a tag called
dnsname on the instance. It should be the full hostname, including domain, that you want for the
instance. If you set dnsname to be
webdev1.ec2.example.com , then this script will attempt to create an A record for
webdev1.ec2.example.com when it runs. It will look for
ec2.example.com in the hosted zones of Route53, and then will
try to create an A record into that zone.
Install the Script
You need to install this script and make it run at boot time somehow. This might mean creating a script in
a Linux instance. In my case, I just saved the script in
/home/ec2-user/auto-update-r53.py and then I added this line to
The script uses an “UPSERT” call to Route53. That is, if the A record already exists, it is updated. If the A record does not exist, it is inserted. Update + insert = UPSERT.
The script does a few things:
- It reads the dnsname tag from the instance metadata.
- It reads the public IP address from the instance metadata.
- It searches for the domain in the list of hosted zones at Route53.
- It sends an UPSERT request to create the record in the zone.
If all goes well, it will print a single line of output like this, which will be visible in the instance log.
setting webdev1.ec2.example.com to 184.108.40.206
All hosts that have this privilege can create, modify, and change any record in the zone. Nothing restricts these hosts from modifying important records like NS records, CNAMES, or any other legitimate record in the zone. This is probably appropriate for a sandbox or dev environment and not suitable for a production environment.
Domain Names and the Same Origin Policy
It is dangerous to allow development code to run using an important top level domain. If www.example.com is the web address for an online retail web site, it would be a bad idea to allow developers to create hosts with names like www.ec2.example.com. Various web browser security mechanisms (e.g., the Same Origin Policy) will tend to trust subzones the same as the root zone. Web browsers with cookies for production use at www.example.com would very likely send those same cookies to www.ec2.example.com (depending on how the application is written). And this could expose security tokens from production to developer web applications who should not see them.
A simple solution to this trust problem is to buy a separate domain that is never used in production and use it for these dynamic hosts. If www.example.com is our online commerce site, we could use one of the other top level domains for our development environments. We could register something like example.tech or example.build for our development work, and then use webdev1.ec2.example.tech as our dynamic host name, while www.example.com remains the name of the production web site.
Policy and Role Complexity
This example used one policy per zone. If you have hundreds of zones that you manage this way, that approach might become cumbersome with so many policies. Instead of having individual policies that control access to a single domain, you could potentially create a single policy that grants privileges to lots of domains (e.g., all the development and test subzones). This will make it a bit easier to manage, but it also makes the security less fine-grained. The right balance depends on the number of domains you manage and how complex the security roles are in your operations teams.
This technique can be extended deeper to subzones. If we want to have additional zones that indicate AZ or region, we
can. This could easily be extended to create hostnames like
webdev1.az1a.eu-west-1.ec2.example.com. It requires creating
additional hosted zones and glue records. It also requires creating more IAM policies that grant control to the
different zones. But the script that is launched on each instance is identical. It will determine its region, public IP
address, hostname, and zone from the instance metadata and it will make the correct API calls.
This example uses UPSERT to create an A record if it doesn't exist, or to update a record if it already does exist.
That's an easy way to work but it has a potential side-effect. If you don't manage tags carefully, you can have two
hosts that both create the same hostname in DNS. There will be no error and it will successfully overwrite the existing
DNS name. This might not be the behaviour you want. You might end up with instances that are still running, but are no
longer reachable via the DNS name that you meant for them to have. If two instances are launched with
webdev1.ec2.example.com as their dnsname tag, both will attempt to set the A record in the
Both will succeed. The second one will be reachable by that DNS name, and the first one will have no DNS name associated
with it. One solution to this problem is to simply be meticulous with tagging. The other possible solution is to change
UPSERT to CREATE in the Python script. The CREATE command will fail if an A record already exists with the same name.
Using CREATE instead of UPSERT requires deleting A records manually, though (or creating a different automated process
that deletes them).
Using this technique, instances can be stop and started at will, but they will use a consistent DNS name. Even though their IP address changes each time they start, the hostname will be the same. There's no need to use an elastic IP address for short-lived instances that have public IPs.