I’ve been using AWS since EC2 was in limited beta. At 42Lines we’ve built a lot of infrastructure on AWS that pre-dates most current AWS features like IAM. While we’ve been using IAM for all new projects, we recently decided to enforce multi-factor authentication (MFA) for all infrastructure and cease all use of account (root) keys. This led to the following adventure; the theme you’ll notice is adding support for security tokens. Security token support is required as using keys issued with MFA have three parts (access_key, secret_access_key, security_token) rather than the two (access_key, secret_access_key) most AWS users would find familiar.
We needed a way to easily authenticate IAM users with MFA while in their shell environments and manage their (many) related shell environment variables. This led to the creation of some code that will be open-sourced shortly.
We needed a way for our existing Perl-based orchestration/launching system to assume IAM roles. Net::Amazon::EC2 doesn’t support the IAM API, so I decided to add a Python script to make the AssumeRole call and emit the temporary credentials as JSON. I really dislike having multiple languages for a tool, but our long term plan is to move this tool to Python anyway, so this step aligns with our long term plans. The Python script for making the AssumeRole call will also be a part of the aforementioned open-sourcing of code.
However, boto‘s STS does not support passing a security token. So, I proposed a fix for that.
Our Perl tool would be run (in most cases) by MFA IAM users, so the Perl tool needed to pass a security token for the calls it makes. I previously added support for IAM temporary security credentials from the on-instance metadata service and support for launching with IAM roles to Net::Amazon::EC2. I now needed to add passing the security token when creating the connection object; hence this proposed change.
Further, as our Perl AWS tool was from nearly the beginning of AWS time it used SimpleDB to keep track of various instance information. This is because it was written before tags existed in AWS. I didn’t bother to see if the SimpleDB Perl library had been updated to support STS issued temporary credentials; I took a side trip to move all information formerly stored in SimpleDB to EC2 instance tags and remove all use of SimpleDB.
Finally there were some boto-using command line utilities (including the new MFA-related ones I wrote) that now needed to pass a security token to boto. I’d much rather handle this via environment variables (as opposed to adding an option to every script that is passed to the boto connection object), so tonight I finished this adventure by writing and submitting another proposed patch to boto.
The end result of this overly long adventure is all staff using MFA tokens for all AWS access. Hopefully all of my patches are adopted upstream and I can remove the custom-built packages from our repos.
If the holidays allow I’ll publish a Github repo of my new MFA helper scripts in the next few days.