Reusing Automation : CloudFormation Config Sets

AWS CloudFormationAWS CloudFormation is a orchestration tool which deploys infrastructure. It supports nested CloudFormation templates. Nested CloudFormation templates provide a way to reuse large components of infrastructure, such as VPCs, Subnets, Auto Scaling Groups and EC2 Instances. CloudFormation Config Sets provide an alternative reuse mechanism, within an EC2 instance.

User Data

Passing a user data script can customize EC2 instances at launch time. The cloud-init process reads and executes the script, which must be less than 16,384 bytes long.

AWS CloudFormation adds the cfn-init process. cfn-init greatly simplifies the customization process and is not subject to the 16,384 byte restriction of the userData script.

CloudFormation Config Sets

A simple cfn-init configuration contains a single configuration section, called “config”. If cfn-init is called without the –c parameter, “default” is invoked by default.

This simple template starts a CentOS AMI in the US-East-1 (N. VA) region. It installs the CloudFormation tools using a user data script, and then runs cfn-init without the –c option (line 43). The CloudFormation configuration then uses the “default” configSet (lines 46-48) and  installs MariaDB from the CentOS repository:

Another example of a simple template also starts a CentOS AMI in the US-East-1 (N. VA) region. It also installs the CloudFormation tools using a user data script, and then also runs cfn-init without the –c option. The CloudFormation configuration then installs the Apache Web Server from the CentOS repository and then PHP:

 

Rather than having two different CloudFormation scripts, with “default” sections, we can include “db”, “httpd” and “all” Config Sets, and use the “-c” parameter to select between them. The following CloudFormation template adds a parameter, mode, which switches between the ConfigSets:

 

When the AWS CLI CloudFormation call includes --parameters ParameterKey=mode,ParameterValue=httpd,UsePreviousValue=true, that forces cfn-init to use -c httpd. When that happens, the logs show:

When the AWS CLI CloudFormation call includes --parameters ParameterKey=mode,ParameterValue=db,UsePreviousValue=true, that forces cfn-init to use -c db. When that happens, the logs show:

When the AWS CLI CloudFormation call includes --parameters ParameterKey=mode,ParameterValue=all,UsePreviousValue=true, that forces cfn-init to use -c db. When that happens, the logs show:

Using this technique, you can maintain just a single CloudFormation template, and use it to create many different combinations of EC2 instances. It is also easier to cut and paste various Config Sets than to extend UserData blocks.

Share
2 comments
  1. This is great! Thanks for posting. One issue I’m running into is going back and forth between config sets. Let’s say you wanted to update the stack to a state that does not have one of the installations that it previously had. When I update my stack, the EC2 instance comes back up with the installation that I no longer want still installed. My understanding is that this should function as a “Desired State Machine”. So, if my configset previously had installed {a, b, c} and my second config set installs {a, b}, if I was running with {a, b, c} and update to my trimmed down stack, then my EC2 instance should come back up with just {a, b}. Seems to not behave this way. I can create new and choosing the appropriate parameter I get expected results..either {a,b} or {a,b,c}, but can’t update between the two. Any thoughts or suggestions?

    • CloudFormation really doesn’t behave like a Configuration Management system, like Chef, Puppet or Salt Stack which act like Desired State Machines as you put it. For AWS resources, when doing updates it knows whether to modify or recreate a resource, but if a resource is modified outside of the scope of CloudFormation, the update can fail. I guess CloudFormation doesn’t actually track the Config Sets executed for a given EC2 instance, and it doesn’t have enough information to undo them even if it did. If you are looking for this type of behavior, you may want to check out OpsWorks, which provides AWS managed Chef.

Add Comment

Required fields are marked *. Your email address will not be published.