How to Adopt Continuous Optimization in AWS Using CloudFormation

Most developers are familiar with continuous integration/continuous delivery (CI/CD). At a high level, CI/CD provides the ability to automatically integrate, test, and deliver new iterations of a product and code with little to no human intervention. Continuous optimization (CO) is the additional concept of ensuring that with every iteration of the product or code, the optimal infrastructure is systemically selected.

This article discusses how you can enable continuous optimization within your DevOps pipeline to optimize infrastructure allocation for any application.

Continuous Optimization in Your DevOps Pipeline

Let’s start by looking at some of the basic components of a DevOps pipeline:

Code Generation

a set of tools designed to automate the code merge/build process. This can include repositories to hold templates and code and technologies to build code, for example, AWS Code Commit, GitHub, AWS Code Build, and others

Infrastructure Deployment

orchestration technologies like infrastructure as code that automate the allocation and configuration of infrastructure, for example, AWS CloudFormation, Terraform, Ansible, Chef, and others

Application Deployment

the final stage where the application or new code is deployed and configured on the procured infrastructure, for example, AWS Code Deploy, Lambda, and others

Infrastructure Deployment is where we will introduce continuous optimization, which relies on two critical technologies: 1) infrastructure as code, and 2) a centralized system of record.

• Infrastructure as code is a collection of technologies that enable users to manage (create, update or destroy) cloud resources through templated descriptions of that infrastructure. Typically provided in either a JSON or YAML format.
• A centralized system of record (persistent database) in the context of CO, is a centralized repository of configuration parameters that can be natively referenced and leveraged by the infrastructure as code technology.

As a best practice when using infrastructure as code, configuration parameters that can possibly change in the future should be surfaced out of the templates and stored in a centralized database. There are numerous benefits to this approach, one of which is all future changes to those elements being managed through infrastructure as code can be implemented without modification to the underlying templates.

For the purposes of this article, we will use CloudFormation (an AWS infrastructure as code technology) and Amazon EC2 Systems Manager Parameter Store (an AWS configuration and secrets store) to implement continuous optimization. This diagram describes the data flow that we will implement.

Continuous optimization relies on the optimization of key parameters that control infrastructure allocation which can be defined through the resource specification blocks. Some examples of these parameters include but are not limited to, EC2:InstanceType, RDS:DBInstanceClass, ASG:MinSize, ASG:MaxSize, etc. In order to deliver the updated change through CloudFormation, there are two critical items to consider.

How Should CloudFormation Reference Parameter Store?

CloudFormation templates can natively reference Parameter Store through ‘parameter resources’ available as a feature in CloudFormation. Due to limitations in the number of definable parameter resources (60), it’s not practical to implement the reference mechanism in this way.

Instead we can introduce another resource into the template, which can act as a provider to extract optimization insights from Parameter Store. The individual infrastructure resources (i.e. EC2, RDS, ASG, etc.) can then directly reference this new resource for recommendations.

We can leverage CloudFormation CLI to build such a resource provider which has a resource type specification and handlers that control API interactions with the underlying AWS or third-party services. These interactions may include create, read, update, delete and list (CRUDL) operations for resources. You would use resource providers to model and provision resources using CloudFormation. For example, AWS::EC2::Instance is a resource type by Amazon EC2 provider.

The open source AWS CloudFormation CLI project will walk you through how to build a custom resource provider. The resource provider can be used as a platform to query any third party system (i.e. Parameter Store) for insights. These insights should be delivered back to CloudFormation using the supplied responseURL.

How Can We Generate Optimization Insights?

At the crux of solving the continuous optimization challenge is to ensure you have automatable insights populated within Parameter Store. Unfortunately, a discussion on how to accurately generate such insights requires advanced analysis of infrastructure performance metrics and is therefore beyond the scope of this blog.

One way to generate these optimization insights is through Densify. We specialize in the analysis of cloud and container workloads to determine optimal supply allocations for performance and cost. We ultimately deliver these configuration settings to your preferred centralized system of record—for example ,AWS Parameter Store.

Example Implementation

For this example, we will use the following components:

1. Densify as the optimization platform to generate insights ready for automation
2. An existing CloudFormation resource provider built by Densify to reference recommendations from Parameter Store

If you want to follow along, please ensure you have a Densify instance running and connected to your AWS environment. If you haven’t already, you can visit the Densify signup page to get access to a free, no-obligation trial.

Let’s consider an example where we have a CloudFormation template managing a set of EC2, RDS, and ASG resources. There are two steps to enabling Continuous Optimization:

Enable the Densify Resource Provider

You can enable this resource provider by running this CloudFormation template in any account you wish to automate. Keep in mind that this account should be connected to Densify (as a prerequisite). CloudFormation is a regional technology and as such the resource provider must be enabled in each region you wish to automate.

Start by running this template in any region. You will be presented with the following Parameters input screen:

1. Enter in the Densify login details. This will enable the resource provider to access the optimization insights stored in Densify.
2. Select how the resource provider will acquire these recommendations. Either directly from Densify or via a key-value store (i.e. Parameter Store).
3. Specify which regions to activate the provider in comma-delimited format

Modify Existing CloudFormation Templates (3 Steps)

This section details how to modify your existing CloudFormation templates manually. If you would like an automated approach, please skip to the next subsection.

Step 1: Add the Densify Resource Provider

In order for your existing CloudFormation templates to make use of the new resource provider, there needs to exist a mechanism for CloudFormation to reference that resource provider. This can be done by introducing the densify infrastructure resource into your template.

  DensifyResourceProvider:
    Type: Custom::DensifyResourceProvider
    Properties:
      ServiceToken: arn:aws:lambda:<Region>:<Account>:function:DensifyResourceProvider
      LastRefreshTime: !Ref 'DensifyLastUpdatedTimestamp'
      <logical_name_of_resource>.<parameter_to_optimize>: <default_value>

The areas that need customization are the highlighted text. The ServiceToken contains the ARN to the deployed resource provider. The region and account need to be updated accordingly.

For every resource that you would like to optimize you need to enter a row to specify the initial default sizes.

ASGInstance.MinSize: 1
ASGInstance.MaxSize: 3
ASGInstance.InstanceType: t2.micro

Step 2: Reference the Provider

For each of the individual resources you wish to optimize we need to modify the line holding the parameter in the resource spec. For example for EC2, we are looking to update the Properties:InstanceType parameter. These parameters will be updated to a GetAtt with the following nomenclature, DensifyResourceProvider.<logical_name_of_resource>.<parameter_to_optimize>

Step 3: Add the Last Updated Timestamp Parameter

The LastUpdatedTimestamp parameter holds the last date/time a recommendation was generated by Densify for any resource it is currently managing. The parameter resource we will create in the CloudFormation template will directly reference Parameter Store for the value. The Parameter Store entry is initialized in Step 1, when we activated the resource provider.

  DensifyLastUpdatedTimestamp:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /densify/config/lastUpdatedTimestamp

Modify Existing CloudFormation Templates (Automatic): Alternative to Step 2

Densify also provides tooling to automatically modify existing CloudFormation templates. This can simplify the process greatly and eliminates any human error due to manual modification of templates.

Start by running this template in any region. You will be presented with the following Parameters input screen:

1. S3 Bucket Name: The name of the S3 bucket that this CloudFormation template will create to store all modified templates.
2. Stacks to Modify: A specification of which particular stacks in the environment we want to modify in the following format, <REGION>:<StackName>. Region where the stack exists and the name of the stack.
3. Execute Modified Stack: For stacks that have been modified, you can specify whether you want densify to auto update those stacks in place. This is achieved by issuing a CloudFormation stack-update for those stacks using the updated templates. The process will only execute against a stack if that stack does not contain any resources that do not support drift detection and the stack is not drifted.
4. Go to each of your existing templates and verify that they have been updated for automation. If there are some templates that weren’t modified, it’s usually because the associated stack is not currently in sync. For these situations, manual modification of the stack is required—please refer to the previous subsection.

That should be it!

Before running a CloudFormation stack-update, be sure to approve the relevant insights through the Densify console.

To learn more about enabling continuous optimization for your cloud resources via CloudFormation CLI, please visit our CloudFormation Optimization as Code GitHub repository or contact us at answers@densify.com.