Drilldown: AWS Budgets vs Cost Explorer

Sitting alongside each other within the AWS Cost Management group, AWS Budgets and AWS Cost Explorer are complementary services from Amazon Web Services (AWS). Using them together, you can analyze your cost and usage patterns and use that analysis to implement effective governance controls and cost optimizations that support your organization.

AWS Cost Explorer helps you to explore and analyze spending across your AWS accounts, offering a wide range of filtering options to help you dig down into the details.

With AWS Budgets you can set up the reports and alarms you need to keep a close eye on your spending.

We will look into both services in more detail and see how they differ and how they can be used together.

AWS Budgets

What are AWS Budgets?

When you have responsibility for cost control, you set spending limits and want to know when actual spending nears or exceeds those limits. This can be especially important in a cloud computing context, where users and automated systems have higher freedom to create resources by design.

AWS Budgets gives you two mechanisms to keep track of spending.

You can see how your spending is changing on a daily basis through the AWS Management Console, but AWS Budgets allows you to have this view delivered as a regular email report without logging in. Within AWS Budgets, a budget is a single, regularly updating cost or usage metric relating to your AWS resources. The rich structure of this AWS data allows you to set up multiple budgets to track this data across different departments or drilling down into different systems and resource types.

Instead of relying on people reading regular reports to spot when costs are at risk of overrunning, you can set up alerts for a budget, which can notify the right people or trigger automated responses such as applying restrictions to the account or shutting down some AWS resources.

The combination of routine reports and event-driven alerts, with appropriate recipients, is a strong foundation for cost control.

In addition to service cost, AWS Budgets lets you track service usage, and utilization and coverage for Reserved Instances and Savings Plans.

Creating budgets

There is an AWS setup wizard that starts when you create a budget through the console. Within the wizard, there are “Info” links at the top of each page, which lead to more detailed instructions and AWS documentation.

Most of the filters will be already familiar or clear to AWS users, with the possible exceptions of Usage Type and Usage Type Groups. These are summarily documented for both AWS Budgets and AWS Cost Explorer, and more information can be found in the Cost Management Blog.

Budget alerts

Alerts are attached to a budget and can be created when creating or editing that budget. They consist of a threshold and a notification. The threshold contains a trigger, which determines whether the alert fires when today’s actual usage or the forecast usage for the budget period crosses the threshold.

You can also configure alerts to trigger automated actions when they fire. An action can be one of:

1. Attach an IAM policy to a user, group, or role.
2. Attach a Service Control Policy (SCP) to an Organizational Unit or the organization root.
3. Stop specific EC2 or RDS instances.

Forecasting in AWS Budgets

Forecasting gives you an additional option for how you set up your alerts. Using actual spend, you might set up an alert to notify you at 80% of your monthly budget, say. Using forecasting, you might want an alert based on projected overrun, say 105%. You can use both types of alerts with the same budget - a budget will support up to 5 alerts. The caveats with forecasting are 1) that forecast alerts won’t fire at all unless you already have enough (roughly 5 weeks) usage data, and 2) as with any forecasting, although there will be some intelligence in the underlying algorithm, it’s only based on your previous usage patterns and may be inaccurate.

Budget Reports

Budget reports are sent by email on a regular cadence.

AWS keeps these reports simple (the only complexity is in the budgets themselves) - for a single report you can customize:

1. Which budgets are included (you can include more than one).
2. How often the report should be sent - daily, weekly, or monthly.
3. Who should receive the report - a list of email addresses.

AWS Budgets limits & pricing

Refer to these AWS documentation links to find current pricing and limit levels:

• AWS Cost Management Pricing
• AWS Billing Quotas and Restrictions

Creating budgets, alerts and notifications is free of charge.

Budgets configured with automated actions incur a daily charge (the first two are free).

Each budget report delivered incurs a fixed cost (does not vary by the number of budgets or recipients).

You can create up to 50 budget reports, and each report can contain up to 50 different budgets. This gives a total limit of 2,500 budgets that can be sent in reports from AWS Budgets, though you can create up to 20,000 budgets.

AWS Cost Explorer

What is AWS Cost Explorer?

AWS Cost Explorer gives you:

1. An interface for exploring your cost and usage data using more complex querying functionality.
2. Automatic recommendations for cost savings, for Savings Plans and Reservations, based on your usage.

The graph you see when creating a budget in AWS Budgets is a simplified version of the Cost Explorer chart. There is a little extra functionality (and better visual design) here. You have full control over the reporting date, can visualize costs broken down by one dimension, can set both inclusion and exclusion filters, and can see more granular forecasting details.

Similar filtering functionality is available in both tools, and although there is currently no way to create a budget directly from Cost Explorer, a common way to use the two tools is to experiment rapidly with different filters in Cost Explorer, and once you find a useful filter set that gives you insights you want to keep track of, save it as a report, and then go back to AWS Budgets and create a budget that uses the same filters.

If you select a Usage Type or Usage Type Group in a filter, you can see relevant usage data on the chart below the cost data.

Forecasting in Cost Explorer

You can see more from forecasting in Cost Explorer than in Budgets. Although it’s not compatible with the grouping feature, when you have forecasting turned on (Check “Show forecasted values”) it gives you both a forecast value and a confidence/prediction interval around it for each time point. AWS discusses its forecasting in more detail in its documentation.

Our other articles have discussed motivations for introducing a detailed tagging strategy for your AWS accounts, for making full use of AWS Organizations, and for utilizing Cost Categories. Cost Explorer is one of the places where these decisions can really start to pay off, as you can filter and group on all three of these, quickly segmenting your organization’s cost and usage data by its own logical structure.

You can save complex filters as reports in Cost Explorer, and you can use one of several built-in reports as a starting point for filtering your data. Unlike AWS Budgets, there is no functionality for delivering the reports, but you can download a report’s underlying data table as a CSV file, and you can share a report using its URL with anyone who has access to Cost Explorer in the account.

Savings Plans and Reservations

Recommendations for Savings Plans and Reservations can be found in the side navigation when in Cost Explorer. Although in the navigation it does not look like they are so closely connected, Cost Explorer needs to be enabled before either set of recommendations or the associated utilization and coverage reports are available. If you are prepared to make longer-term commitments to using AWS resources, then these pages can be a great source of cost savings.

Reports in AWS Cost Explorer

A Cost Explorer report is simply a configuration of one of its charts, which is either built-in or saved. There are currently five different report types, but unless you are looking at Savings Plans and Reservations, the Cost and usage report is the one you will be working with.

The Reports page gives a condensed summary of the configuration of each report.

Hourly and Resource granularity

For cost and usage reports, there is an option to show hourly data points. One use case for this would be to match your resource usage to your customer traffic patterns at this level. You can find the setting to enable this under Preferences in the console left navigation. The same option also makes it possible to filter and group by individual EC2 instances.

AWS Cost Explorer limits and pricing

Refer to these AWS documentation links to find current pricing and limit levels:

• AWS Cost Management Pricing
• AWS Billing Quotas and Restrictions

You can save up to 50 ​​reports per account, and a report can use up to 100 filters.

Using Cost Explorer through the AWS Management Console is free of charge, but programmatic access to the API (only used by engineers, or when building custom solutions, or when using a third-party integration) incurs a per-request cost.

Hourly and Resource granularity, turned off by default, incurs costs for storage and access if enabled.

Best practices for using AWS Budgets and AWS Cost Explorer

1. Implement AWS Organizations, tagging, and Cost Categories. These will allow you to visualize cost and usage data at granular levels that correspond to your unique business structure.
2. Create and save custom reports in Cost Explorer to answer cost and usage questions and help optimize your spending.
3. Use AWS Budgets to establish robust controls around key cost and usage metrics.
4. Test reports and alarms to ensure that they are configured correctly and are delivered successfully.

How can I go further?

There are two main directions where other AWS services can extend what we’ve covered with Budgets and Cost Explorer.

If you want to go even deeper into your billing data, you can look at AWS Cost and Usage Reports. Despite the naming similarity to the main Cost Explorer report, this is a separate service that you can turn on, to have Amazon send raw billing data to an S3 bucket you own. This raw billing data describes exactly what Amazon charges you for at the most granular level. You can inspect the raw files manually, or you can use Athena to run arbitrary queries against the whole dataset. Note that even the raw billing data is sent in batches, two or three times a day, and it is not currently possible to access anything like real-time AWS billing data.

The second direction is to apply more intelligent analysis to the data. AWS Cost Anomaly Detection can supplement your analyses by letting AWS use its AI services to detect anomalies, automatically accounting for natural patterns.

In addition to AWS services and in-house solutions, you might also look at third-party tools that create enriched insights from AWS billing data, or offer useful integrations.

Conclusion

AWS Cost Explorer and AWS Budgets use the same underlying billing data to support you in different FinOps tasks. You can use them together to understand, optimize, and control your AWS costs, and get better value for money from the cloud.