Analysis: AWS Analyze

#340460

Description

The /analysis/cloud/aws/analyze resource is used to collect AWS CloudWatch data and initiate optimization analysis with the cloud infrastructure collected. Below are the series of processes that occur when the initial /analysis/cloud/aws/analyze request is triggered:

set up and initiate data collection of the AWS account and schedule it to run automatically on a nightly basis;
- the first data collection audit will collect up to 60 days of data, if available;
- subsequent data collection will collect the last day's data, if available;
initiate analysis on the data collected using the default policy;
- subsequent analysis is scheduled to run on a nightly basis after data collection;
- you have the option to override the default policy used during an analysis (see GET /analysis/cloud/aws/policy for a list of available AWS policies);
- you have the option to configure a webhook URI, where the results will be sent upon analysis completion (see Add webhook to an analysis for details).

While data collection or analysis is in progress, you can check the status (using the /analysis/cloud/aws/<analysisId>/status resource) or wait for the results to be published to an optional webhook URI.

The reporting database update is typically scheduled to run automatically, on a nightly basis after data collection and analysis are complete. This scheduled job produces reports for each instance recommendation, which is useful for analysts or application owners. These application owner reports are created on a nightly basis after the scheduled analysis, and may therefore only be available on the following day for a new analysis. Exact timing depends on the size of your environment.

Ad-Hoc Tasks

Generally you do not need to run once-off tasks as both data collection and analysis tasks are scheduled automatically. In cases where you need make an ad-hoc request in addition to the scheduled job, the functionality exists for this endpoint.

Historical Data Collection

When Densify initiates data collection, normally the first audit collects 60 days of historical data. This is run once and subsequent audits collect only the last day's data.

If the initial historical audit has been disabled for performance reasons, you will need to run an ad-hoc task to collect the historical data.

A connection to the specified cloud account must already exist before you can run an ad hoc audit. When you execute an ad hoc refresh an audit task will be configured but a new connection will not be created. If the cloud connection does not already exist and the API POST contains triggerAdhocAudit=true, then you will get an error message.

If there is more than one account associated with the specified account ID (i.e. a payer account with many linked accounts), the Densify API handles it in the same way that analyses are currently rerun using the POST operation.

Once the audit is complete you need to rerun the associated analyses as indicated below or you can wait for the next scheduled execution of the analyses and RDB populate.

Analysis Update

You can make an ad-hoc request to refresh an existing analysis, outside of the scheduled nightly run using /analysis/cloud/<aws|azure|gcp>/analyze. This manual, ad hoc analysis request does not perform data collection or reporting database updates. It only runs the analysis on the existing data collected with the following behavior:

If a new policy is provided, the analysis will use the new policy/settings. If no policy is provided, the analysis will use the existing policy that is already configured for the analysis.
If a new webhook is provided, the analysis will send results to the new webhook URI. If no webhook is provided, the analysis will send results to the existing webhook, if configured.
If the same analysis is already running, the request does not proceed and an appropriate message is returned.
If the specified analysis has data collection scheduled within 30 minutes, the request does not proceed and an appropriate message is returned. For example, if data collection is scheduled to run at 12:05 AM, and you initiate a manual, ad hoc analyze request at 11:45 PM, then the analysis will not proceed and an error message is returned.

Prerequisite Configuration

Before you can collect AWS CloudWatch data, you need to create an IAM role for Densify to have a trust relationship with your AWS account that contains data. See AWS Data Collection Prerequisites for an IAM Role for details on how to set up the IAM role for data collection.

Resource

/analysis/cloud/aws/analyze

/analysis/cloud/aws

Supported Operations

Table: AWS Analyze Supported Operations

Operation	HTTP Method	Input	Output	Description
Run AWS data collection and analysis	POST /analysis/cloud/aws/analyze	Request Body Parameter: accountId roleArnName roleExternalId connectionName (optional) policyInstanceId (optional) webHook (optional)	message status	This resource operation is used to: Collect AWS cloud data connected via IAM role access. Run analysis on AWS cloud data collected. (Optional) Send results to webhook receiving application. Schedule data collection and analysis processes each night subsequent to the initial request. Example: Running AWS Data Collection and Analysis
Re-run AWS data analysis	POST /analysis/cloud/aws/analyze	Request Body Parameter: accountId roleArnName roleExternalId policyInstanceId(optional) webHook(optional)	href message status	This resource operation is used to re-run an analysis that already exists. You can specify an updated policy and/or webhook to use for the analysis. Data collection is not run. Data collection only occurs during the first /analyze request, and is then scheduled to run nightly The updated policy or webhook is saved and will be used in future, scheduled analyses. You cannot initiate a request if data collection or the analyses are in progress or within 30 minutes of the time that these tasks are scheduled to run.
List all generated analyses	GET /analysis/cloud/aws/	Path Parameter: N/A Request Body Parameter: N/A	Lists all analyses that have been created with details.	This resource operation is used to obtain the analysis ID that is required for other operations.
Run the 60-day historical AWS audit	POST /analysis/cloud/aws/analyze	Request Body Parameter: accountId roleArnName roleExternalId triggerAdhocAudit	href message status	This resource operation is used to re-run an audit for which a connection and daily, scheduled audit already exists. You can optionally specify the number of days of historical data to collect. If not specified the previous 60 days from yesterday's date are collected. If you initiate an audit request when data collection or analysis is already running or within 30 minutes of the time that these tasks are scheduled to run, then the request will fail and an error message is returned.
Update AWS credentials	PUT /analysis/cloud/aws/ <analysisId>	Path Parameter: analysisId Request Body Parameter: roleArnName roleExternalId	message status	This resource operation is used to update AWS account's IAM role ARN and External ID for the next scheduled analysis. Specify the updated roleArnName and roleExternalId in the request body. Example: Updating AWS Credentials
Modify the policy	PUT /analysis/cloud/aws/ <analysisId>	Path Parameter: analysisId Request Body Parameter: policyInstanceId	message status	This resource operation, with a policyInstanceId in the request body, is used to update the policy used in the next scheduled AWS Analysis. Example: Updating AWS Policy The policy used for the analysis is listed in the output with the recommendation results. If you change the policy while the analysis is in progress, the change will not affect the current analysis execution and will be used for the next analysis refresh.
Update AWS credentials and policy	PUT /analysis/cloud/aws/ <analysisId>	Path Parameter: analysisId Request Body Parameter: roleArnName roleExternalId policyInstanceId	message status	This resource operation is used to update AWS account's IAM role ARN, External ID, and the policy used in the next scheduled Analysis. Example: Updating AWS Credentials and Policy

Parameters

Path Parameters

Table: AWS Analysis Path Parameters

Parameter Name	Type	Description
analysisId	string	The unique referenced ID of the AWS analysis.

Request Body Parameters

Table: AWS Analysis Request Body Parameters

Parameter Name	Type	Description
accountId	string	The AWS account ID with the CloudWatch data to collect. See AWS Data Collection Prerequisites for an IAM Role for details on setting up the ARN.
connectionName (optional)	string	Use the connection name to clearly identify this connection within Densify. This name will appear in the Saved Connections list in theDensify UI. By default, the connection name is set to the Subscription ID. The connection name must be unique within the Azure connection type section, so if the name is already in use, the request fails with an error message. This connection name can be used for filtering. Note:The Connection Name is limited to 32-characters.
roleArnName	string	The Amazon Resource Name (ARN) for the IAM role that you created in AWS to collect data. See AWS Data Collection Prerequisites for an IAM Role for details on setting up the ARN. To update the ARN, refer to Update AWS credentials.
roleExternalId	string	The external ID specified for Densify when the IAM role was created. See AWS Data Collection Prerequisites for an IAM Role for details on setting up the ARN. To update the external ID, refer to Update AWS credentials.
policyInstanceId (optional)	string	The cloud policy used for optimization analysis. This parameter is optional and used to override the default policy for AWS cloud analysis. See GET /analysis/cloud/aws/policy.
triggerAdhocAudit (optional)	string	The flag to trigger an ad-hoc 60-day historical audit. This parameter is optional and used to run the 60-day historical audit immediately and once only. Typically the 60-day historical audit is run first, when data collection is initiated for the specified AWS account. If this audit has been disabled for performance reasons, it can be run to collect the historical data as compute resources are available. A connection to the specified AWS account must already exist before you can use this flag. A once-off task will be configured and a new connection will not be created, if the connection does not already exist. If there is no existing connection AND the API post contains ad hoc=true, then you will see an error message.
webHook (optional)	uri authType authValue	The webhook definition to an external application. Optimization results are sent to the webhook-defined application when analysis is complete. See Parameters for details of each parameter in the webhook definition.

Response

Table: AWS Analysis Response Schema

Element	Type	Filter/Sort	Description
href	string		The referenced resource to the analysis entity. See Analysis: Entity for details of the analysis entity resource. When a new analysis is requested from the /analyze resource, the entity ID will not be available until after data collection completes and the analysis entity is created.
message	string		The message for the status response is returned.
status	number		The HTTP response code of the request. Possible status values include: 200—success with request; 400—invalid parameters; 401—authentication failed; 404—resource not found; 500—internal server error.

Examples

Example: Running AWS Data Collection and Analysis

The following example shows you how to initiate AWS data collection and analysis, and send the results to a WebHook.

Example: Running AWS Data Collection and Analysis

Request:

Copy

POST /analysis/cloud/aws/analyze
{
  "accountId": "188665225420",
  "roleArnName": "arn:aws:iam::188665225420:role/ReadOnlyAccess",
  "roleExternalId": "password",
  "policyInstanceId": "4a63f651-a583-4157-97ff-35651370ffbe",
  "webHook": {
       "uri": "http://mywebhookserver/webhook/results",
       "authType": "basic",
       "authValue": "tester:testerpassword"
  }
}

Response:

Copy

{
  "href": "Not available",
  "message": "OK",
  "status": 200
}

Example: Updating AWS Credentials

The following example shows you how to update your AWS account's IAM role ARN and External ID.

Example: Updating AWS Credentials

Request:

Copy

PUT /analysis/cloud/aws/9a5d2d55-6d85-4fde-8bab-fcd0cef8c5bf
{
    "roleArnName": "arn:aws:iam::506226932793:role/my_new_readonly",
    "roleExternalId": "NewPassword"
}

Response:

Copy

{
  "message": "ok",
  "status": 200
}

Example: Updating AWS Policy

The following example shows you how to update the policy used in your AWS analysis. The new policy will be used in the next scheduled analysis. To obtain the policy instance ID (i.e. policyInstanceId), refer to the Analysis: Policy resource.

Example: Updating AWS Policy

Request:

Copy

PUT /analysis/cloud/aws/9a5d2d55-6d85-4fde-8bab-fcd0cef8c5bf
{
    "policyInstanceId" : "0c0ef18b-9367-4071-b733-396f63e51925"
}

Response:

Copy

{
  "message": "ok",
  "status": 200
}

Example: Updating AWS Credentials and Policy

The following example shows you how to update both the AWS credentials and policy in your analysis, at the same time. The new credentials and policy will be used in the next scheduled analysis.

Example: Updating AWS Credentials and Policy

Request:

Copy

PUT /analysis/cloud/aws/9a5d2d55-6d85-4fde-8bab-fcd0cef8c5bf
{
    "roleArnName": "arn:aws:iam::506226932793:role/my_new_readonly",
    "roleExternalId": "NewPassword",
    "policyInstanceId" : "0c0ef18b-9367-4071-b733-396f63e51925"
}

Response:

Copy

{
  "message": "ok",
  "status": 200
}

Example: Running the 60-Day Historical Audit

The following example shows you how to run the initial 60-day historical data collection.

Example: Running 60-Day Historical Audit

Request:

Copy

POST /analysis/cloud/aws/analyze
{
  "accountId": "188665225420",
  "roleArnName": "arn:aws:iam::188665225420:role/ReadOnlyAccess",
  "roleExternalId": "password",
  "triggerAdhocAudit": "Yes",
  "startDayOffset":"20" ,
  "endDayOffset":"10"
}

Response:

Copy

{
  "href": "/analysis/aws/209726931496/status",
  "message": "Analysis in progress",
  "status": 200
}