Using the Public Cloud Connections Wizard

Using the Public Cloud Connections Wizard

#380290

The Public Cloud Connections wizard guides you through the steps to create and schedule daily data collection from your public cloud instances. Collected configuration and workload data is then analyzed and used to recommend sizing and placement optimization opportunities.

The Public Cloud Connections wizard is accessible from the Densify Console's landing page or from the Public Cloud >  Add Cloud Connection dropdown menu. See Navigating Densify for details.

The following cloud platforms are supported through the Public Cloud Connections wizard:

Note: Collection of billing data is currently not supported for Google Cloud Platform.

You can modify the connections once they have been created:

You may also need to do the following:

Additionally, you can collect and analyze data through the Densify API. Refer to the following use case for an example of API data collection:

Configuring a Connection to AWS

You can connect to your AWS account with the following method:

  • IAM Role—An IAM cross-account role that establishes a trust relationship between AWS accounts. In order to create the connection you need the prerequisite information, including the external ID and Role ARN. See AWS Data Collection Prerequisites for an IAM Role for details on obtaining this information.

You need to create a separate cloud connection for resource utilization metrics and for the billing data that you want to collect. For example, if you need to collect both workload and billing data from the same account, you need to create two connections using the same credentials.

Using the Public Cloud Connections Wizard—IAM Role

To learn more watch the video:

  1. On the Densify landing page, click Connect to your AWS accounts.
  2. or

    from the Densify Console, navigate to Public Cloud > Add Cloud Connection.

  3. Click on the Amazon Web Services tab.
  4. Select the connection type:
    • Resource Utilization Metrics— This connection type is for collecting CloudWatch data.
    • Billing— This connection type is for collecting billing data. The collection of billing data must be enabled separately. Contact [email protected] to enable this feature.
  5. Select Connect Using: "IAM Role".
  6. The Densify Account ID is predefined. This value must match the value you entered when you created the role in AWS.
  7. Enter the AWS-specific connection parameters, as listed in the table below. Refer to AWS Data Collection Prerequisites for an IAM Role for details on creating the accounts and obtaining the credentials.
  8. Table: AWS Connection Parameters - IAM Role

    Field

    Description

    External ID

    The external ID specified for Densify, when you created the IAM role in AWS.

    If you need to edit or review a saved connection, for security reasons, you will need to re-enter the external ID.

    Role ARN

    The Amazon Resource Name (ARN) for IAM role that you created in AWS.

    Note: The External ID and the Role ARN are encrypted and are not displayed as plain text in log files.

  9. Once your AWS connection parameters are entered, verify your connection using Click to Verify Account Connection.
    • If the credentials are valid, you will be connected and authenticated. Once the account is verified, Densify discovers the AWS account ID and displays the ID number. See Table: AWS Connection Parameters - Billing below.
    • For billing connections, in addition to the discovery of the AWS account ID, the linked accounts connected to the payer account, CUR report configuration, and S3 bucket(s) are also discovered and validated. See Table: AWS Connection Parameters - Billing below.
    • Note: The number of instances returned is a high level estimate obtained when Densify verifies the connectivity. It may not match the number of instances discovered during the more detailed, scheduled data collection.

    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. See AWS Data Collection Prerequisites for an IAM Role for details.

    The Save Connection button is disabled until the connection is successfully verified (for both resource utilization and billing connections), in both create new and edit modes.

  10. When account verification is successful the following additional information is displayed:
  11. Table: AWS Connection Parameters - Billing

    Field

    Description

    AWS Account ID

    The AWS account ID is automatically discovered based on the validated access key ID and the secret access key combination. This parameter is automatically populated after verification and cannot be modified.

    Connection Name

    This field is displayed only after the connection has been validated.

    Use the connection name to clearly identify the AWS connection in Densify. This name will appear in the Saved Connections list and based on the connection type, it will appear under the corresponding Resource Utilization Metrics or Billing heading.

    The connection name is initially set to the AWS Account ID. You can modify the connection name to something meaningful for quick identification.

    The connection name must be unique within the AWS connection type section, so if the name is already in use, you are prompted to enter a new connection name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from entering a string that is more than 32 characters.

    Billing Report

    When creating billing connections, a list of available Cost and Usage Reports (CUR) is displayed.

  12. After you have specified the connection name and/or selected the billing report, click Save Connection.
  13. The connection status, at the bottom of the page, will be updated. You will also see the connection listed in the Saved Connections section, under the Resource Utilization Metrics heading for a configuration/workload connection or under the Billing heading for a billing connection.
  14. For resource utilization data, the initial audit collects the last 60 days of data, if available, and each, subsequent daily audit collects the last 24 hours of data. For billing data, one month of historical data is collected, if available. This value is configurable Contact [email protected] for details.

  15. If you want to add another connection, click Add New Connection and follow steps 3 to 7 again to configure the connection.
  16. Close the Public Cloud Connections page once you have completed creating all your AWS connections.

Once the connection has been verified data collection starts. You will be contacted, within one business day, once your data collection and analysis are complete. Subsequent data collection, analyses, and database refresh are scheduled to run on a nightly basis.

Configuring a Connection to Microsoft Azure

To learn more, watch the following video:

There are two methods for creating Azure connections:

Click the Microsoft Azure tab to create a connection to your Azure environments. You will enter your Connection Name and an account user name and password in order to collect configuration, workload or billing data from your hosted infrastructure.

You need to create a separate cloud connection for resource utilization metrics and for the billing data that you want to collect. For example, if you need to collect both workload and billing data from the same account, you need to create two connections using the same credentials.

Your information is verified and the connection can then be saved and is scheduled to run regularly. In order to create the connection you need the relevant account information. See Microsoft Azure Data Collection Prerequisites for details on the obtaining the required credentials to enable data collection.

Using the Public Cloud Connections Wizard

  1. On the Densify landing page, click Connect to your AWS accounts.
  2. or

    from the Densify Console, navigate to Public Cloud > Add Cloud Connection.

  3. Click on the Microsoft Azure tab.
  4. Click the appropriate radio button to select the type of connection to create:
    • Resource Utilization Metrics—This connection type is for collecting utilization data.
    • Billing—This connection type is for collecting billing data.
  5. Enter the Azure-specific connection parameters. Refer to one of the following topics to obtain the required information:
  6. Table: Account Information - User Credentials

    Field

    Description

    User Name

    Enter the user name associated with your Azure account.

    Password

    Enter the user's password.

    Connection Name

    Use the connection name to clearly identify the Azure subscription in Densify. This name will appear in the Saved Connections list and based on the connection type, it will appear under the corresponding Resource Utilization Metrics or Billing heading.

    The connection name must be unique within the Azure connection type section, so if the name is already in use, you are prompted to enter a new connection name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from entering a string that is more than 32 characters.

    Table: Account Information - Service Principal

    Field

    Description

    Application ID

    Specify the Application ID/Service Principal. Both the ID and the corresponding key are provided when you create the application through your Azure portal. 

    Secret Key

    Enter the key corresponding to your application/service principal. This is called the Client Secret in the Azure portal interface.

    Tenant ID

    The tenant ID corresponds to the local Active Directory.

    Connection Name

    Specify a name for the connection. This name will appear in the Saved Connections list. If a name is not specified, the Application ID is used.

    If the name is already in use or the connection exists, you are prompted to enter a new connection/account name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from entering a string that is more than 32 characters.

  7. Once your information is entered for all fields, click Click to Verify Account Connection to validate the credentials.
    • If the credentials are valid, you will be connected and authenticated. Once the account is verified, all subscriptions that are associated with the account or Application ID are listed. Select the subscriptions that you want to include in the audit.
    • For billing connections, you can select the subscriptions for which you want to collect billing data.
    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. The user account/Service Principal to be used for data collection only requires the "Reader" role privileges to collect both utilization and billing data. See Microsoft Azure Data Collection Prerequisites for details.
  8. Once the connection has been verified, click Save Connection.
  9. The connection status, at the bottom of the page, will be updated. You will also see the connection listed in the Saved Connections section, under the Resource Utilization Metrics heading for a configuration/workload connection or under the Billing heading for a billing connection.
  10. For resource utilization data, the initial audit collects the last 60 days of data, if available, and each, subsequent daily audit collects the last 24 hours of data. For billing data, one month of historical data is collected, if available. This value is configurable Contact [email protected] for details.

  11. Click Add New Connection, to add another connection.

Once the connection has been verified data collection is scheduled.

You will be contacted within one business day, once your data collection is complete.

Note: If you create another connection, you must ensure subscriptions are not included in more than one cloud connection.

Configuring a Connection to Google Cloud Platform 

To learn more watch the video:

Click the Google Cloud tab to configure a GCP connection. You will need to provide a Connection Name and a Service Account Key File to establish a connection and collect configuration and workload data from GCP. See Google Cloud Platform Data Collection Prerequisites for details on obtaining the required credentials and configuring your GCP project to enable data collection. After your credentials are verified, you can save the connection and schedule data collection to run regularly.

Adding a GCP Connection

  1. On the Densify landing page, click Connect to your AWS accounts.
  2. or

    from the Densify Console, navigate to Public Cloud > Add Cloud Connection.

  3. Click the Google Cloud tab.
  4. Enter the GCP-specific account information. Refer to Google Cloud Platform Data Collection Prerequisites for details on creating the accounts and obtaining the credentials.
  5. Table: Account Information

    Field

    Description

    Service Account Key File (JSON)

    Specify the JSON file that contains the service accounts keys. This file is created using the Google Cloud Identity and Access Management (IAM) API or through the GCP console. See Google Cloud Platform Data Collection Prerequisites for details.

    Client Email

    This is the email address and ID associated with the service account. The information will be populated automatically from the Service Account Key File. See Google Cloud Platform Data Collection Prerequisites for details.

    Client ID

    Connection Name

    Specify a name for the connection. This name will appear in the Saved Connections list. If a name is not specified, the Client ID is used.

    If the name is already in use or the connection exists, you are prompted to enter a new connection/account name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from entering a string that is more than 32 characters.

  6. Once your values are entered for all fields, verify your connection using Click to Verify Account Connection and Discover Project(s).
    • If the credentials are valid, you will be connected and authenticated. A timestamp of the successful validation is displayed.
    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. See Google Cloud Platform Data Collection Prerequisites for details on GCP account requirements.
    • All projects that are associated with the service account are listed In the Projects In Scope section.
  7. From the Projects In Scope section, select the projects to include in the audit. The newly discovered project has a New, Not Saved status.
    • If a project discovered with the service account has already been saved and scheduled for auditing within Densify(duplicate project realized), then the current status of the project will be displayed and the project is not selectable.
  8. Once the project(s) have been selected, click Save Connection.
    • An audit is scheduled for the project(s) associated with your saved connection, and the status for the project(s) is now Scheduled.
    • The tab is updated with the number of connections saved in brackets.
    • You will also see the saved connection listed in the Saved Connections section on the left side of the page.
    • The initial data collection audit will pick up the last 60 days of data, if available, and each daily audit will collect the last 24 hours of data.
  9. If you want to add another connection, click Add New Connection and follow steps 3 - 6 again to configure the connection.
  10. Note: If you have many projects, select the projects to be included and then click Save Connection Densify creates the required structures within the Densify database and initiates a 60-day audit to collect historical data for each of the selected projects. This takes some time and you cannot create another connection while the audit is in progress

  11. When you are done adding the connection, close the Public Cloud Connections page.

Once the connection has been verified data collection starts. You will be contacted, within one business day, once your data collection and analysis are complete. Subsequent data collection, analyses, and database refresh are scheduled to run on a nightly basis.

Editing a GCP Connection

From the Google Cloud tab in the Public Cloud Connections wizard, you can perform the following modifications:

  • Update the service account key (JSON) file — If your service account key file has changed, you can upload the new file and re-discover the projects associated with the service account.
  • Update the list of projects associated with the service account—If the projects associated with the service account have changed, you can re-discover the updated projects.
  • Projects in this list are from the service account, as of the latest data collection. You can deselect projects and these will be removed from data collection. If you deselect all projects associated with the GCP connection and save the connection, Densify will remove the connection.

Note: You cannot modify a connection (add/remove/modify projects) while data collectiom is running. Data collection runs when the connection is first created and then on a nightly basis.

  1. From the Public Cloud Connections wizard >  Google Cloud tab, select the connection you want to edit in the Saved Connection section on the left side of the page. The Accounts Information and Projects In Scope area is populated with the GCP connection details.
  2. If you need to update the service account JSON file, click Browse ... to select the updated JSON file and click Import to upload the new file. You will need to discover projects associated with this new service account key file, follow the next step to update the connection.
  3. If you want to re-discover and update projects associated with the current service account JSON file, click the Click to Refresh Account Connection and Discover Projects button.
    • If the credentials are valid, you will be connected and authenticated. A timestamp of the successful validation is displayed.
    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. See Google Cloud Platform Data Collection Prerequisites for details on GCP account requirements.
  4. From the Projects In Scope section, if you want to select additional projects to be audited, select the desired projects and click Save Connection.
  5. The Projects In Scope list will not be up to date if projects in the service account are modified after the last Click to Refresh Account Connection and Discover Projects action.

  6. From the Project In Scope section, if you want to remove existing audited projects, de-select the desired projects and click Save Connection.
  7. If you de-select all projects associated with the GCP connection and try to save the connection, Densify displays a message indicating the connection will be deleted if you save an empty project connection.

  8. When you are done modifying the connection, close the Public Cloud Connections wizard.

After the you modify the connection data collection, analysis, and database refresh are scheduled to run on a nightly basis.

Other Public Cloud Connections Features

These are common Public Cloud Connections features:

Accessing the Wizard from the Densify Console

Once you have collected data and are using the Densify Console you will not see the Densify landing page.

You can still access the Cloud Connections wizard from the Public CloudAdd Cloud Connection menu.

Editing a Connection

You can edit a saved connection directly from the Public Cloud Connections wizard.

You cannot edit the user name or credentials. If you want to change the credentials, you need to delete the existing connection and create a new one with the new credentials.

  • For AWS connections you can modify the External ID.
  • For Azure connections you can change the Subscriptions In Scope.
  • For Google Cloud connections you can change the and provide a new Service Account Key File. See Editing a GCP Connection, above.
  1. Select a cloud provider. Only connections for the selected provider are displayed in the Saved Connections list.
  2. Select the connection to be modified from the Saved Connections list. The current connection settings are displayed. Displayed settings change depending on the selected cloud provider:
  3. Review the existing information and make your changes, as required.
  4. Click Click to Verify Account Connection.
  5. Once the connection has been verified, click Save Connection to update the connection.

Reviewing a Connection

You can edit a saved connection directly from the Public Cloud Connections wizard. You can also view the status of the most recent connection.

You cannot edit the user name or equivalent settings. If you need to change the credentials, you will need to delete the existing connection and create a new one.

This feature can be useful to update passwords that expire regularly. You can also review the status of recent audits.

  1. Select a cloud provider. Only connection for the selected provider are displayed in the Saved Connections list.
  2. Select the connection to be modified from the Saved Connections list. The current connection settings are displayed. Displayed settings change depending on the selected cloud provider:
  3. The connection status for all accounts/subscriptions or projects contained in the selected connection are displayed,
  4. A scroll bar is displayed if there are too many accounts to display in the window.

  5. Review the existing information and update the password, if required.
  6. Click Click to Verify Account Connection to validate the new setting.
  7. Once the connection has been verified, click Save Connection to update the connection.

Deleting a Connection

You can delete a configured connection directly from the Public Cloud Connections wizard.

  1. Select the required Cloud provider tab. Only the connections for the selected tab are displayed in the Saved Connections
  2. Select the connection to be deleted from the Saved Connections list and click Delete Connection.

When you delete the connection the audit workspace folder and all of the contained audits are deleted. The account name is removed and is no longer displayed in the Saved Connections list.

Changing Credentials

You need to delete and recreate the AWS and Azure connections to change credentials. You can edit a GCP account to reference a new Service Account Key File. See Editing a GCP Connection, above.

When you delete a connection, the corresponding cloud environments are not deleted. They are no longer refreshed as there is no new data collection. Historical data for the account is maintained in the database. If you re-create a connection for the same account using the wizard, the new audits automatically link to the existing cloud environment and historical data.

Configuring Public Cloud Connections to use a Proxy Server

You can configure your public cloud connection to use a proxy server, for additional security. You need to provide the following information to [email protected]:

  • Proxy server name
  • Proxy server access port
  • User Name and encrypted password, if authentication is required

Next Steps

Once the connection has been created, you will be contacted within one business day to confirm your data has been loaded and analyzed. You can then do the following: