Using the Public Cloud Connections Wizard

Using the Public Cloud Connections Wizard

#380290

The Public Cloud Connections wizard guides you through the steps to create and schedule daily data collection from your public cloud instances. Collected configuration and workload data is then analyzed and used to recommend sizing and placement optimization opportunities.

The Public Cloud Connections wizard is accessible from the Densify Console's landing page or from the Public Cloud >  Add Cloud Connection dropdown menu. See Navigating Densify for details.

The following cloud platforms are supported through the Public Cloud Connections wizard:

Note: Collection of billing data is currently not supported for Google Cloud Platform.

You can also modify the connections once they have been created:

 

Additionally, you can collect and analyze data through the Densify API. Refer to the following use case for example of API data collection:

Configuring a Connection to AWS

You can connect to your AWS account with the following method:

  • IAM Role—An IAM cross-account role that establishes a trust relationship between AWS accounts. In order to create the connection you need the prerequisite information, including the external ID and Role ARN. See AWS Data Collection Prerequisites for an IAM Role for details on obtaining this information.

You need to create a separate cloud connection for resource utilization metrics and for the billing reports you want to collect. For example, if you need to audit both workload and billing data under the same account, you will need to create two connections using the same credentials.

Using the Public Cloud Connections Wizard—IAM Role

To learn more watch the video:

  1. On the Densify landing page, click Connect to your AWS accounts.
  2. or

    from the Densify Console, navigate to Public Cloud > Add Cloud Connection.

  3. Click on the Amazon Web Services tab.
  4. Select the connection type:
    • Resource Utilization Metrics— This connection type is for collecting CloudWatch data.
    • Billing— This connection type is for collecting billing data. The collection of billing data must be enabled separately. Contact your Cloud Advisor to enable this feature.
  5. Select Connect Using: "IAM Role".
  6. The Densify Account ID is predefined. This value must match the value you entered when you created the role in AWS.
  7. Enter the AWS-specific connection parameters, as listed in the table below. Refer to AWS Data Collection Prerequisites for an IAM Role for details on creating the accounts and obtaining the credentials.
  8. Table: AWS Connection Parameters - IAM Role

    Field

    Description

    External ID

    The external ID specified for Densify, when you created the IAM role in AWS.

    If you need to edit or review a saved connection, for security reasons, you will need to re-enter the external ID.

    Role ARN

    The Amazon Resource Name (ARN) for IAM role that you created in AWS.

    Note: The External ID and the Role ARN are encrypted and are not displayed as plain text in log files.

  9. Once your AWS connection parameters are entered, verify your connection. Click Click to Verify Account Connection.
    • If the credentials are valid, you will be connected and authenticated. Once the account is verified, Densify discovers the AWS account ID and displays the ID number. See Table: AWS Connection Parameters - Billing below.
    • For billing connections, in addition to the discovery of the AWS account ID, the linked accounts connected to the payer account, CUR report configuration, and S3 bucket(s) are also discovered and validated. See Table: AWS Connection Parameters - Billing below.
    • Note: The number of instances returned is a high level estimate obtained when Densify verifies the connectivity. It may not match the number of instances discovered during more detailed, scheduled data collection.

    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. See AWS Data Collection Prerequisites for an IAM Role for details.

    The Save Connection button is disabled until the connection is successfully verified (for both resource utilization and billing connections, in both new and edit modes.

  10. When account verification is successful the following additional information is displayed:
  11. Table: AWS Connection Parameters - Billing

    Field

    Description

    AWS Account ID

    The AWS account ID is automatically discovered based on the validated access key ID and the secret access key combination. This parameter is automatically populated after verification and cannot be modified.

    Connection Name

    The connection name is used to identify the public cloud connection in Densify. This name will appear in the Saved Connections list and based on the connection type, it will appear under the corresponding Resource Utilization Metrics or Billing heading.

    The connection name is initially set to the AWS Account ID. You can modify the connection name to something meaningful for quick identification. The connection name must be unique within the AWS connection type section, so if the name is already in use, you are prompted to enter a new connection name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from editing the Connection Name to a string that is more than 32 characters.

    Billing Report

    When creating billing connections, a list of available Cost and Usage Reports (CUR) is displayed.

  12. After you have specified the connection name and/or selected the billing report, click Save Connection.
  13. The connection status, at the bottom of the page, will be updated. You will also see the connection listed in the Saved Connections section, under the Resource Utilization Metrics heading for a configuration/workload connection or under the Billing heading for a billing connection.
  14. For resource utilization data, the initial audit collects the last 60 days of data, if available, and each daily audit collects the last 24 hours of data. For billing data, one month of historical data is collected, if available. This value is configurable. Contact your Cloud Advisor for details.

  15. If you want to add another connection, click Add New Connection and follow steps 3 to 7 again to configure the connection.
  16. Close the Public Cloud Connections page once you have completed creating all your AWS connections.

Once the connection has been verified data collection starts. Your Cloud Advisor will contact you, within one business day, once data collection has completed and the cloud environments are created. Subsequent data collection, analyses, and database refresh are scheduled to run on a nightly basis.

Configuring a Connection to Microsoft Azure

To learn more, watch the video,

There are currently two methods for creating an Azure connection:

Click on the Microsoft Azure tab to create a connection to your Azure Cloud Computing environments. You will enter your Connection Name and an account user name and password in order to collect configuration, workload or billing data from your hosted infrastructure.

You need to create a separate cloud connection for resource utilization metrics and for the billing reports you want to collect. For example, if you need to audit both workload and billing data under the same account, you will need to create two connections using the same credentials.

Your information is verified and the connection can then be saved and scheduled to run regularly. In order to create the connection you need the relevant account information. See Microsoft Azure Data Collection Prerequisites for details on the obtaining the required credentials to enable data collection.

Using the Public Cloud Connections Wizard

  1. On the Densify landing page, click Connect.
  2. Click on the Microsoft Azure tab.
  3. Click the appropriate radio button to select the type of connection to create. You can collect resource
  4. Select the connection type:
    • Resource Utilization Metrics— This connection type is for collecting utilization data.
    • Billing— This connection type is for collecting billing data.
  5. Enter the Azure-specific connection parameters. Refer to one of the following topics to obtain the required information:
  6. Table: Account Information - User Credentials

    Field

    Description

    Connection Name

    Specify a name for the connection. This name will appear in the Saved Connections list. If a name is not specified, the account number is used.

    If the name is already in use or the connection exists, you are prompted to enter a new connection/account name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from editing the Connection Name to a string that is more than 32 characters.

    The initial data collection audit will pick up the last 60 days of data, if available, and each daily audit will collect the last 24 hours of data.

    User Name

    Enter the user name associated with your Azure account.

    Password

    Enter the user's password.

    Table: Account Information - Service Principal

    Field

    Description

    Application ID

    Specif y the Application ID/Service Principal. Both the ID and the corresponding key are provided when you create the application through your Azure portal. 

    Secret Key

    Enter the key corresponding to your application/service principal. This is called the Client Secret in the Azure portal interface.

    Tenant ID

    The tenant ID corresponds to the local Active Directory.

    Connection Name

    Specify a name for the connection. This name will appear in the Saved Connections list. If a name is not specified, the Application ID is used.

    If the name is already in use or the connection exists, you are prompted to enter a new connection/account name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from editing the Connection Name to a string that is more than 32 characters.

    The initial data collection audit will pick up the last 60 days of data, if available, and each daily audit will collect the last 24 hours of data.

  7. Once your information is entered for all fields, click Click to Verify Account Connection to validate the credentials.
    • If the credentials are valid, you will be connected and authenticated. Once the account is verified, all subscriptions that are associated with the account or Application ID are listed. Select the subscriptions to include in the audit.
    • For billing connections, you can select the subscriptions for which you want to collect billing data.
    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. The user account/Service Principal to be used for data collection only requires the "Reader" role privileges to collectDensify both utilization and billing data. See Microsoft Azure Data Collection Prerequisites for details.
  8. Once the connection has been verified, click Save Connection.
  9. The connection status, at the bottom of the page, will be updated. You will also see the connection listed in the Saved Connections section, under the Resource Utilization Metrics heading for a configuration/workload connection or under the Billing heading for a billing connection.
  10. For resource utilization data, the initial audit collects the last 60 days of data, if available, and each daily audit collects the last 24 hours of data. For billing data, one month of historical data is collected, if available. This value is configurable. Contact your Cloud Advisor for details.

  11. Click Add New Connection, to add another connection.

Once the connection has been verified data collection is scheduled.

Your Cloud Advisor will contact you, within one business day, once data collection has completed.

Note: If you create another connection, you must ensure subscriptions are not included in more than one cloud connection.

Configuring a Connection to Google Cloud Platform 

To learn more watch the video:

Click the Google Cloud Platform (GCP) tab to configure a GCP connection. You will need to provide a Connection Name and a Service Account Key File in order to collect configuration and workload data from GCP. See Google Cloud Platform Data Collection Prerequisites for details on obtaining the required credentials and configuring your GCP project to enable data collection. After your credentials are verified, you can save the connection and schedule data collection to run regularly.

Adding a GCP Connection

  1. On the Densify landing page, click Connect to your GCP Projects. The Public Cloud Connections wizard is displayed.
  2. Click on the Google Cloud Platform tab.
  3. Enter the GCP-specific account information. Refer to Google Cloud Platform Data Collection Prerequisites for details on creating the accounts and obtaining the credentials.
  4. Table: Account Information

    Field

    Description

    Connection Name

    Specify a name for the connection. This name will appear in the Saved Connections list. If a name is not specified, the account number is used.

    If the name is already in use or the connection exists, you are prompted to enter a new connection/account name.

    Note: The Connection Name is limited to 32 characters. The Connection Wizard prevents you from editing the Connection Name to a string that is more than 32 characters.

    The initial data collection audit will pick up the last 60 days of data, if available, and each daily audit will collect the last 24 hours of data.

    Service Account Key File (JSON)

    Specify the JSON file that contains the service accounts keys. This file is created using the Google Cloud Identity and Access Management (IAM) API or through the GCP console. See Google Cloud Platform Data Collection Prerequisites for details.

    Client Email

    This is the email address and ID associated with the service account. The information will be populated automatically from the Service Account Key File. See Google Cloud Platform Data Collection Prerequisites for details.

    Client ID

  5. Once your values are entered for all fields, verify your connection: click Click to Verify Account Connection and Discover Project(s).
    • If the credentials are valid, you will be connected and authenticated. A timestamp of the successful validation is displayed.
    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. See Google Cloud Platform Data Collection Prerequisites for details on GCP account requirements.
    • In the Projects In Scope section, all projects that are associated with the service account are listed.
  6. From the Projects In Scope section, select the projects to include in the audit (the newly discovered project has a New, Not Saved status).
    • If a project discovered with the service account has already been saved and scheduled for auditing within Densify(duplicate project realized), then the current status of the project will be displayed and the project is not selectable.
  7. Once the project(s) have been selected, click Save Connection.
    • An audit is scheduled for the project(s) associated with your saved connection, and the status for the project(s) is now Scheduled.
    • The Google Cloud Platform tab is updated with the number of connections saved in brackets.
    • You will also see the saved connection listed in the Saved Connections section on the left side of the page.
    • The initial data collection audit will pick up the last 60 days of data, if available, and each daily audit will collect the last 24 hours of data.
  8. If you want to add another connection, click Add New Connection and follow steps 3 - 6 again to configure the connection.
  9. Note: If you have many projects, select the projects to be included and then click Save Connection Densify creates the required structures within the Densify database and initiates a 60-day audit to collect historical data for each of the selected projects. This takes some time and you cannot create another connection while the audit is in progress

  10. When you are done adding the connection, close the Public Cloud Connections page.

Once the connection has been verified data collection starts. Your Cloud Advisor will contact you, within one business day, once data collection has completed and the cloud environments are created. Subsequent data collection, analyses, and database refresh are scheduled to run on a nightly basis.

Editing a GCP Connection

From the Google Cloud Platform (GCP) tab in the Public Cloud Connections wizard, you can perform the following modifications:

  • Update the service account key (JSON) file — If your service account key file has changed, you can upload the new file and re-discover the projects associated with the service account.
  • Update the list of projects associated with the service account — If the projects associated with the service account has changed, you can re-discover the updated projects.
  • Select existing projects to be audited — For the projects listed in the Projects In Scope area, you can select the projects to be audited (note that the existing Projects In Scope list might not be up to date if projects in the service account has been modified after the last project discovery action).
  • Deselect projects to be audited — You can deselect projects to be audited. If you deselect all projects associated with the GCP connection and save the connection, the system will remove the connection with no projects.

Note: You cannot modify a connection (add/remove/modify projects) while data collectiom is running. Data collection runs when the connection is first created and then on a nightly basis.

  1. From the Public Cloud Connections wizard >  Google Cloud Platform tab, select the connection you want to edit in the Saved Connection section on the left side of the page. The Accounts Information and Projects In Scope area is populated with the GCP connection details.
  2. If you need to update the service account JSON file, click Browse ... to select the updated JSON file and click Import to upload the new file. You will need to discover projects associated with this new service account key file, follow the next step to update the connection.
  3. If you want to re-discover and update projects associated with the current service account JSON file, click the Click to Refresh Account Connection and Discover Projects button.
    • If the credentials are valid, you will be connected and authenticated. A timestamp of the successful validation is displayed.
    • If the credentials cannot be validated, then review the displayed error message and correct your credentials. It is possible that the user account does not have the required permissions. See Google Cloud Platform Data Collection Prerequisites for details on GCP account requirements.
    • In the Projects In Scope section, all projects that are associated with the service account are listed.
  4. From the Projects In Scope section, if you want to select additional projects to be audited, select the desired projects and click Save Connection.
    • Keep in mind that the Projects In Scope list will not be up to date if projects in the service account are modified after the last Click to Refresh Account Connection and Discover Projects action.
  5. From the Project In Scope section, if you want to remove existing audited projects, de-select the desired projects and click Save Connection.
    • If you de-select all projects associated with the GCP connection and try to save the connection, the system will display a message informing you that the connection will be deleted if you save an empty project connection.
  6. When you are done modifying the connection, close the Public Cloud Connections wizard.

After the you modify the connection data collection, analysis, and database refresh are scheduled to run on a nightly basis.

Other Public Cloud Connections Features

These are common Public Cloud Connections features:

Accessing the Wizard from the Densify Console

Once you have collected data and are using the Densify Console you will not see the Densify landing page.

You can still access the Cloud Connections wizard from the Public CloudAdd Cloud Connection menu.

Once the reporting database (RDB) has been updated, the data will be displayed in your Densify Console reports. When you create a cloud connection through this wizard the RDB populate task is created and scheduled to run nightly, after data is collected and analyzed.

Reviewing and Editing a Connection

You can edit a saved connection directly from the Public Cloud Connections wizard. You can also view the status of the most recent connection.

You cannot edit the user name or equivalent settings. If you need to change the credentials, you will need to create a new connection. You can update the password, verify the connection and then save your changes

This feature can be useful to update passwords that expire regularly. You can also review the status of recent audits.

  1. Select a cloud provider. Only connection for the selected provider are displayed in the Saved Connections list.
  2. Select the connection to be modified from the Saved Connections list. The current connection settings are displayed. Displayed settings change depending on the selected cloud provider:
  3. The connection status for all accounts/subscriptions or projects contained in the selected connection are displayed,
  4. A scroll bar is displayed if there are too many accounts to display in the window.

  5. Review the existing information and update the password, if required.
  6. Click Click to Verify Account Connection to validate the new setting.
  7. Once the connection has been verified, click Save Connection to update the connection.

Deleting a Connection

You can delete a configured connection directly from the Public Cloud Connections wizard.

  1. Select the required Cloud provider tab. Only the connections for the selected tab are displayed in the Saved Connections
  2. Select the connection to be deleted from the Saved Connections list and click Delete Connection.

When you delete the connection the audit workspace folder and all of the contained audits are deleted. The account name is removed and is no longer displayed in the Saved Connections list.

The corresponding cloud environments are not deleted. They are no longer refreshed as there is no new data collection. Historical data for the account is maintained in the database. If you re-create a connection for the same AWS account using the wizard, the new audits automatically link to the existing cloud environment and historical data.

Configuring Public Cloud Connections to use a Proxy Server

You can configure your public cloud connection to use a proxy server, for additional security. You need to provide the following information to your Cloud Advisor:

  • Proxy server name
  • Proxy server access port
  • User Name and encrypted password, if authentication is required

Next Steps

Once the connection has been created, you will be contacted by your Cloud Advisor within one business day to confirm your data has been loaded and analyzed. You can do the following: