Microsoft Azure Data Collection Prerequisites for a Service Principal

Microsoft Azure Data Collection Prerequisites for a Service Principal

#410010

Overview

A Service Principal is an instance of an application that is within your on-premises Active Directory (AD) and that is allowed to access one or more resources within your AD. You can delegate the minimum required permissions to the service principal so that it can be used to create a connection to Densify to collect both utilization and billing data.

Note: Azure Stack is not supported. This data collection method currently only supports Azure Cloud.

If you are using a standalone Azure Active Directory account (i.e. an Azure AD), then see Microsoft Azure Data Collection Prerequisites.

To learn more about the Azure prerequisites for a service principal, watch the video: Data Collection Prerequisites for Azure Using a Service Principal.

The Service Principal possesses the following characteristics:

  • An application that is created within your Active Directory. This application will be associated with the user that creates it.
  • A service principal is then created for that application;
  • The service principal is granted access to the Azure subscription.

The following procedure allows you to use a federated Azure Active Directory (AD) account to create a cloud connection. In this case a Service Principal is required to connect Densify to your Azure subscriptions.

Note: In this case "federated" indicates that your on-premise Active Directory is linked with your Azure Cloud's Active Directory so you can use your existing on-premise AD credentials to access your Azure portal.

Requirements to Create a Cloud Connection

To connect Densify to your Azure subscriptions, you need the following items to complete the audit setup:

Additionally, the application must be assigned the role of "Reader" for each of the subscriptions from which data will be collected. The Reader role is sufficient for both resource utilization and billing data.

If you using reservations, the application must also be assigned the role of "Reader" for each reservation order. See Working with Reservations for details on additional configuration.

Required Account Permissions

In order to create the account required for Densify, you must have admin/owner privileges for your Azure portal. Typically, your on-premise Active Directory is federated with the Azure AD. In this case, the admin access is likely provided from an on-premise Active Directory account (via federation) and will possess the required Azure admin privileges.

The user account/Service Principal to be used for data collection only requires the "Reader" role privileges to collectDensify both utilization and billing data.

Collecting Billing Data

When collecting billing data, no additional configuration is required within your Azure subscription. You need to create a cloud connection using the same information used for collection of utilization data, as listed above.

Densify collects and analyzes the data and then generates various cost optimization reports. You can review these reports through the Cloud Cost Intelligence console. See Viewing Your Public Cloud Billing Reports.

If you are using Azure reservations, see Working with Reservations for details of addtional configuration.

Limitation on Types of Subscriptions

Due to limitations when using the Microsoft Azure APIs, only the following types of subscriptions are collected:

  • Enterprise Enrollments;
  • Web Direct Subscriptions.

Currently the following subscriptions are not supported for billing optimization:

  • MS-AZR-0145P (CSP)
  • MS-AZR-0146P (CSP)
  • MS-AZR-159P (CSP)
  • MS-AZR-0036P (sponsored)
  • MS-AZR-0143P (sponsored)
  • MS-AZR-0015P (internal)
  • MS-AZR-0144P (DreamSpark)

The APIs are continually updated to support other types of Azure subscriptions. See Microsoft's Consumption API Overview for details.

Obtaining the Tenant ID/Directory ID

The tenant ID corresponds to the Azure Active Directory (AD).

  1. Login into your Azure account and click on Azure Active Directory > Properties .
  2. In the Properties pane, copy the Directory ID (e.g. 6c9190a7-bca6-4fcd-b35e-36378aadc). This corresponds to the Tenant ID.

Obtaining the Application ID and Client Secret

You now need to create a new application registration. This will be the service principal for Densify. You will need the Application ID and corresponding client secret (key), to complete the Densify connection.

  1. Click Azure Active Directory > App Registration.
  2. In the App Registration pane, click New registration.
  3. In the Create pane enter the following information:
    • The Name of the application (e.g. Densify_Connection).
    • Select who can access the application. Leave the default of "Single Tenant".
    • Select the Redirect URI (optional) as “Web” and specify a Sign-on URI (e.g. https://Densify.com).

    The Register button becomes available once you enter valid data.

  4. Click Register to create and register the application. This is the service principal that Densify will be use to collect data.
  5. Click the Owned applications tab and then select your newly created entry (Densify_Connection) to access the registered app dashboard.
  6. Copy the Application ID (e.g. 7d16fbf8-1fc3-4e08-b48a-626afc4cc). You need to copy this ID and save it to a location from which you can easily retrieve it. You will need the Application ID to create the Densify connection.
  7. Click Certificates and secrets to see the page.
  8. Click on New client secret to create a new key.
  9. In the Add a new client secret pane, enter a Description (e.g. DensifyKey) and an expiration period (e.g. 1 year, 2 years or never expires).
  10. Click Add and then copy the secret key Value (e.g. 1uXnI:WR4gYbkW3Z:Tq/jvsX1bkwyM6sSq+WpObL). Copy this key to a location from which you can easily retrieve it. You will need this key to create the Densify connection.

Assigning Access to Subscriptions

The application registered above, now needs access to each of your subscriptions. You need to assign the "Reader" role to the Application for each subscription being audited. Use the following process:

  1. Navigate to Subscriptions in the main menu. You may need to click on All services to see Subscriptions.
  2. Click on a subscription to select and open the configuration pane.
  3. Click Access Control (IAM).
  4. Click Add > Add role assignment.
  5. In the Add role assignment pane select the Role of "Reader".
  6. Ensure Assign Access to is set to "Azure AD user, group or service principal".
  7. Search for or scroll to locate the application (Densify_Connection), created above.
  8. Select your application. It will appear under Selected members.
  9. Click Save to save these changes.
  10. Repeat this process for each subscription to be included in data collection.

Once the account has been configured you can use the tenant ID, application ID and secret key to create the cloud connection as outlined in Using the Public Cloud Connections Wizard.

Note: If you using reservations, the application must also be assigned the role of "Reader" for each reservation order. See Working with Reservations for details on additional configuration.

Advanced Topics

Configuring the Collection of Memory Metrics

You can configure your VM so that memory metrics can be collected as part of the Densify audit. There are two ways to configure memory metrics for each VM depending on whether the VM is a classic V1 guest or a V2 guest and which OS the VM is running.

Note: Ensure you have the latest version of the diagnostic agent running on each VM.

Table: Versions of Azure VMs and Collection of Memory Metrics

Azure Version

Windows OS

Collected Memory Metric

Linux OS

Collected Memory Metric

Classic Portal

Method B

  • \Memory\Available Bytes

Method B

  • \Memory\UsedMemory
  • \Memory\PercentUsedMemory

Azure Resource Manager

Method A

  • \Memory\Committed Bytes
  • \Memory\Available Bytes

Method B

  • \Memory\UsedMemory
  • \Memory\PercentUsedMemory

Method A - Collection of Memory Metrics

Use the following procedure to configure memory metrics for Azure Resource Manager VMs that are configured with a Windows OS:

  1. Navigate to Virtual machines in the main menu.
  2. Click on the VM to be modified.
  3. In the monitoring section, click on Monitoring > Diagnostics and then select the Basic metrics checkbox.

By default, these metrics are turned on when you create a VM.

Method B - Collection of Memory Metrics

Use the following procedure to configure memory metrics for Azure Classic Portal VMs (Windows and Linux) and Azure Resource Manager VMs configured with a Linux OS:

  1. Navigate to Virtual machines in the main menu.
  2. Click on the VM to be modified.
  3. In the monitoring section, click on Monitoring > Diagnostics Settings and then select Performance Counters from the top menu.
  4. Select the CPU, Memory, Disk and Network metrics check boxes.

Working with Reservations

When you buy an Azure reservation, access is restricted to the user who bought it (e.g. user-adm@IT.on.microsoft.com) and the account administrator (e.g. IT@Densify.com). In order to collect data for this reservation, you will need to add Reader access to the reservation order.

Note Granting access to the Reservation only, is insufficient. Reader access must be granted to the reservation order.

When you give a user permission to manage a reservation, that does not give that user rights to manage the subscription. Allowing Densify to access the subscription does not also allow it to the reservation automatically.

To delegate Reader access for a reservation order:

  1. Login into your Azure portal
  2. Click All services and navigate to Reservations.
  3. In the Reservation pane you will see a list showing all the reservations for which you have the Owner or Reader role. Click a reservation link to open the configuration pane.
  4. Click Reservation order ID to select and open the Reservation order pane.
  5. Click Access Control (IAM) in the left side-menu.
  6. Select Add > Add role assignment
  7. In the Add role assignment pane select Reader. Reader provides the minimum required permissions.
  8. Ensure Assign Access to is set to “User, group, or service principal”
  9. Search or scroll to locate the application (Densify_Connection), that you created previously.
  10. Locate and select your application. It will appear under Selected members.
  11. Click Save to save your changes.
  12. Repeat this process for each reservation order to be included in Densify's data collection.

If the role assignment is done through the reservation order, then Densify can collect data for all reservations in the selected order. At the time of purchase, a reservation order has one Reservation; however, actions such as splitting, merging, obtaining a partial refund, or exchanging create new reservations under the reservation order.

Creating the Cloud Connection in Densify

Once all of the prerequisites are complete, you can create the cloud connection through the Cloud Connection wizard. See Using the Public Cloud Connections Wizard.

Modifying Your Azure Cloud Connection

When you create the Azure cloud connection for the first time, Densify discovers all of the subscriptions, associated with the user or service principal. Upon saving the connection it will schedule data collection from each of the discovered and selected subscriptions.

If subsequently, subscriptions are added, they will not be included in data collection. Additionally, subscriptions that are removed will continue to be included, resulting in wasted time and resources. To add new subscriptions or remove old ones, edit the cloud connection. See Reviewing and Editing a Connection.