Containers Data Forwarder with Densify Operator

Containers Data Forwarder with Densify Operator

#410220

The Data Forwarder is an on-demand container that collects your container environment data from Kubernetes via Prometheus and forwards that data to Densify for analysis.

This topic shows you how to configure and deploy the Data Forwarder using the Densify Operator, which is currently supported on Red Hat Openshift. If you do not want to use the Densify Operator on Openshift, you can still deploy the Data Forwarder with YAML files. See Containers Data Forwarder with Authenticated Prometheus for YAML file deployment instructions.

Contact your Cloud Advisor for Densify credentials and to enable your Densify instance with container optimization after the Data Forwarder is deployed.

Prerequisites

The following items are necessary for Densify container data collection and optimization.

  1. Densify account— Contact Densify for details of your subscription or sign up for a free trial.
  2. Kubernetes or OpenShift must be deployed.
    • kubelet with cAdvisor is required to provide workload data to Densify
  3. kube-state-metrics—This service monitors the Kubernetes API server and generates metrics from the various objects inside the individual Kubernetes components. This service provides orchestration and cluster level metrics such as deployments, pod metrics, resource reservation, etc. The collected metrics allow Densify to get a complete picture of how your containers are setup. i.e. Replica Sets, Deployments, Pod and Container Labels.
  4. Prometheus—Provides the monitoring/data aggregation layer. It must be deployed and configured to collect kube-state-metrics and cAdvisor/kubelet metrics.

The following items are not mandatory but provide additional environment information for Densify's container optimization analysis.

  1. Node Exporter—Collects data about the nodes, on which the containers are running. This provides the classical host-related metrics such as CPU, mem, network, etc.
  2. openshift-state-metrics—These metrics provide additional details for OpenShift-specific items such as Cluster Resource Quotas (CRQ).

Contact your Cloud Advisor or support@Densify.com for configuration details.

Note: The Data Forwarder is only supported on Linux OS and AMD64 architecture.

Deploying Data Forwarder for Data Collection

To deploy the Data Forwarder, you need to do the following from the Red Hat Openshift Container Platform console:

  1. Install the Densify Operator as an admin user. See Installing the Densify Operator.
  2. Create the Densify Data Forwarder instance. See Deploying the Data Forwarder.

  3. Review the Data Forwarder pod processes, logs, cron job, and ensure that container metric data files were sent to your Densify instance. See Reviewing the Data Forwarder Pod.

Installing the Densify Operator

To install an Operator, you will need admin privileges to the Red Hat Openshift Container Platform. Perform the following steps as an admin user:

  1. From the Red Hat Openshift Container Platform console, navigate to OperatorsOperatorHubMonitoring from the left menu.
  2. Select the Densify Operator tile. The Densify Operator details panel is displayed.
  3. Click Install. The Install Operator page is displayed.
  4. Select the following configuration values on the Install Operator page:
  5. Configuration

    Select Value

    Upgrade Channel

    v1.0

    Install Mode

    All namespace on the cluster (default)

    Installed Namespace

    Select the namespace, or leave as:

    openshift-operators

    Approve Strategy

    Automatic

  6. Click Install. The Installed Operators page is displayed with the Densify Operator installation status.

Note: After the Densify Operator is installed, other users with operator access can see the Densify Operator under the Installed Operators page.

Deploying the Data Forwarder

Red Hat Openshift Container Platform users are able to deploy the Densify Data Forwarder once the Densify Operator is installed.

  1. From the Red Hat Openshift Container Platform console, navigate to OperatorsInstalled Operators in the left menu.
  2. Click the Densify Operator link.
  3. From the Densify Operator's Details tab, a description of the operator, prerequisites, basic installation instructions, configurations, contact and documentation links, and cluster version details are displayed.

  4. From the Densify Operator tile in the Details tab, click Create Instance.
  5. The Create Densify page is displayed.

  6. Configure the Densify Operator (i.e. Densify Data Forwarder) parameters in either the Form View or the YAML View. Both views have identical configuration parameters. Review the Table: Required Connectivity Parameters for the Data Forwarder table for a list of parameter descriptions required. If you want to change the default value of additional parameters, add the additional parameters in the Form View.

  7. Click Create.

  8. This action deploys and schedules the Densify Data Forwarder pod for data collection. This operation takes you back to the Densify Operator page, where the operator object is displayed.

  9. Click the operator object link. The Densify Operator Overview page is displayed with conditions and pod status.
  10. You can click on the Resources tab to view the config map used for this pod.

Reviewing the Data Forwarder Pod

Once the Densify Operator instance has been created, you can review the Data Forwarder pod status and check if your Densify instance has received the container metric data files.

  1. From the Red Hat Openshift Container Platform console, navigate to WorkloadsPods in the left menu.
  2. Details and status of pods are listed on this page. The name of the pod is a combination of name specified in the Create Instance page and the name identified in the config map.

  3. Click on the Densify pod link to review further details.
    • Click on the Details tab to review pod memory, CPU, disk, and network I/O usage.
    • Click on the YAML tab to see the yaml files used for this pod.
    • Click on the Logs tab to review the pod processing logs. The number of metric data files sent to your Densify instance should be displayed in the logs.
  4. From the Red Hat Openshift Container Platform console, navigate to WorkloadsCron Jobs in the left menu.
  5. Details and schedule of cron jobs are listed on this page. The name of the Data Forwarder cron job displayed on this page is a combination of name specified in the Create Instance page and the name identified in the config map.

When the Data Forwarder pod completes successfully, contact your Cloud Advisor to enable your Densify instance with container optimization.