Enable Continuous Efficiency for Kubernetes Clusters With an Existing Delegate

Updated 1 day ago by Archana Singh

Harness CE monitors cloud costs of your Kubernetes clusters, namespaces, nodes, workloads, and labels. This topic describes how to enable Continuous Efficiency (CE) for Kubernetes.

CE is integrated with a Kubernetes cluster by using the Harness Kubernetes Delegate installed in the cluster, and the Harness Kubernetes Cloud Provider that uses the Delegate for authentication. For information on setting up and Kubernetes Delegate, see Kubernetes Quickstart and Connect to Your Target Kubernetes Platform.

You can use a single Harness Delegate to access multiple Kubernetes clusters.

To enable the Harness Kubernetes Delegate to monitor your cluster costs, the Service account you used to install and run the Harness Kubernetes Delegate will be granted a special ClusterRole to access resource metrics. This process is described in this topic.

In this topic:

Before You Begin

Prerequisites

  • Each Kubernetes cluster you want to monitor must have a Harness Delegate and Cloud Provider associated with it.
  • Before enabling CE for Kubernetes, you must ensure the utilization data for pods and nodes is available. To do so, perform the following steps:

Step 1: Install Kubernetes Metrics Server

Metrics Server must be running on the Kubernetes cluster where your Harness Kubernetes Delegate is installed.

Metrics Server is installed by default on GKE and AKS clusters, however you need to install it on the AWS EKS cluster.
  1. Metrics Server is a cluster-wide aggregator of resource usage data. It collects resource metrics from kubelets and exposes them in Kubernetes API server through Metrics API. For more information, see Installing the Kubernetes Metrics Server from AWS.

    To install metrics server on your EKS clusters, run the following command:

    kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml

Step 2: Provide CE Permissions to Harness Delegate

Bind the cluster-admin ClusterRole to a user account. Next, you will use this user account to create a ClusterRole and bind it to the Service account used by the Delegate.

  1. Bind a user account to the user in cluster-admin ClusterRole. You will use this user account to create a ClusterRole and bind it to the Harness Kubernetes Delegate Service account later.

    kubectl create clusterrolebinding cluster-admin-binding \
    --clusterrole cluster-admin \
    --user <firstname.lastname@example.com>
  2. Obtain the Service account name and namespace used by the Harness Kubernetes Delegate. By default, when you installed the Kubernetes Delegate, the following were used:

    name: default
    namespace: harness-delegate
    If you have changed these, obtain the new name and namespace.
  3. Download the ce-default-k8s-cluster-role.yaml file from Harness.

    The Subjects section of the ClusterRoleBinding is configured with the default Delegate Service account name (default) and namespace (harness-delegate).

    If you have changed these defaults, update the ce-default-k8s-cluster-role.yaml file before running it.
  4. Once you have downloaded the file, connect to your Kubernetes cluster and run the following command in your Kubernetes cluster:

    kubectl apply -f ce-default-k8s-cluster-role.yaml
  5. Verify that you have all the required permissions for the Service account using the following commands:
    kubectl auth can-i watch pods 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces
    kubectl auth can-i watch nodes 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces

    kubectl auth can-i get nodemetrics 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces
    kubectl auth can-i get podmetrics 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces
    Here is an example showing the commands and output using the default Delegate Service account name and namespace:

    $ kubectl auth can-i watch pods --as=system:serviceaccount:harness-delegate:default --all-namespaces
    yes
    $ kubectl auth can-i watch nodes --as=system:serviceaccount:harness-delegate:default --all-namespaces                                                                    
    yes
    $ kubectl auth can-i watch nodemetrics --as=system:serviceaccount:harness-delegate:default --all-namespaces                                                              
    yes
    $ kubectl auth can-i watch podmetrics --as=system:serviceaccount:harness-delegate:default --all-namespaces 
    yes

Step: Enable Continuous Efficiency

To enable CE in your cloud environment, you simply need to enable it on the Harness Kubernetes Cloud Provider that connects to your target cluster.

After enabling CE, it takes about 2-3 hours for the data to be available for viewing and analysis.
  1. In Continuous Efficiency, click Settings.
  2. Select the Kubernetes cluster for which you want to enable Continuous Efficiency.
  3. In Display Name, enter the name that will appear in CE Explorer to identify this cluster. Typically, this is the cluster name.
  4. In Cluster Details, select:
    1. Inherit from selected Delegate: (Recommended) Select this option if the Kubernetes cluster is the same cluster where the Harness delegate was installed. 
      1. Delegate Name: Select the Delegate. For information on adding Selectors to Delegates, see Delegate Installation.
    2. Enter manually: In this option, the Cloud Provider uses the credentials that you enter manually. The Delegate uses these credentials to send deployment tasks to the cluster. The Delegate can be outside or within the target cluster.
      Use this option, if you wish to use a single Delegate to access multiple Kubernetes clusters.
      1. Master Url: The Kubernetes master node URL. The easiest method to obtain the master URL is using kubectl:
        kubectl cluster-info
  5. Click Next.
  6. Select the checkbox Enable Continuous Efficiency and click Submit.

The Kubernetes Cloud Provider is now listed under Efficiency Enabled.

As noted earlier, after enabling CE, it takes about 2-3 hours for the data to be available for viewing and analysis.

Once CE has data, the cluster is listed in Cost Explorer. The cluster is identified by the Display Name you used in the Kubernetes Cloud Provider.

When you enable Kubernetes with CE, Kubernetes utilization events are automatically collected via the Harness Delegate. This information is used to show you historical cost data for your resources. The cost is calculated based on the publicly available catalog price information.

Troubleshooting

  1. If the Cloud Provider listed in Setup is listed with the following error message, you need to review the steps earlier in this topic.
    No Delegate has all the requisites to access the cluster <cluster-name>.
  2. If the Cloud Provider listed in Setup is listed with the following Invalid request error message, you need to download the ce-default-k8s-cluster-role.yaml file from Harness again. The Subjects section of the ClusterRoleBinding is configured with the default Delegate Service account name (default) and namespace (harness-delegate). If you have changed these defaults, update the ce-default-k8s-cluster-role.yaml file before running it. See Step 2: Provide CE Permissions to Harness Delegate.

Next Steps

How did we do?