Set Up Cost Visibility for Kubernetes Using an Existing Delegate

Updated 1 month ago by Archana Singh

Harness Continuous Efficiency (CE) monitors cloud costs of your Kubernetes clusters, namespaces, nodes, workloads, and labels. This topic describes how to connect your Kubernetes to CE using an existing Harness Delegate.

In this topic:

Before You Begin

Review: Is Harness Delegate Running in Your Cluster?

Before connecting your Kubernetes cluster to Harness CE, review the following:

  • Provide CE Permissions to Harness Delegate: If the Delegate is already running in the Kubernetes cluster that you want to monitor, the Service account you used to install and run the Harness Kubernetes Delegate is granted a special ClusterRole for accessing the resource metrics. For more information, see Provide CE Permissions to Harness Delegate.

  • Use Single Harness Delegate to Access Multiple Kubernetes Clusters: You can use a single Harness Delegate to access multiple Kubernetes clusters. To do so, you need to enter specific credentials manually. For more information, see Enter manually.

Prerequisites

Step 1: Install Kubernetes Metrics Server

Metrics Server must be running on the Kubernetes cluster where your Harness Kubernetes Delegate is installed.

Metrics Server is installed by default on GKE and AKS clusters, however you need to install it on the AWS EKS cluster.
  1. Metrics Server is a cluster-wide aggregator of resource usage data. It collects resource metrics from kubelets and exposes them in Kubernetes API server through Metrics API. For more information, see Installing the Kubernetes Metrics Server from AWS.

    To install metrics server on your EKS clusters, run the following command:

    kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml

Step 2: Provide CE Permissions to Harness Delegate

Bind the cluster-admin ClusterRole to a user account. Next, you will use this user account to create a ClusterRole and bind it to the Service account used by the Delegate.

  1. Bind a user account to the user in cluster-admin ClusterRole. You will use this user account to create a ClusterRole and bind it to the Harness Kubernetes Delegate Service account later.

    kubectl create clusterrolebinding cluster-admin-binding \
    --clusterrole cluster-admin \
    --user <firstname.lastname@example.com>
  2. Obtain the Service account name and namespace used by the Harness Kubernetes Delegate. By default, when you installed the Kubernetes Delegate, the following were used:

    name: default
    namespace: harness-delegate
    If you have changed these, obtain the new name and namespace.
  3. Download the ce-default-k8s-cluster-role.yaml file from Harness.

    The Subjects section of the ClusterRoleBinding is configured with the default Delegate Service account name (default) and namespace (harness-delegate).

    If you have changed these defaults, update the ce-default-k8s-cluster-role.yaml file before running it.
  4. Once you have downloaded the file, connect to your Kubernetes cluster and run the following command in your Kubernetes cluster:

    kubectl apply -f ce-default-k8s-cluster-role.yaml
  5. Verify that you have all the required permissions for the Service account using the following commands:
    kubectl auth can-i watch pods 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces
    kubectl auth can-i watch nodes 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces

    kubectl auth can-i get nodemetrics 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces
    kubectl auth can-i get podmetrics 
    --as=system:serviceaccount:<your-namespace>:<your-service-account> 
    --all-namespaces
    Here is an example showing the commands and output using the default Delegate Service account name and namespace:

    $ kubectl auth can-i watch pods --as=system:serviceaccount:harness-delegate:default --all-namespaces
    yes
    $ kubectl auth can-i watch nodes --as=system:serviceaccount:harness-delegate:default --all-namespaces                                                                    
    yes
    $ kubectl auth can-i watch nodemetrics --as=system:serviceaccount:harness-delegate:default --all-namespaces                                                              
    yes
    $ kubectl auth can-i watch podmetrics --as=system:serviceaccount:harness-delegate:default --all-namespaces 
    yes

Step: Enable Continuous Efficiency

To enable CE in your cloud environment, you simply need to enable it on the Harness Kubernetes Cloud Provider that connects to your target cluster.

Once you enable CE, for the first cluster the data is available within a few minutes for viewing and analysis. However, you will not see the idle cost because of the lack of utilization data. CE generates the last 30 days of the cost data based on the first events.

From the second cluster onwards, it takes about 2–3 hours for the data to be available for viewing and analysis.
  1. In Continuous Efficiency, click Settings.
  2. Select the Kubernetes cluster for which you want to enable Continuous Efficiency.
  3. In Display Name, enter the name that will appear in CE Explorer to identify this cluster. Typically, this is the cluster name.
  4. In Cluster Details, select:
    1. Inherit from selected Delegate: (Recommended) Select this option if the Kubernetes cluster is the same cluster where the Harness delegate was installed. 
      1. Delegate Name: Select the Delegate.
    2. Enter manually: In this option, the Cloud Provider uses the credentials that you enter manually. The Delegate uses these credentials to send deployment tasks to the cluster. The Delegate can be outside or within the target cluster.
      Use this option, if you wish to use a single Delegate to access multiple Kubernetes clusters. To do so, you need to enter specific credentials manually. For more information, see Enter manually.
      1. Master Url: The Kubernetes master node URL. The easiest method to obtain the master URL is using kubectl:
        kubectl cluster-info
  5. Click Next.
  6. Select the checkbox Enable Continuous Efficiency and click Submit.

The Kubernetes Cloud Provider is now listed under Efficiency Enabled. Once CE has data, the cluster is listed in Cost Explorer. The cluster is identified by the Display Name you used in the Kubernetes Cloud Provider.

When you enable Kubernetes with CE, Kubernetes utilization events are automatically collected via the Harness Delegate. This information is used to show you historical cost data for your resources. The cost is calculated based on the publicly available catalog price information.

Troubleshooting

  1. If the Cloud Provider listed in Setup is listed with the following error message, you need to review the steps earlier in this topic.
    No Delegate has all the requisites to access the cluster <cluster-name>.
  2. If the Cloud Provider listed in Setup is listed with the following Invalid request error message, you need to download the ce-default-k8s-cluster-role.yaml file from Harness again. The Subjects section of the ClusterRoleBinding is configured with the default Delegate Service account name (default) and namespace (harness-delegate). If you have changed these defaults, update the ce-default-k8s-cluster-role.yaml file before running it. See Step 2: Provide CE Permissions to Harness Delegate.

Next Steps

How did we do?