Harness Delegate Overview

Updated 3 days ago by Michael Cretzman

The Harness Delegate is a service you run in your local network or VPC to connect all of your artifact, infrastructure, collaboration, verification and other providers with the Harness Manager.

As explained in Harness Key Concepts, when you set up Harness for the first time, you install a Harness Delegate in your target infrastructure (for example, Kubernetes cluster, ECS cluster, EC2 subnet, Pivotal Cloud Foundry space, etc). Once the Delegate is installed, you can set up the resources and model your release process.

Most importantly, the Delegate performs all deployment operations.

For all the topics related to the Harness Delegate, see Manage Harness Delegates.

In this article:

Limitations and Requirements

See Delegate Requirements and Limitations.

What Data does the Delegate Send to the Harness Manager?

The Delegate and the Harness Manager (via SaaS) establish a Secure WebSocket channel (WebSocket over TLS) to send new Delegate task event notifications (not the tasks themselves) and exchange connection heartbeats. In the case that the WebSocket connection is dropped, the Harness Delegate falls back to outbound-only, polling-based task fetch.

  • Heartbeat - The Delegate sends a heartbeat to let the Harness Manager that it is running.
  • Deployment data - The information from the API executions the Delegate performs are sent to the Manager for display in the Deployments page.
  • Time series and log data for Continuous Verification - The Delegate connects to the verification providers you have configured and sends their data to the Manager for display in Harness Continuous Verification.

Where do I Install the Delegate?

  • Evaluating Harness - When evaluating Harness, you might want to install the Delegate locally. Ensure that it has access to the artifact sources, deployment environments, and verification providers you want to use with Harness.
  • Development, QA, and Production - The Delegate should be installed behind your firewall and in the same VPC as the micro-services you are deploying. The Delegate must have access to the artifact servers, deployment environments, and cloud providers it needs.

Install the Harness Delegate

To install the Delegate:

  1. Sign into the Harness Manager.
  2. Download and install the Harness Delegate.
    1. Click Setup.
    2. Click Harness Delegates.
    3. Click Download Delegate, and then select your Delegate type.
      You can download the Delegate scripts, or copy and use the cURL commands to install the Delegate.
    To install using cURL, you enter a name for the Delegate and select a Profile (Primary is the default). Next click Copy Download Link. Connect to the machine or instance where you want to run the Delegate, and paste in the cURL command to download the Delegate.
    1. To use the Shell Script, unzip the file and run the start.sh script ($ ./start.sh).
    2. To use Docker, unzip the file and run the start script ($ ./launch-harness-delegate.sh). You will see the Docker image pull, for example:
    latest: Pulling from harness/delegate
    297061f60c36: Pull complete…
    Status: Downloaded newer image for harness/delegate:latest
    1. For Kubernetes, unzip the file and navigate into the extracted folder in a Terminal. Next, log into your cluster via the Terminal, and run the following commands (ensure you have kubectl installed and credentials to access your Kubernetes cluster):

      $ kubectl apply -f harness-delegate.yaml (installs or replaces the Harness Delegate)
      $ kubectl get pods -n harness-delegate (get pod names)
      $ kubectl logs <pod-name> -n harness-delegate -f (see startup logs)
      $ kubectl exec <pod-name> -n harness-delegate -it -- bash (run a shell in a pod)

      For example, with Google Cloud Platform, you can click the Connect button for your Kubernetes cluster, copy the Command-line access command, and the paste and run it in a Terminal running on the host where you downloaded the harness-delegate.yaml file. Once you are connected, you simply cd to the folder where harness-delegate.yaml is located and run:

      $ kubectl apply -f harness-delegate.yaml.
      If needed, edit the harness-delegate.yaml you unzipped to change the namespace, set proxy settings, or to enter a Delegate description.
  3. Once the Delegate is installed, the Delegate is listed in the Delegates page in a few moments.

The Delegates page provides a list of installed Delegates. The information displayed includes:

  • Hostname.
  • IP Address.
  • Status.
  • Last heartbeat. This is the last time the Delegate communicated with the Harness Manager. The Delegate pings the Manager once per minute.
  • Delegate version.

Installation Example: Google Cloud Platform

The following steps describe how to install the Kubernetes Delegate in a Kubernetes cluster in Google Cloud Platform. The steps assume you have Google Cloud SDK installed, as it is used to execute gcloud command-line commands. For more information on installing the Google Cloud SDK, see Install the Latest Cloud SDK version from Google.

A Kubernetes cluster in Google Cloud Platform has a variety of settings that are different from other vendors. For example, when creating your cluster in Google Cloud Platform, you might need to add more or less CPUs or RAM to your nodes than you would with another cloud vendor or an Internal cluster.
  1. Sign up for a free tier Google Cloud Platform account.
  2. In Google Cloud Platform, create a Kubernetes cluster.
    1. In Kubernetes Engine, click CREATE CLUSTER.
    2. Choose the Standard cluster template.
    3. Customize the template:
      Name: Enter the name for the cluster.
      Location type: Select Zonal for this example.
      Zone: Select the default Compute Engine compute zone, as the location type is Zonal.
    4. For Node pools, do the following:
      Number of nodes: Enter 3 nodes.
      Machine type: Enter 4vCPU, and click Customize. In Memory, enter 12GB. 8GB is for the Delegate. The remaining memory is for Kubernetes.
    5. Click the Advanced edit button.
    6. In the Edit node pool panel, below Size, click Enable autoscaling.
    7. Enter a Minimum of 3 and a Maximum of 8.
      When you are done, your setting will look something like this:
    8. Click Save.
    9. Click Create to create the cluster. It will take a few minutes to create the Kubernetes cluster.
  3. When the cluster is created, click the Connect button.
  4. In the Connect to the cluster dialog, copy the Command-line access command.
  5. On your local machine, open a Terminal, and paste the command, and press Enter. You are now connected to the Kubernetes cluster.

    By default, your cluster should be able to connect to the Internet. If you run into errors, ensure the GCE firewall isn’t blocking port 22 to your VMs. For more information, see SSH troubleshooting from Kubernetes.
    1. Let's quickly confirm that the cluster you created can connect to the Harness platform. Enter the following command:

      wget -p https://app.harness.io/ -O /dev/null
      A successful connection will display the following output:
      Resolving app.harness.io (app.harness.io)...
      Connecting to app.harness.io (app.harness.io)||:443... connected.
      HTTP request sent, awaiting response... 200 OK
  6. Download the Kubernetes Delegate from Harness.
    1. In Harness, click Setup, and then click Harness Delegates.
    2. Click Download Delegate and then click Kubernetes YAML.
    3. In the Delegate Setup dialog, enter a name for the Delegate, such as harness-sample-k8s-delegate, select a Profile (Primary is the default), and click Download. the YAML file is downloaded to your machine.
  7. Install the Delegate in your cluster.
    1. In the Terminal you used to connect to the Kubernetes cluster, navigate to the folder where you downloaded the Harness Delegate YAML file.
      For example, cd ~/Downloads.
    2. Extract the YAML file: tar -zxvf harness-delegate-kubernetes.tar.gz.
    3. Navigate to the harness-delegate folder that was created:
      cd harness-delegate-kubernetes
    4. Paste the following installation command into the Terminal and press enter:
      kubectl apply -f harness-delegate.yaml
      You will see the following output (this delegate is named k8s-test):
      namespace/harness-delegate created
      clusterrolebinding.rbac.authorization.k8s.io/harness-delegate-cluster-admin created
      statefulset.apps/k8s-test-vutpmk created
  8. Run this command to verify that the Delegate pod was created:
    kubectl get pods -n harness-delegate

    You will see output with the status Pending. The Pending status simply means that the cluster is still loading the pod.
  9. Wait a few moments for the cluster to finish loading the pod and for the Delegate to connect to Harness Manager.
  10. In Harness Manager, in the Harness Delegates page, the new Delegate will appear. You can refresh the page if you like.
StatefulSets is used for the the Kubernetes Delegate to ensure the same pod name is used each time a pod is scheduled.

You can also open a shell session to a Delegate pod using the following command:

kubectl exec -n harness-delegate <pod_name> -it -- /bin/bash

ECS Delegate

For detailed steps on setting up a Harness ECS Cluster Delegate, see 1 - Harness ECS Delegate in the AWS ECS Deployments guide.

Harness includes an ECS Task Spec Delegate that you can run as an ECS service in your ECS cluster. You simply download the ECS task definition from Harness and then install the Delegate using aws ecs CLI commands or using the ECS console.

Although it can help simply deployments, the Harness ECS Delegate service does not need to be installed in a cluster the same VPC as the target ECS cluster (the cluster where Harness will deploy your container and image).

When you download the ECS Task Spec Delegate you get a compressed TAR file. Extract the file and use the installation steps in the README or in see 1 - Harness ECS Delegate of the AWS ECS Deployments guide. A summary of the steps follows:

Here is an example of a task definition that creates a Harness ECS Delegate as a service in your ECS cluster:

"containerDefinitions": [
"portMappings": [
"hostPort": 8080,
"protocol": "tcp",
"containerPort": 8080
"cpu": 1,
"environment": [
"name": "ACCOUNT_ID",
"value": "xxxx"
"value": "xxxx"
"value": "delegateprod.txt"
"value": "https://app.harness.io/storage/wingsdelegates"
"name": "DELEGATE_TYPE",
"value": "ECS"
"value": "example-ecs-delegate"
"value": ""
"name": "DEPLOY_MODE",
"value": "KUBERNETES"
"value": "https://app.harness.io"
"name": "POLL_FOR_TASKS",
"value": "false"
"value": "watcherprod.txt"
"value": "https://app.harness.io/storage/wingswatchers"
"memory": 6144,
"image": "harness/delegate:latest",
"essential": true,
"hostname": "Harness-ECS-Delegate",
"name": "ecs-delegate"
"memory": "6144",
"requiresCompatibilities": [

"cpu": "1024",
"family": "harness-delegate-task-spec"

First, in a terminal, navigate to the extracted folder. Next, log into AWS via the terminal and register the ECS task definition using the Harness ECS Delegate Task Spec file ecs-task-spec.json:

$ aws ecs register-task-definition --cli-input-json file://ecs-task-spec.json

The command to create the ECS service for the ECS Delegate using the task definition is:

$ aws ecs create-service --service-name <delegate_name> --task-definition harness-delegate-task-spec --cluster <cluster_name> --desired-count 1

The cluster must have a minimum of 8GB of memory (t2.large minimum). Run the above command and the ECS Delegate will install as a service in your cluster. The Delegate will appear in the Setup > Harness Delegates page in a few minutes.

If you are using Fargate, follow the steps in the README.

Helm Delegate and Rancher

You can install and run the Helm Delegate natively in a Kubernetes cluster or using the Rancher Kubernetes management platform.

See Using the Helm Delegate.

Add Self-Signed Certificates for Delegate Connections

The Harness Delegate makes outbound connections to the resources you set up in Harness as Artifact Servers, Verification Providers, and so on. These platforms typically use public certificates that ship with the OS and Java runtime environments, and so no additional steps are needed.

But if you are using self-signed certificates, you will need to add them to the Delegate.

These certificates are stored in the JRE keystore on the hosts running the Delegate (or truststore for back-end application certificates), and you can import the certificates manually or using a Harness Delegate Profile.

For steps on setting up the profile, see Import a Certificate.

How Does Harness Manager Identify Delegates?

All Delegates are identified by your Harness account ID. But, depending on the type of Delegate, there are additional factors.

For Delegates running on virtual machines, such as the Shell Script and Docker Delegates running on an AWS EC2 instance, the Delegate is identified by the combination of Hostname and IP:

Therefore, if the hostname or IP changes on the VM, the Delegate cannot be identified by the Harness Manager. The IP used is the private IP. The Delegate connects to the Harness Manager, but the Harness Manager does not initiate a connection to the Delegate, and so the public IP address of the Delegate is not needed, typically.

For Kubernetes and ECS Delegates, the IP can change if a pod is rescheduled, for example. Consequently, Kubernetes Delegates are identified by a suffix using a unique six letter code in their Hostname (the first six letters that occur in your account ID):

ECS Delegates are identified by the Hostname entered when the ECS Delegate Task Spec is downloaded:

How Does Harness Manager Pick Delegates?

When a task is ready to be assigned, the Harness Manager first validates its lists of Delegates to see which Delegate should be assigned the task.

The following information describes how the Harness Manager validates and assigns tasks to a Delegate:

  • Heartbeats - Running Delegates send heartbeats to the Harness Manager in 1 minute intervals. If the Manager does not have a heartbeat for a Delegate when a task is ready to be assigned, it will not assign the task to that Delegate.
  • Selectors and Scoping - For more information, see Delegate Selectors and Delegate Scope.
  • Whitelisting - Once a Delegate has been validated for a task, it is whitelisted for that task and will likely be used again for that task. The whitelisting criteria is the URL associated with the task, such as a connection to a cloud platform, repo, or API. A Delegate is whitelisted for all tasks using that URL. The Time-To-Live (TTL) for the whitelisting is 6 hours, and the TTL is reset with each successful task validation.
  • Blacklisting - If a Delegate fails to perform a task that Delegate is blacklisted for that task and will not be tried again. TTL is 5 minutes. This is true if there is only one Delegate and even if the Delegate is selector for that task with a Selector, such as with a Shell Script command in a Workflow.

Delegate High Availability (HA)

You might need to install multiple Delegates depending on how many Continuous Delivery tasks you do concurrently, and on the compute resources you are providing to each Delegate. Typically, you will need one Delegate for every 300-500 service instances across your applications.

In addition to compute considerations, you can enable High Availability (HA) for Harness Delegates. HA simply involves installing multiple Delegates in your environment.

For example, in Kubernetes deployments, you can set up two Kubernetes Delegates, each in its own pod in the same target K8s cluster. Simply edit the Kubernetes Delegate spec you download from Harness, harness-kubernetes.yaml, to have multiple replicas:

apiVersion: apps/v1beta1
kind: StatefulSet
harness.io/app: harness-delegate
harness.io/account: xxxx
harness.io/name: test
name: test-zeaakf
namespace: harness-delegate
replicas: 2
harness.io/app: harness-delegate
If you want to install Kubernetes Delegates in separate clusters, do not use the same harness-kubernetes.yaml and name for both Delegates. Download a new Kubernetes YAML spec from Harness for each Delegate you want to install. This will avoid name conflicts.

For ECS deployments, set up two ECS Delegates as separate services in the same target ECS cluster. For Shell Script Delegates in AWS for example, set up two Delegate instances in the same target VPC.

In every case, the Delegates need to be identical in terms of permissions, keys, connectivity, etc.

With two or more Delegates running in the same target environment, HA is provided by default. One Delegate can go down without impacting Harness' ability to perform deployments. If you want more availability, you can set up three Delegates to handle the loss of two Delegates, and so on.

Two Delegates in different locations with different connectivity do not support HA. For example, if you have a Delegate in a Dev environment and another in a Prod environment, the Dev Delegate will not communicate with the Prod Delegate or vice versa. If either Delegate went down, Harness would not operate in their environment.

Delegate Proxy Settings

All of the Delegate settings include proxy settings you can use to change how the Delegate connects to the Harness Manager.

By default, the Harness Delegate uses HTTP and HTTPS in its Proxy Scheme settings.

Shell Script Proxy Settings

Run the setup-proxy.sh script included in the Delegate download and provide the proxy information.

  • Proxy URL - The syntax for the proxy URL must be of the form http[s]://<host>[:port].

The following settings are optional:

  • Username - Enter username if required.
  • Password - Enter password.
  • Suffixes - Enter a comma-separated list of suffixes that do not need the proxy. For example, .company.com,hostname,etc. Do not use leading wildcards.
  • Manager - Select Yes or No to have the Delegate bypass proxy settings to reach the Harness Manager.

The output will look something like this:


PROXY_MANAGER=false means that we selected Yes in response to the Bypass proxy settings to reach Harness manager? question.

PROXY_SCHEME is for the Delegate to reach the proxy server, not for Delegate connections beyond that.

Do not edit the setup-proxy.sh script. Simply run the script and provide the proxy information.

Docker Proxy Settings

The proxy settings are in the launch-harness-delegate.sh file:

-e NO_PROXY= \

Kubernetes Proxy Settings

The proxy settings are in the harness-delegate.yaml file:

- name: PROXY_HOST
value: ""
- name: PROXY_PORT
value: ""
value: ""
- name: NO_PROXY
value: ""
value: "true"
- name: PROXY_USER
name: doc-example-proxy
name: doc-example-proxy

The PROXY_MANAGER setting determines whether the Delegate bypasses proxy settings to reach the Harness Manager in the cloud. If you want to bypass, enter false.

Delegate Scope

By default, a Delegate will perform all connections you set up and processes you run. To limit, or scope, a Delegate, you can set specific boundaries and associations for each Delegate you run. You can scope a Delegate to:

  • A specific command.
  • Application.
  • Environment.
  • Environment type.
  • Infrastructure Definition.

Delegate Scope Example

You need to restrict access to a production environment. Without defining a scope, the Delegate scans all network ports to map out the Environment and performs checks to access the Artifact Servers and Cloud Providers it needs. By defining a scope, you can limit the Delegate to connect to only the specific Environments, Applications, and Infrastructure Definition it needs. 

Defining a scope is optional, but if you use scoping you can prevent the Delegate from constantly looking for what’s available or accessible.

You can apply multiple scopes to a Delegate.

Scope a Delegate

To scope a Delegate, do the following:

  1. On the Harness Delegates page, which lists all of the Delegates, select a Delegate and click Scope Limited To or Scope Excluded.
    The Delegate Scope dialog appears.
  2. Click New scope, and then click ADD SCOPE. The dialog changes to display the scope properties.
  3. Enter a name for the scope, and then select its properties, such as the command, Harness Application, Environment, and Infrastructure Definition to which the scope applies.

    You must pick at least one Delegate Scope property.
    If you clicked Scope Limited To, then the Delegate is only used for the properties selected.
    If you clicked Scope Excluded, then the Delegate is excluded from running for the properties selected.
  4. Click SUBMIT. The scope is added to the Delegate.

Best Practices and Notes

The following information will help you prevent many Delegate issues.

Only Use Scope Excluded

The most important decision when creating a Delegate Scope is selecting Scope Limited To or Scope Excluded.

Scope Limited To is rarely used by Harness users. Scoping one Delegate to the Production Environment Type and a second Delegate to the Non-Production Environment Type might appear logical, but your Harness deployments typically involve operations that do not fall into either of those Environment Types and, consequently, there is no Delegate scoped to perform them.

It is better to use Scope Excluded to exclude one Delegate from the Production Environment Type and the other Delegate from the Non-Production Environment Type. This way, either Delegate can still perform general operations outside of Environment Type.

Scope Options are ANDs

The fields in the Delegate Scope dialog narrow the scope of the Delegate's operations as you move down the dialog. For example, you can pick an operation in Command, and the limit its scope further by picking the Application and Environment.Think of using the Delegate Scope dialog fields as ANDs: Command AND Application AND Environment Type, and so on.

If you want to OR Delegate Scope settings, you need to make separate Delegate Scopes and apply them to the same Delegate.
Use a Display Name

Ensure that you enter a name in Display Name. This will help you and others quickly select the scope in the future.

Delegate Profiles

A Delegate Profile is mandatory. The Primary Profile is the default and contains no script. You can add a script to it, or create and apply new Profiles for your Delegate.

A Delegate Profile enables you to run a startup script on the host/container/pod for a Harness Delegate when the Delegate is installed. You can create a single Delegate Profile and apply it to multiple Delegates.

See the following Delegate Profile topics:

Delegate Selectors

Delegate Selectors were formerly called Tags. Now, Tags refers only to tagging Harness Application components as described in Using Tags.

When Harness makes a connection via its Delegates, it will select the best Delegate according to its history or round robin between delegates. To ensure a specific Delegate is used by a Harness entity, you can scope the delegate as explained in Delegate Scope, or you can add Selectors to Delegates and then reference the Selectors in commands and configurations.

For example, in a Delegate listing in Harness Delegates, click Edit next to Selectors.

In the Edit Selectors dialog, type in the Selector name, such as Local, and press Enter. Click SUBMIT. The Selector is added.

Elsewhere, such as in a Shell Script command in a Workflow, you can use the Selector to ensure that this Delegate is used when executing the command.

You must enter all of the Selectors that the Delegate you are targeting uses. If the Delegate has three Selectors, and you only enter one in Selectors, that Delegate might not be used.

The most common use of a Delegate Selector is in a Cloud Provider. For example, you can install a Delegate in an AWS VPC and select in with a name such as ecs-delegate.

When you set up an AWS Cloud Provider, you can select the Delegate installed in your AWS VPC using its Selector.

Now the AWS Cloud Provider will use the Delegate's credentials to connect with AWS.

Delegate Logs

The Delegate log is named delegate.log, typically with the date it was created in the name, such as delegate.2019-04-10.log.

The log is located in the same folder as the Delegate.

Kubernetes Delegate

To copy the delegate.log from the container running the Delegate, you can use the kubectl cp command. For example, if the Delegate pod name is harness-sample-k8s-delegate-yvcrcl-0 and the namespace is harness-delegate, the following command will copy the log to the Desktop of johndoe:

kubectl cp harness-delegate/harness-sample-k8s-delegate-yvcrcl-0:/opt/harness-delegate/delegate.log /Users/johndoe/Desktop

If you simply want to verify the location of the delegate.log, use the kubectl exec command to find the log. For example, if the Delegate pod name is harness-sample-k8s-delegate-yvcrcl-0 and the namespace is harness-delegate, the following command will display the contents of the container filesystem:

kubectl exec harness-sample-k8s-delegate-yvcrcl-0 --namespace=harness-delegate -- ls -la

This command will give you an output such as:

total 55300
drwxr-xr-x 1 root root 4096 Jun 17 14:21 .
drwxr-xr-x 1 root root 4096 Jun 10 06:34 ..
drwxr-xr-x 3 root root 4096 Jun 17 14:20 .cache
drwxr-xr-x 3 root root 4096 Jun 17 14:21 1.0.33801
drwxr-xr-x 6 root root 4096 Jun 17 14:21 client-tools
-rw-r--r-- 1 root root 384 Jun 17 14:21 config-delegate.yml
-rw-r--r-- 1 root root 339 Jun 17 14:20 config-watcher.yml
-rw-r--r-- 1 root root 10866281 Jun 17 18:59 delegate.log
-rwxr-xr-x 1 root root 6033 Jun 17 14:20 delegate.sh
drwxr-xr-x 6 uucp 143 4096 Oct 6 2018 jre1.8.0_191
-rwxr-xr-x 1 root root 3731 Jun 10 06:31 modify_scripts.sh
drwxr-xr-x 4 root root 4096 Jun 17 14:20 msg
-rw-r--r-- 1 root root 5466 Jun 17 18:58 mygclogfilename.gc
-rw-r--r-- 1 root root 256 Jun 17 14:20 nohup-watcher.out
-rw-r--r-- 1 root root 95 Jun 17 14:20 proxy.config
-rwxr-xr-x 1 root root 121 Sep 19 2018 run.sh
-rwxr--r-- 1 root root 2071 Jun 17 14:20 setup-proxy.sh
-rwxr--r-- 1 root root 9167 Jun 17 14:20 start.sh
-rwxr--r-- 1 root root 2263 Jun 17 14:20 stop.sh
-rw-r--r-- 1 root root 45188053 Jun 17 14:20 watcher.jar
-rw-r--r-- 1 root root 467054 Jun 17 18:59 watcher.log

The delegate.log file is listed. Next, you can SSH into the container:

kubectl exec -it harness-sample-k8s-delegate-yvcrcl-0 --namespace=harness-delegate -- /bin/bash

Next, you can copy the Delegate log to your local machine or use Bash commands such as cat and pipe to grep search the log. For step on searching logs, see Diagnose Issues.

Delegate Updates

The Delegate updates automatically. The Delegate installation also installs a Watcher program that checks the Harness cloud periodically for new versions.

Watcher checks regularly with the Harness cloud to see what published Delegate versions should be running. Watcher ensures there is exactly one Delegate process of each published version running.

If there is a published version that is not running, Watcher downloads the jar file for that version securely over HTTPS, installs it, and updates the Delegate connection to the Harness Manager. There is no downtime.


See Troubleshooting.

How did we do?