Using CLI Plugins in Harness PCF Deployments

Updated 4 months ago by Michael Cretzman

Harness supports all Cloud Foundry plugins from the CF plugin marketplace, PCF Network, and in-house, and enables you to run and use them in Harness PCF Workflow steps.

Harness also includes first-class support for the App Autoscaler plugin, enabling you to create it as part of your Harness Workflow, bind it to your app, and enable or disable it as needed. Here is the App Autoscaler option as part of the App Setup command.

In this topic:

CF Plugin Usage Overview and Requirements

Harness runs CF plugins using the Workflow command CF Command. CF Command automatically sets the CF_PLUGIN_HOME directory, logs in (using the Harness PCF Cloud Provider), and runs the plugin using the script in CF Command.

Requirements for Running Plugins

To run plugins using CF Command, you must have the following:

CF CLI Installed on Harness Delegates

The CF CLI must be installed on the Harness Delegates used in deployment. This is a requirement for any PCF deployment with Harness.

The CF CLI can be installed on the Delegate(s) using a Delegate Profile script. For more information, see Delegate Profiles.

Here is a Delegate Profile example that installs the CF CLI:

sudo wget -O /etc/yum.repos.d/cloudfoundry-cli.repo https://packages.cloudfoundry.org/fedora/cloudfoundry-cli.repo
sudo yum -y install cf-cli

In Harness, click View Logs to see the successful installation:

Here is the log:

...
Installed:
cf-cli.x86_64 0:6.47.2-1

Complete!

Here is a Delegate Profile example using apt-get:

apt-get install wget
wget -q -O - https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | apt-key add -
echo "deb https://packages.cloudfoundry.org/debian stable main" | tee /etc/apt/sources.list.d/cloudfoundry-cli.list
apt-get update
apt-get install cf-cli

A single Delegate Profile can be used on all Delegates to ensure that any Delegates used have the CF CLI installed.

For more information, see Delegate Profiles and Installing the cf CLI from PCF.

Plugins Installed on Harness Delegates

The plugin you want to run must be installed on the Harness Delegates that CF Command will use. You can tag a Harness Delegate and then select the Tag in the CF Command, ensuring that the CF Command runs your plugin on a Harness Delegate with the plugin installed.

You can install the plugin on the Harness Delegate using the same Delegate Profile you use to install the CF CLI on the Delegate(s).

Here is an example installing the CF CLI and Create-Service-Push plugin:

sudo wget -O /etc/yum.repos.d/cloudfoundry-cli.repo https://packages.cloudfoundry.org/fedora/cloudfoundry-cli.repo
sudo yum -y install cf-cli

echo y | cf install-plugin -r CF-Community "Create-Service-Push"

If you are using the Kubernetes, ECS, or Helm Delegates, you can select the Profile when you download a new Delegate script. Typically, you will be using a Shell Script Delegate for PCF deployments. In that case, simply apply the Profile to each new Delegate:

Create-Service-Push Installed on Delegate
The Create-Service-Push plugin must be installed on the Delegate(s) to use the App AutoScaler plugin.

The Create-Service-Push plugin reads in a services' manifest.yml file, creates the services listed in it, and pushes an application. Create-Service-Push extends cf push.

If you want to create PCF services from the inline or remote manifest files set up in your Harness Service Manifests section, you need to have Create-Service-Push installed on the Delegate.

For example, you can see the cf create-service-push command used to run a plugin defined in the manifest here:

You can install the Create-Service-Push plugin in a Delegate Profile by itself:

echo y | cf install-plugin -r CF-Community "Create-Service-Push"

or in the Delegate Profile that installs the CF CLI:

sudo wget -O /etc/yum.repos.d/cloudfoundry-cli.repo https://packages.cloudfoundry.org/fedora/cloudfoundry-cli.repo
sudo yum -y install cf-cli

echo y | cf install-plugin -r CF-Community "Create-Service-Push"

Click View Logs to see the successful installation:

Searching CF-Community for plugin Create-Service-Push...
Plugin Create-Service-Push 1.3.1 found in: CF-Community
Attention: Plugins are binaries written by potentially untrusted authors.
Install and use plugins at your own risk.
Do you want to install the plugin Create-Service-Push? [yN]: y
Starting download of plugin binary from repository CF-Community...

0 B / 9.82 MiB [------------------------------------------------------] 0.00% 9.82 MiB / 9.82 MiB [==============================================] 100.00% 0sInstalling plugin Create-Service-Push...
OK

Plugin Create-Service-Push 1.3.1 successfully installed.

Running CF Plugins using the CF Command

Once the CF plugin has been installed on a Harness Delegate, you can simply add the CF Command to your Workflow to run the plugin.

  1. In your PCF Workflow, decide where you want to execute a CF CLI command. If you want to run a plugin, you will likely want to add the CF Command to the Setup section.
  2. click Add Command. Add Command appears.
  3. Click CF Command. CF Command appears.

As the commented-out text states, the CF Command will perform the login steps of using the CF CLI. So you do not need to include login credentials in CF Command. CF Command will use the credentials set up in your Harness PCF Cloud Provider.

Scripts

In Script, enter your CF CLI commands.

There are two built-in Harness PCF variables you can use to reference the manifest and vars files used by the plugin you want to run:

  • If you are using inline Manifest files, the variable ${service.manifest} refers to the folder containing your manifest files.

  • If you are using remote Manifest files via a Git repo, ${service.manifest} refers to the folder containing your manifest files and ${service.manifest.repoRoot} refers to the root folder of the repo.

You can use the variables together to point to different locations. For example, here the manifest.yml file is one folder and the vars.yml is located using a path from the repo root folder:

cf create-service-push --service-manifest ${service.manifest}/manifest.yml --no-push --vars-file ${service.manifest.repoRoot}/QA/vars.yml
cf plugins | grep autoscaling-apps

These variables appear when you type ${service in Script:

Environment Service Overrides, such as PCF Manifests Override, do not apply to or override the ${service.manifest} variable. The ${service.manifest} variable only looks in the Harness Service.

You can also use variables in your script to templatize paths to manifest files. For example, if your Workflow Environment were templatized (see Template a Workflow), you can use the Environment variable ${env.name} in your path, like this:

${service.manifest.repoRoot}/${env.name}/vars.yml

When the Workflow is deployed, the user will have to provide a name for the Environment to use. The same name will be substituted for ${env.name} in the path in your script.

This substitution can be useful if you have folder names in your remote Git repo that match Harness Environment names, such as QA and PROD. The same Workflow and CF Command can be used for both Environments and use manifest files in separate repo folders.

Delegate Tags

In order for the plugin in your script to execute, the Harness Delegate(s) running the script must have the plugin installed.

Unless all of your Harness Delegates have the plugin installed, you can refer to the specific Delegates with the plugin installed using Delegate Tags. Add the Delegate Tag(s) for the Delegates with the plugins installed.

If you do not add any Delegates Tags to the CF Command, when the CF Command runs, Harness will only use Delegates that have the CF CLI installed.

However, if you are running plugins in CF Command, Harness cannot know which Delegates have the plugins installed.

This is why the Delegate Tags setting ensures that CF Command only executes on Delegates that can run the plugins mentioned in the CF Command script.

App Autoscaler CLI Plugin

The App Autoscaler plugin has first-class support in Harness, enabling you to ensure app performance and control the cost of running apps.

The following diagram illustrates how you can define, bind, create, and use App Autoscaler with the PCF apps deployed by Harness.

If you are using the App Autoscaler plugin, then autoscaling is applied after the final phase of deployment.

Once all phases are completed and the number of old version instances has reached the desired number, then the final number of instances will be as configured as defined by the Autoscaler.

For example, if a deployment results in 4 new instances, but Autoscaler is set to min 8 and max 10, Harness will set the desired number of instances to the minimum value. So the total number of new instances is 8.

To use App Autoscaler, you must have the following requirements:

  1. The App Autoscaler plugin must be installed on the Delegate(s) that will execute PCF deployments. The steps in this section assume that the App Autoscaler plugin is installed on your Delegates.
    Because of limitations in the CF CLI, the best way to install the App Autoscaler plugin on the Delegate is the following:
    1. Download the release from the Pivotal App Autoscaler CLI Plugin page.
    2. Store the release in a repo in your network that can be accessed by the Harness Delegate. This will allow you to use cURL to copy the release to the Delegate host(s).
    3. Install the App Autoscaler plugin on your Delegates using a Delegate Profile.
      This profile will run each time the Delegate is restarted and the CF CLI cannot reinstall the plugin simply, so you must uninstall the plugin and then reinstall it in your Delegate Profile:
cf uninstall-plugin "App Autoscaler"
curl /path/to/release-in-repo
cf install-plugin local-path/binary

Click View Logs on the Delegate Profile to see the successful installation.

You can also choose to install the plugin manually on each Delegate using the steps provided by Pivotal, in Using the App Autoscaler CLI.

  1. Create the App Autoscaler service in your target Pivotal space.
    In this topic we will show you how to use your app manifest to create the App Autoscaler brokered service in your target Pivotal space.
 applications:
- name: Test
...
create-services:
- name: "myautoscaler"
broker: "app-autoscaler"
plan: "standard"
  1. Bind the App Autoscaler service to the app you are deploying using the services parameter in your manifest file.
 applications:
- name: Test
...
services:
- myautoscaler
create-services:
- name: "myautoscaler"
broker: "app-autoscaler"
plan: "standard"
  1. Configure the App Autoscaler service using a manifest file:
---
instance_limits:
min: 1
max: 2
rules:
- rule_type: "http_latency"
rule_sub_type: "avg_99th"
threshold:
min: 10
max: 20
scheduled_limit_changes:
- recurrence: 10
executes_at: "2032-01-01T00:00:00Z"
instance_limits:
min: 10
max: 20
  1. Create the App Autoscaler using the Create-Service-Push plugin in a CF Command.
  2. Select Use App Autoscaler Plugin in the App Setup step in your Workflow.

When you deploy your Workflow, Harness creates the App Autoscaler service according to the configuration you defined and binds it to the app you are deploying.

Use a New App Autoscaler Service

This section provides the steps for using Harness to configure and create the App Autoscaler service in your target Pivotal space, bind it to your app, and enable or disable it for the app your are deploying.

The steps listed here assume you have installed the App Autoscaler plugin on the Harness Delegate(s) that will deploy your app.

Inline manifest and configuration files are used in the following procedure, but the same process can be followed using remote files. For information on using remote files from a Git repo, see Remote Manifest and Variable Files.

To use a new App Autoscaler service, do the following:

  1. Define the App Autoscaler Service in Your Manifest File
  2. Bind the App Autoscaler Service to Your App
  3. Add Your App Autoscaler Manifest File
  4. Create the App Autoscaler Service Using CF Command
  5. Enable App Autoscaler in the App Setup Step
1. Define the App Autoscaler Service in Your Manifest File
  1. In your Harness PCF Service, select your manifest.yml file and click Edit.
  2. Add a create-services block that describes the App Autoscaler service you want to create:

    ...
    create-services:
    - name: "myautoscaler"
    broker: "app-autoscaler"
    plan: "standard"
    ...
    Now that the App Autoscaler service is defined in your manifest.yml, you can bind it to the app.
2. Bind the App Autoscaler Service to Your App
  1. In applications, add a services block with the name of the App Autoscaler service:
...
services:
- myautoscaler

create-services:
- name: "myautoscaler"
broker: "app-autoscaler"
plan: "standard"
...

For more information on services, see the Pivotal documentation.

Now that the App Autoscaler service is defined and bound to your app, you can add an App Autoscaler manifest file that configures the settings for the service.

  1. Click Save to save your manifest.yml file.
3. Add Your App Autoscaler Manifest File

App Autoscaler manifest files are described in Configure with a Manifest from Pivotal.

  1. In your Harness Service, click the options button on Files, and then click Add File.
  2. In Add File, enter the name of the App Autoscaler manifest file, such as autoscaler.yml. The file is added to the Manifests section.
    You can use any name for the App Autoscaler manifest file. Harness will determine which file to use for the service.
  3. Select the App Autoscaler manifest file and click Edit.
  4. Configure your rules, add instance limits, and set scheduled limit changes for the service. Here is an example:

     ---
    instance_limits:
    min: 1
    max: 2
    rules:
    - rule_type: "http_latency"
    rule_sub_type: "avg_99th"
    threshold:
    min: 10
    max: 20
    scheduled_limit_changes:
    - recurrence: 10
    executes_at: "2032-01-01T00:00:00Z"
    instance_limits:
    min: 10
    max: 20
  5. Click Save. The App Autoscaler manifest file is complete.

4. Create the App Autoscaler Service Using CF Command

To create the App Autoscaler Service, you add a CF Command to your Workflow that uses the Create-Service-Push plugin.

If the App Autoscaler service is already created and running in your target space, you can skip this step. When Harness deploys the app that is already bound to the App Autoscaler service, it will use the existing App Autoscaler service.

If the App Autoscaler service is already running, you do not need to remove the CF Command from your Workflow. Harness will check to see if the App Autoscaler service exists before creating it.

To create the App Autoscaler service using CF Command, do the following:

  1. Open the Harness Workflow that will deploy the Harness Service containing manifests for the app and its bound App Autoscaler service.
  2. In your Workflow, click Add Command anywhere before the App Setup command. Typically, this will be in Setup.
  3. In Add Command, click CF Command. CF Command appears.
  4. In Script, enter the following command:

    cf create-service-push --service-manifest ${service.manifest}/manifest.yml  --vars-file ${service.manifest}/vars.yml --no-push
    In this example, the app manifest file is named manifest.yml. You can replace manifest.yml with the name of your app manifest. You do not need to specify the name of the App Autoscaler manifest.

    For inline manifest files, you can use the ${service.manifest} variable. For remote manifest files stored in Git, you can use both the ${service.manifest} and ${service.manifest.repoRoot} variables. For more information, see Scripts.

    The --no-push parameter creates the services but does not push the app. The app will be pushed by the App Setup command. If you omit --no-push then App Setup will create a new revision of the app. For this reason, it is a best practice is always include --no-push.
  5. Ensure that you enter the Delegate Tags for the Delegates that have the CF CLI and Create-Service-Push plugin installed. For more information, see Delegate Tags.
  6. Click Submit. The CF Command is added.

Next you can enable the App Setup command to use the App Autoscaler service.

5. Enable App Autoscaler in the App Setup Step

The App Setup command in a Workflow includes a Use App Autoscaler Plugin setting so you can enable and disable autoscaling as needed.

Select Use App Autoscalar Plugin to enable the App Autoscaler service bound to your app.

When you deploy your Workflow, the App Autoscalar service is created using the command create-service app-autoscaler standard myautoscaler:

# ------------------------------------------ 

# CF_HOME value: /Users/johndoe/pcf/harness-delegate/repository/pcfartifacts/RR4DmcgKSzylo4enFUK5gw
# CF_PLUGIN_HOME value: /Users/johndoe
# Performing "login"
API endpoint: api.run.pivotal.io

Authenticating...
OK

Targeted org Harness

Targeted space AD00001863

API endpoint: https://api.run.pivotal.io (API version: 3.77.0)
User: john.doe@harness.io
Org: Harness
Space: AD00001863
# Login Successful
# Executing pcf plugin script :
Found Service Manifest File: /Users/johndoe/pcf/harness-delegate/repository/pcfartifacts/RR4DmcgKSzylo4enFUK5gw/manifests/deploy.yml
myautoscaler - will now be created as a brokered service.
Now Running CLI Command: create-service app-autoscaler standard myautoscaler
Creating service instance myautoscaler in org Harness / space AD00001863 as adwait.bhandare@harness.io...
OK
|
--no-push applied: Your application will not be pushed to CF ...
# Exit value =0

---------- PCF Run Plugin Command completed successfully

Use an Existing App Autoscaler Service

You might already have the App Autoscaler service running in your target space, and so some of the steps in Use a New App Autoscaler Service can be skipped.

If you already have the App Autoscaler service running in your target Pivotal space, then you can simply bind the service in the app manifest file using the services parameter and enable the Use App Autoscalar Plugin in App Setup. You do not need to set up the following:

  • You do not need create-services in the manifest.yml file for your app.
  • You do not need a manifest file for the App Autoscaler service.
  • You do not need to use the CF Command and cf create-service-push to create the App Autoscaler service.

Including any of these unnecessary components will not cause a problem. Harness automatically checks for an existing App Autoscaler service before creating a new service.

Plugin Directory

By default, the CF CLI stores plugins in $CF_HOME/.cf/plugins, which defaults to $HOME/.cf/plugins. For most cases, this location does not need to change.

To change the root directory of this path from $CF_HOME, set the CF_PLUGIN_HOME environment variable.

For example:

export CF_PLUGIN_HOME='<path to plugin home>'

You can set the CF_PLUGIN_HOME environment variable before you install the Delegate. This will ensure that the Delegate Profile that you use to install the CF CLI uses the new CF_PLUGIN_HOME.

For more information, see Changing the Plugin Directory from Pivotal.


How did we do?