Workflows

Workflows define the deployment orchestration steps, including how a Service is deployed, verified, rolled back, and more.

Some of the common Workflow types are Canary, Blue/Green, Rolling. An Application might have different deployment orchestration steps for different Environments, each managed in a Workflow.

For information about how workflows are used in a Kubernetes deployment, see Kubernetes and Harness FAQ.

Intended Audience

  • DevOps

Before You Begin

Add a Workflow

The following steps cover the common workflow setup.

To add a workflow, do the following:

  1. Click Setup.
  2. Click the application where you want to put the workflow.
  3. Click Workflows.
  4. Click Add Workflow. The Workflow dialog appears.
  5. Give your workflow a name and description that tells users its purpose. For example, if the workflow takes a service with a Docker artifact and deploys it to a Kubernetes environment in GCP, you might name the workflow docker-to-k8s-GCP.
  6. In Workflow Type, select the type of workflow you want to perform. For a summary of the types, see Workflow Types below.
  7. In Environment, select the environment where you want to deploy the service. Select from the environments you added in Add an Environment.
  8. In Service, select the service you want to deploy.
  9. In Service infrastructure, select one of the service infrastructures you configured for the environment you selected.
  10. Click SUBMIT. The workflow is created. Here is an example of a Basic Deployment.

If this is a Basic Deployment, you might need to update one step before using the workflow. For example, in the figure above, the Upgrade Containers step requires attention. In the case of other deployment types, there might be additional steps to configure.

Workflow Types

The following table lists the workflow types available when creating a workflow.

Workflow Type

Description

Basic

A Basic deployment selects nodes and installs a service.

Multi-Service

A Multi-Service uses one or more phases composed of separate steps.

Canary

A Canary deployment rolls out a new app version to small sets of users in separate phases, tests and verifies it at each phase, gradually rolling it out to your entire infrastructure.

Build

A Build deployment simply collects artifacts.

Rolling

A Rolling deployment lets you gradually roll out your deployment, enabling and disabling services as necessary.

Blue/Green

In a Blue/Green deployment, network traffic to your service/artifact is routed between two identical environments called blue (staging) and green (production). Both environments run simultaneously, containing different versions or the service/artifact.

When you submit, the workflow will display the steps needed to perform the workflow type you selected.

Workflow Steps and Commands

Different workflow deployment types involve different workflow steps. Each type of step, such as Deploy Containers or Verify Service, has specific commands available to it.

To add commands to steps, do the following:

  1. Under the workflow step, click Add Command.
    For setup or pre-deployment steps, such as Setup Container, you will see Cloud, Flow controls, and other general commands:
  2. To use a template, click Add Command, and then click From Template Library. For more information, see Use Templates.
  3. If a step has multiple commands, you can arrange them. Mouseover the command and then click the down arrow.
  4. You can control how multiple commands under a step are executed. Click the vertical ellipsis next to a step with multiple commands. The execution drop-down appears.

Barriers

When deploying interdependent services such as micro-services or a large and complicated application there might be a need to coordinate the timing of the different components' deployments. A common example is the need to verify a group of services only after all the services are deployed successfully.

Harness workflows address this scenario using barriers. Barriers allow you to synchronize different workflows in your pipeline, and control the flow of your deployment systematically.

Barriers only have an effect when two or more Workflows use the same barrier name and are executed in parallel in a Pipeline. When executed in parallel, both Workflows will both cross the barrier at the same time.

If a workflow fails before reaching its barrier point, the workflow signals the other workflows that have the same barrier, and the other workflows will react as if they failed as well. At that point, each workflow will act according to its Failure Strategy.

To use a barrier, do the following:

  1. In your workflow, click Add Command.
  2. Under Flow Control, click Barrier. The Barrier dialog appears.
  3. In Identifier, enter a name for the barrier. This name must be identical to all the other related barriers in the other workflows you want to impact.
  4. In Timeout, enter the timeout period, in milliseconds. For example, 600000 milliseconds is 10 minutes. The timeout period determines how long each workflow with a barrier must wait for the other workflows to reach their barrier point. When the timeouts expire, it is considered a deployment failure.
You can have multiple barriers in a workflow. Ensure the identifier string for each related barrier across the different workflows matches.

Workflow Phases

In multi-phase deployments, such as a Canary Deployment, workflow steps are grouped into phases. Here is a Canary Workflow before the phases and sub-steps are added:

To add a phase to a multi-phase deployment, do the following:

  1. In Deployment Phases, click Add Phase. The Workflow Phase dialog appears.
  2. In Service, select the service to use for this phase.
  3. In Service Infrastructure, select the service infrastructure to use for this phase.
  4. If you want to override a variable that is defined in the service you selected, in Service Variable Overrides, click Add. Enter the name of the variable to override and the override value.
  5. Click SUBMIT. The new phase workflow appears. Complete the phase workflow as you would any other workflow. You can also define the Rollback Steps for this phase.
  6. When you are done, click the name of the workflow in the breadcrumbs to return to the workflow overview and see the phase added to Deployment Phases.

Verify Steps

One of the most important steps in a workflow is verification. A verification step ties your workflow and your verification provider to Harness Continuous Verification features.

In order to obtain the names of the host(s) or container(s) where your service is deployed, the Verify Steps should be defined in your workflow after you have run at least one successful deployment.

To set up a verification step, do the following:

  1. In a workflow, in Verify Service, click Add Verification. The Add Command dialog appears.
  2. In Verifications, click the verification provider connected to the service you are deploying in this workflow. The configuration dialog for the service appears.
  3. Fill out the verification service dialog and click SUBMIT. The Verify Service step displays the verification provider.
In a multi-phase deployment, the verification steps are in the sub-steps within each phase.

You can read more about the different verification integrations in Continuous Verification and Verification Providers.

Rollback Steps

You define the steps of a workflow rollback in Rollback Steps. Typically you want to rollback failed containers and container orchestration setup. You can also verify that the rollback has restored the last working version of your service.

For Docker, Kubernetes, AWS CodeDeploy, and Lambda deployments, Harness rolls back the deployment to the state that received the new code. In case of a JAR, WAR, RPM, TAR, ZIP and other deployments, Harness provides default rollback steps (Disable, Stop, Deploy, Enable, Wrap-up). You can add custom commands in cases where you need to customize the rollback procedure.

If you deploy a workflow and choose the Abort option during the running deployment, the Rollback Steps for the workflow are not executed. Abort stops the deployment execution without rollback or clean up. To execute the Rollback Steps, click the Rollback button.

To set up rollback steps, do the following:

  1. In a workflow, click Rollback Steps to see the default steps. Here is an example of the default rollback steps in a workflow that deploys a Docker image to a Kubernetes cluster:

Notification Strategy

By default, when a workflow fails, the Account Administrator is notified. You can specify a notification strategy for a Workflow (or Workflow phase in Canary or Multi-Service Workflow) that sends notifications using different criteria.

To add a notification strategy, do the following:

  1. In a Workflow, click Notification Strategy. The default notification step appears.
  2. To edit the default strategy, click the pencil icon next to the strategy item.
  3. To add a new notification strategy, click Add Notification Strategy. The Notification Strategy dialog appears.

    Field

    Description

    Condition

    Click the drop-down to select the condition that will execute the notification strategy (Failure, Success, Paused). Click all the conditions that apply.

    Scope

    Select Workflow or Workflow Phase (for Canary or Multi-Service) as the scope for the condition(s) you selected.

    User Group

    Select the User Group to notify when the condition is met within the scope. For information on setting up the notification channels for a User Group, see User Notifications and Alert Settings.

    You can also enter ${ to see a list of variables that can be used. These can be variables specified at the Service, Environment, and Workflow level.

    For example, you could create a Service variable named OpsAdmin and use that variable in User Group. Next, you could overwrite that variable in Environment with a new value, and then use that Environment variable in User Group. Or, you could create a Workflow variable named StageOpsAdmin and use that in Notification Group.

    For the Workflow scope, you may select Environment (${environmentVariable.varName}) and Workflow (${workflow.variables.varName}) variables.

    For the Workflow Phase scope, you may select Service (${serviceVariable.varName}), Environment, and Workflow variables.

Failure Strategy

The Failure Strategy defines how your Workflow handles different failure conditions. For example, if you are deploying a Service to a cluster of 100 nodes, what percentage of connectivity errors would you allow before failing the deployment?

To add a failure strategy, do the following:

  1. In a workflow, click Failure Strategy. The default failure strategy appears.

    The default failure strategy is to fail the workflow if there is any application error, and to rollback the workflow execution. You can modify the default strategy or additional strategies.
  2. Click Add Failure Strategy. The Failure Strategy dialog appears.

The dialog has the following fields:

  • Failure - Select the type of error, such as Verification, Application, etc.
  • Scope - Select the scope of the strategy. Select Workflow or Workflow Phase.
  • Action - Select the action for Harness to take in the event of a failure, such as a retry or a rollback.

Add Workflow Variables

You can set variables in the Workflow Variables section of your workflow and use them in the workflow step commands and settings. For information on variables and expressions, see Variables and Expressions in Harness and Passing Variables into Workflows and Pipelines from Triggers.

To use workflow variables, do the following:

  1. In a workflow, click the pencil icon next to Workflow Variables. The Workflow Variables dialog appears.
    The Workflow Variables dialog has the following fields.

    Field

    Description

    Variable Name

    Enter a name for the variable. When the variable is referenced elsewhere in you Harness application, the variable name is used.

    Type

    Select the data type, such as Text, Number, or Email.

    Allowed Values

    Enter a comma-separated list of values that users can select. The list will appear as a dropdown menu when the Workflow is deployed.

    Default Value

    Enter a value for the variable. A value is not mandatory.

    Required

    Select this option to enforce that a value for the variable is provided before the workflow is executed.

    Fixed

    Select this option if the value of the variable specified here must not be changed.

    Description

    Provide a description of the variable that lets others know its purpose and requirements.

  2. In a command, use your variable by typing a dollar sign ($) and the first letter of your variable. Harness will load matching variable names. The syntax for variable names is ${workflow.variables.name}. For example, if you created a variable named Url, the variable name is ${workflow.variables.Url}.
    When you deploy the workflow, by itself or as part of a pipeline, the variables are displayed in the workflow execution step. if the variables require values, you will enter the values when you add the workflow to a pipeline in the Stage dialog or in the New Deployment dialog when you deploy the workflow.

Template a Workflow

You can turn a workflow into a workflow template ("templatize it") by using variables for important settings such as Environment, Service, and Service Infrastructure. When the workflow is deployed, the user must provide values for the settings you have defined as variables.

When you turn a workflow into a template, if the workflow still retains references to the entities it was originally created with (such as a service), attempts to delete these entities will result in an error because the template is still using them. You will need to remove the references from the template before you can delete them.

To turn a workflow into a template, do the following:

  1. In a workflow, click the vertical ellipsis next to the Deploy button, and click Edit. The Workflow dialog appears. The workflow settings that may be turned into variables have a [T] button in their fields.
  2. For each setting you want to turn into a variable, click the [T] button in its field. The field values are replaced by variables.


    You can pass in variables from triggers to set values for these workflow variables. For more information, see Passing Variables into Workflows and Pipelines from Triggers.
  3. Click SUBMIT. The new variables are displayed under Workflow Variables.
  4. To see how the workflow variables are used, click Deploy. The Start New Deployment dialog appears, displaying the variables you created in the Workflow Variables section.

When this deployment is run, users can select from the options for each setting from the Value drop-down. Now the workflow is a template that can be used by multiple services.

In the Workflows page, workflow templates are identified with a template icon.

The template option for workflow settings fields is available in the main Workflow dialog in some Workflow Types, whereas in multi-phase deployment types, it will be available in the sub-step workflow dialogs for each phase.

Templatized Workflows in Pipelines

Once you have templatized a workflow, you can use it in multiple stages of a pipeline.

For example, you can templatize the Environment and Service Infrastructure of a workflow, and then use the same workflow for both the QA and Production stages of a pipeline. When you add the workflow to each stage, you simply provide QA and Production-specific values for Environment and Service Infrastructure variables.

Workflow Variables in QA Stage of Pipeline

Workflow Variables in Production Stage of Pipeline

Deploy a Workflow

Typically, deployments are performed in Pipelines, which are collections of one or more workflows. You can deploy individual workflows within the Workflow Overflow.

To deploy a workflow, do the following:

  1. In a workflow, click the Deploy button.
  2. Provide values for any required variables. These are workflow variables created in the Workflow Variables section of the workflow.
  3. Complete the Start New Deployment dialog and click SUBMIT. The workflow is run according to the workflow steps.
  4. After a workflow is deployed, its deployment information is available on the Workflows page.
  5. Click a Timestamp for a workflow to view a deployment.
For information about using Triggers to deploy Workflows, see Triggers and Triggers and Queued Workflows.

Abort or Rollback a Running Deployment

If you deploy a workflow and choose the Abort option during the running deployment, the Rollback Steps for the workflow are not executed. Abort stops the deployment execution without rollback or clean up. To execute the Rollback Steps, click the Rollback button.

Abort Button

Rollback Button

Clone a Workflow

You can use a workflow as a template for future workflows by cloning the workflow. When you clone a workflow, you can select what application to clone the workflow into. Cloning a workflow can increase workflow development rapidly, and ensure consistency in workflows across applications.

To clone a workflow, do the following:

  1. In the Workflows page, or in the page for an individual workflow, click the vertical ellipsis ⋮.
  2. Click Clone. The Clone Workflow dialog appears.
  3. Give the workflow a new name and description.
  4. In Target Application, select the application where you want the workflow cloned. If you do not select an application, the workflow is cloned into its current application.
    The environment, service infrastructure, and nodes used by the workflow are not cloned into the target application. You will need to set these up in the target application.
  5. In Service Mapping, select the service in the target application to use with the cloned workflow.
  6. Click SUBMIT.
  7. Navigate to the target application and click the workflow. It will have the -clone suffix.
  8. Add the environment, service infrastructure, and nodes needed for the workflow.

Configure as YAML

You can configure a workflow using YAML. You can view the YAML using the main Code editor as described in Configuration as Code, or you can jump directly to the YAML of a specific workflow in the Workflows page.

To configure a workflow as code, do the following:

  1. In the Workflows page, click the code icon </>. The code editor appears, displaying your workflow YAML.
  2. Modify the YAML of the workflow as needed, and then click Save. If you like, you can verify your change in the Harness interface.

Troubleshooting Workflows

The most common error with workflows occurs when an environment's connection to a workflow is changed in such a way as to render it inoperable for a deployment of that workflow. The error is displayed in the Workflows and Continuous Deployment pages:

This error can occur in the following circumstances:

  • A user changes the environment service infrastructure. For example, the user might remove a service infrastructure or change its Deployment Type or Cloud Provider so that the workflow can no longer use the environment to deploy its service(s).
  • A user changes the environment in a workflow. If you open the Workflow dialog and, in Environment, you select a different environment, then a service infrastructure must also be selected in Service Infrastructure.
  • The user clones a workflow to a different application that uses services that cannot deploy to the service infrastructure in that environment.

As a result of these changes, the workflow has no functional service infrastructure specified, and you must select a new service infrastructure for each impacted workflow/phase.

Next Steps

Add a Pipeline


How did we do?