Pipelines

Pipelines define your release process using multiple Workflows and approvals in sequential and/or parallel stages. Pipelines can involve multiple Services, Environments, and Workflows. A Pipeline can be triggered either manually or automatically (see Add a Trigger).

This topic covers the following:

Before You Begin

Create a Pipeline

The following procedure creates a simple pipeline using a single service and a single workflow.

To create a pipeline, do the following:

  1. Click Setup.
  2. Click the application with a service you want to deploy.
  3. Click Pipelines. The Pipelines page appears.
  4. Click Add Pipeline. The Add Pipeline dialog appears.
  5. Enter a name and description for your pipeline and click SUBMIT. The first stage dialog appears. It has the following fields.

    Field

    Description

    Editable Title

    You can add a title to your stage. For example, Prepare. This will be visible in the Pipeline Overview.

    Execution Step

    Select this option to execute this stage when the pipeline runs.

    Approval Step

    Select this option to require approval before the next stage executes. You can use Harness User Groups, a Custom Shell Script, or Jira.

    Step Name

    Name the step in this stage. This name acts as a subheading to the stage name.

    Execution Workflow

    Select the workflow to execute in this pipeline. The workflows in your application are available.

    Execute in parallel with previous step

    Select this option to execute steps in parallel.

    Disable step

    Select this option to disable this step in this stage. This allows you to control which stages and steps in the deployment pipeline are executed.

  6. Click SUBMIT. The pipeline stage and its step are added to the pipeline.
  7. To add another stage to the pipeline, click the plus icon. A new stage dialog appears.

Workflow Variables in Pipelines

Once you have templatized a workflow, you can use it in multiple stages of a pipeline. For example, you can templatize the Environment and Service Infrastructure of a workflow, and then use the same workflow for both the QA and Production stages of a pipeline. When you add the workflow to each stage, you simply provide QA and Production-specific values for Environment and Service Infrastructure variables.

For more information, see Templatized Workflows in Pipelines.

Deploy a Pipeline

Once you have set up a pipeline, you can deploy it, running all the stages within the pipeline. Before running a pipeline, verify that all the artifact servers and cloud providers are available to the accounts you used to set them up.

You can also deploy any workflow, whether it is within a pipeline or not. For more information, see Add a Workflow.

To deploy a pipeline, do the following:

  1. In your application, click a pipeline. The Pipeline Overview appears.
  2. In the pipeline title bar, click Deploy. The Start New Deployment dialog appears.
    When you set up the pipeline stage(s), you picked the workflow(s) the pipeline will execute. The workflow you selected is linked to a service. The Start New Deployment dialog is configured with the services linked to the workflows included in the pipeline. Here is an example of a pipeline linked to three workflows, each with a separate service.
  3. For each service in the pipeline, select Build/Version that the pipeline will deploy.
  4. To run your pipeline, click SUBMIT. The pipeline is deployed and the deployment steps are displayed in the Deployment Detail page.
    Click each phase in the process to see logs and other details of the process.

Abort or Rollback a Running Deployment

If you deploy a pipeline and choose the Abort option during the running deployment, the Rollback Steps for the workflow(s) in the pipeline are not executed. Abort stops the deployment execution without rollback or clean up. To execute the Rollback Steps, click the Rollback button.

Abort Button

Rollback Button

Pipeline Example

In most deployment processes there are multiple stages and approvals. Harness enables you to quickly configure the common three stage deployment pipeline (Dev, Stage, Production) along with the approval steps. Here is an example of a pipeline involving the common deployment process.

In this pipeline, the Dev and Stage workflows are each completed with an approval step where DevOps can verify that the stage has completed successfully without rollback before approving the next stage. Finally, with all stages successful and approved, the pipeline completes with a production release. The production release is also verified and rolled back if necessary.

Jira-based Approvals

For details on Jira integration with Harness, see Jira Integration.

You can add approval steps in both Workflows and Pipelines, and use Jira issue workflow statuses for approval and rejection criteria.

Here is an example of the workflow statuses for a Jira issue:

Here is a Workflow Jira approval step using the same Jira statuses for criteria:

You can add a Jira approval step in a Pipeline. Here is a Pipeline containing two Workflows and a Jira Approval step.

The first Workflow includes a Jira step that creates a new Jira issue, as described in Jira Integration:

The Variable Name is Jiravar and the Scope is Pipeline. This will enable the output variable to be used in any stage of the Pipeline. The output variable uses the issueId property and so the resulting variable reference is ${Jiravar.issueId}. You can reference the Jira issue anywhere in the Pipeline using ${Jiravar.issueId}.

Next, let's add a Pipeline Approval step that uses ${Jiravar.issueId}.

In a Pipeline, click the Insert a Stage (+) button to add the Approval step:

The new stage dialog appears. Click Approval Step to see the approval settings.

The Approval Step has the following fields:

  • Ticketing System - For Jira, select Jira.
  • Jira Connector - Select the Jira account you want to use by selecting the Collaboration Provider you added for the account, as described in Add Jira as a Collaboration Provider.
  • Project - Select the Jira project containing the Jira issue you will use for approval.
  • Key/Issue ID - Enter the output variable for a Jira issue created in a Workflow, such as ${Jiravar.issueId}. Although we are demonstrating approval using the ${Jiravar.issueId} variable, you can also simply enter the Jira Key/Issue ID for any Jira issue in the Jira project.
  • Timeout - Enter how long Harness should wait for the approval or rejection before failing the deployment.
  • Approved Status Criteria - The Field Name, Operator, and Value drop-down menus are populated from the Jira project you selected. Define the approval criteria using the Jira status items.
  • Rejected Status Criteria - Define the rejection criteria using the Jira status items.

When you finished, the approval step will look something like this:

When you deploy the Pipeline, the Approval Stage will display a link to the Jira issue selected for the approval:

Click the link in the Approval step to see the Jira issue in Jira.

Select the Approval criteria. In this example, the status Approved is the approval criteria.

Once the Jira issue is approved, the Approval stage is green in Deployments, and the deployment continues:

ServiceNow-based Approvals

For details on ServiceNow integration with Harness, see ServiceNow Integration.

You can add approval steps in Pipelines and use ServiceNow ticket statuses for approval and rejection criteria.

Here is a Pipeline containing two Workflows with a ServiceNow Approval step in between them.

We will walk through how the ServiceNow ticket is created in the first Workflow, and then how a ServiceNow approval is used in Stage 2.

The first Workflow includes a ServiceNow step that creates a new ServiceNow issue, as described in ServiceNow Integration:

The Variable Name is snow and the Scope is Pipeline. This will enable the output variable to be used in any stage of the Pipeline. The output variable uses the issueId property and so the resulting variable reference is ${snow.issueId}. You can reference the ServiceNow issue anywhere in the Pipeline using ${snow.issueId}.

Next, let's add a Pipeline Approval step that uses ${snow.issueId}.

In a Pipeline, click the Insert a Stage (+) button to add the Approval step:

The new stage dialog appears. Click Approval Step to see the approval settings.

The Approval Step has the following fields:

  • Ticketing System - For ServiceNow, select ServiceNow.
  • ServiceNow Connector - Select the ServiceNow account you want to use by selecting the Collaboration Provider you added for the account, as described in Add ServiceNow as a Collaboration Provider. Use the same provider you used to create the ticket in the Workflow.
  • Ticket Type - Select the ServiceNow ticket type. Use the same type as the ticket you created in the Workflow.
  • Issue Number - Enter the output variable for a ServiceNow issue created in a Workflow, such as ${snow.issueId}. Although we are demonstrating approval using the ${snow.issueId} variable, you can also simply enter the ServiceNow ticket number for any ServiceNow ticket in the account.
  • Timeout - Enter how long Harness should wait for the approval or rejection before failing the deployment.
  • Approved Criteria - The Field NameOperator, and Value drop-down menus are populated from the ServiceNow account you selected. Define the approval criteria using the ServiceNow status items.
  • Rejected/Closed Criteria - Define the rejection criteria using the ServiceNow status items.

When you finished, the approval step will look something like this:

When you deploy the Pipeline, the Approval Stage will display a link to the ServiceNow ticket selected for the approval:

Click the link in the Approval step to see the ServiceNow ticket in ServiceNow.

In ServiceNow, select the Approval criteria you defined. In this example, the status Resolved is the approval criteria.

Once the ServiceNow issue is approved, the Approval stage is green in Deployments, and the deployment continues:

You now have a ServiceNow-based approval in your Pipeline that ensures nothing is deployed without your approval.


How did we do?