Workflow Queuing

Updated 2 weeks ago by Michael Katz

This topic outlines how Harness queues Workflows, to prevent conflicts when two or more Workflows simultaneously deploy to the same target infrastructure. It includes the following sections:

Intended Audience

  • DevOps

Before You Begin

Overview

When multiple Harness Workflows simultaneously deploy to the same infrastructure, there is a risk of conflicts. To prevent these conflicts, Harness normally queues such Workflows in FIFO (First In, First Out) order.

You can override this behavior, as covered below. Queuing is particularly valuable for Pipelines that execute multiple Workflows in parallel.

Acquiring Resource Locks

When pending Workflows are contending for shared infrastructure, Harness places the Workflows in a resource lock queue. The first-launched Workflow gets a resource lock on this infrastructure, which it holds until its deployment is resolved. This temporarily blocks other Workflows from using the infrastructure—only one Workflow at a time can have a lock on a given infrastructure.

When contending for shared infrastructure, most Workflows will therefore display an Acquire Resource Lock step in Harness' Deployments page:

This step appears even if no queue is present, because of Harness' Acquire lock on the targeted service infrastructure default setting (described below). There are two exceptions:

  • No Acquire Resource Lock will appear in Workflows of Build deployment type.
  • No Acquire Resource Lock step will appear in Workflows that have been configured to ignore queueing (see Using Concurrency Strategy to Control Queuing).

Harness seeks to acquire a Resource Lock only once per Workflow. The lock's scope is the current Workflow. The Acquire Resource Lock step occurs in the first deployment or setup phase that follows any Pre‑Deployment phase or step.

This example, using a Pivotal Cloud Foundry Blue/Green Workflow, shows the Acquire Resource Lock step's typical position in a deployment:

Queuing in Action

Let's look at a full example of how queuing works. Assume that we have two similar Harness Kubernetes Workflows, gke demo and gke demo-template-clone. Both deploy to the same infrastructure, because they share the same Infrastructure Definition.

When both Workflows are deployed simultaneously, Harness might deploy gke demo slightly before gke demo‑template‑clone. This scenario applies to a Pipeline whose stages run these Workflows in parallel, but in that order:

If we examine the gke demo-template-clone Workflow's Deployments page, we might initially see something like this:

This deployment is paused at its Acquire Resource Lock step. The Details panel shows why:

The gke demo-template-clone deployment is second in the Resource Lock Queue, so it currently has BLOCKED status.

Meanwhile, the gke demo is first in the queue. If we immediately switch to its Deployments page, we might see something like this:

This Workflow's Acquire Resource Lock step has completed (it has acquired the lock). The Details panel confirms that this first-in-queue deployment has ACTIVE status:

So the gke demo Workflow can now proceed through completion:

Once the gke demo Workflow has completed deployment, this clears the queue. Therefore, the gke demo‑template‑clone can now acquire the lock, and proceed to deploy:

Using Concurrency Strategy to Control Queuing

By default, Harness Workflows have their Concurrency Strategy set to Acquire lock on the targeted service infrastructure. This is the setting that enables queuing for shared infrastructure.

To exempt a Workflow from queuing behavior, click the pencil icon to open the Concurrency Strategy dialog shown here. Set the Concurrency Control drop-down to Synchronization not required, and click Submit.

Queuing with Infrastructure Provisioners

Workflows incorporating Infrastructure Provisioners are queued the same way as Workflows based on predefined infrastructure. Infrastructure Provisioners commands are always added in the Workflow's Pre‑Deployment phase. This sets up the new infrastructure, enabling Harness to properly queue and lock deployments to that infrastructure in the following phase.

Next Steps

To more precisely synchronize multiple Workflows within a Pipeline, use Barriers.


How did we do?