Canary Deployment

A Canary deployment rolls out a new app version to small sets of users in separate phases, tests and verifies it at each phase, gradually rolling it out to your entire infrastructure.

New app code has a relatively small impact because it is only distributed to a small number of users in each phase. Changes can be rolled back quickly in case the new code fails verification. Each phase is independent, and you can add verification and custom commands before moving from one phase to the next.

Why Use Canary Deployment?

Canary deployments reduce the scope, impact, and risk of deploying new software artifacts to production. Instead of deploying a new artifact across all production nodes at the same time, you deploy to a subset of those nodes in phases, with each phase verified to ensure artifacts and your application are behaving appropriately.

For example, here is a three phase Canary deployment across a production stack:

After each deployment phase, the stability, performance, and quality of the artifact(s) and app is verified before allowing the next deployment phase to execute.

What does a Harness Canary Deployment Look Like?

A Canary deployment in Harness is a workflow using the Canary Deployment workflow type. By default, the Canary Deployment workflow looks like this:

Intended Audience

  • DevOps

Before You Begin

This Canary deployment procedure deploys a Docker artifact to a cloud provider. Before beginning the procedure, ensure that you have set up an Artifact Server and a Cloud Provider:

You might also want to review what a Harness application and workflow are, because an application is the logical grouping of artifacts, services, environments, workflows, etc, and Canary Deployment is a Harness workflow type.

The following procedure will touch on all of these entities, also.

Create a Canary Deployment

The following example scenario creates a Canary deployment using a Docker image and the Amazon EC2 container services (ECS) in the AWS Cloud.

To create a Canary deployment, do the following:

  1. Add a new Harness application. For more information, see Application Checklist.
    1. Click Setup, and then click Add Application. The Application dialog appears.
    2. Enter the name for your application, and click SUBMIT. Your new application appears.
    3. Click your application’s name. The application entities appear.An application is a logical grouping of artifacts, services, environments, workflows, pipelines, triggers, and infrastructure provisioners.
  2. Add a service to your application.
    1. Click Services, and then click Add Service. The Service dialog appears.
    2. Enter My Microservice in Name, in Artifact Type, select Docker Image, and click SUBMIT. The new service appears.
    3. Click the name of the service to open the Service page.
    4. In Artifact Source, click Add Artifact Source, and select Docker Registry.
    The Artifact Source dialog appears with settings specific to the Docker source type.

    In order to add a Source Server and select a Docker Image Name, you need to have a Docker Artifact Server configured in Harness. To add a Docker Artifact Server, follow the steps in Add Artifact Servers.
    For detailed information on adding a Docker image service, see Add a Docker Image Service. Another resource is the Harness Quick Start Setup Guide, which walks you through deploying a publicly-available Tomcat artifact from Docker Hub on to a Kubernetes cluster in the Google Cloud Platform.
  3. Define a deployment environment.
    1. In the application page, click Add Environment.
    2. In Name, enter Production, and in Environment Type, choose Production. Click SUBMIT. For more information, see Add an Environment.
    3. In Environment Overview, click Add Service Infrastructure. The Service Infrastructure dialog appears.
    4. In Service, select My Microservice, and then, in Deployment Type, select Amazon ECS (Harness also supports other cloud providers).
    5. In Cloud Provider, choose your Cloud Provider account, region and desired cluster to deploy to.This step assumes you have set up a AWS cloud provider. For more information, see Add Cloud Providers.
    6. Click SUBMIT.
  4. Create a Canary workflow.
    1. In your application page (click My Application in the breadcrumbs), click Workflows.
    2. Click Add Workflow. The Workflow dialog appears.
    3. In Name, enter Microservice Canary Deployment.
    4. In Description, enter 3 Phase Canary Deployment (10%, 50%, 100%).
    5. In Workflow Type, select Canary Deployment and then select the environment you have just created (Production).
    6. Click SUBMIT.
      The workflow wizard appears. It that will guide you through your Canary deployment set up.
  5. Add pre-deployment steps. For an example, we will add a simple pre-deployment step that will notify us that the Canary deployment is about to start via email.
    1. Under Pre-deployment Steps, click Add Step, and select Email.
    2. Enter the email details in To, Subject, and Body.
    3. Click SUBMIT. Prior to any deployment, you have the option to invoke any 3rd party tools via HTTP REST in addition to native integration with providers such as Bamboo and Jenkins jobs.
  6. Add a 3 phase Canary deployment. The 3 phase Canary deployment for our microservice will deploy to 10%, 50%, and 100% of your environment respectively.
    1. Under Development Phases, click Add Phase. The Workflow Phase dialog appears.
    2. In Service, select My Microservice.
    3. In Service Infrastructure, select the service infrastructure you created when you defined your environment.
    4. Click SUBMIT. The Phase 1 page appears.
    5. To enter the number of desired nodes that you want to upgrade in phase 1 (10%), click Upgrade Containers. The Upgrade Containers dialog appears.
    6. In Desired Instances, enter 10 and select Percent from the drop-down.
    7. Click SUBMIT.
    8. To specify how Harness is going to verify the service being deployed as part of this Canary phase, click Add Verification.
      For this simple example, we will just verify with an email, but once you have run a deployment successfully, you can add advanced verification steps involving Verification Providers.
    9. In the Add Command dialog, click Email again.
    10. Fill out the email with you address and a message about approving the deployment phase.
  7. Repeat the above steps for Canary phases 2 and 3, setting them for 50% and 100% Desired Instances in the Upgrade Containers dialogs, respectively. When you are complete, your Canary workflow should look like this:

    At this point you have a few optional steps to consider:
  • Notification Strategy: What events in your Canary workflow do you want to notify people with? For more information, see Notification Strategy.
  • Failure Strategy: What steps do you want to take if any of your Canary phase verifications fail? (The default is an automated rollback.) For more information, see Failure Strategy.
  • Workflow Variables: Do you want to “templatize” this canary workflow so other teams or services can use the same deployment workflow? For more information, see Template a Workflow.

Once completed your workflow should look something like this:

At this point, you are finished your Canary workflow and you could simply deploy it. To demonstrate a realistic scenario, we will add a pipeline and trigger because these are very common CD steps.

  1. Add a pipeline for the Canary workflow.
    A pipeline in Harness is made up of one or more stages that execute in serial or parallel, with each stage referencing a specific deployment workflow. Most Harness customers have a 4 to 5 stage CD pipeline that allows them to deploy their services/artifacts across each of their Dev, QA, Staging, and Production environments. For more information, see Add a Pipeline.
    For our Canary deployment, we will create a simple one-stage pipeline that invokes the Canary workflow we just created.
    1. Click My Application in the breadcrumbs.
    2. Click Pipelines, and in the Pipelines page, click Add Pipeline.
    3. In Name, enter Canary Pipeline, and click SUBMIT.
    4. In the phase dialog, click Execution Step.
    5. In Step Name, enter Production.
    6. In Execute Workflow, select Microservice Canary Deployment, and click SUBMIT.
  2. Add a Trigger to execute the CD pipeline. For more information, see Add a Trigger.
    1. Click My Application in the breadcrumbs.
    2. Click Triggers, and then on the Triggers page, click Add Trigger. The Trigger dialog appears.
    3. Enter the name Microservice Trigger (New Artifact), and then, in Condition, select On New Artifact.
    4. In Artifact Source, select the artifact source we are using.
    5. In Actions, in Execution Type, select Workflow.
    6. In Execute Workflow, select our Canary workflow.
    7. In Service, ensure the From Triggering Artifact Source option is selected.
    8. Accept the defaults and click SUBMIT.

The Canary workflow will now execute when a new artifact is posted to our artifact source.

Next Steps

See a video of how to create a Canary deployment.

How did we do?