Rollback Production Deployments

Updated 4 months ago by Michael Katz

The Rollback Deployment option initiates a rollback of your most-recent successful deployment. This allows rapid, predictable recovery from a deployment that succeeded on technical criteria, but that you want to undo for other reasons.

In this topic:

Review: Required Permissions

The Rollback Deployment option requires the following User Group Account and Application permissions:

  • Account: Manage Applications
  • Application: Rollback Workflow

You can also add the Rollback Workflow Application permission via the GraphQL API:

mutation {
  updateUserGroupPermissions(input: {
    clientMutationId: "123"
    userGroupId: "Gh9IDnVrQOSjckFbk_NJWg"
    permissions: {
      appPermissions: {
        actions:[ROLLBACK_WORKFLOW]
        permissionType: ALL
        applications: {
          filterType: ALL
        }
        deployments: {
          filterTypes: NON_PRODUCTION_ENVIRONMENTS
        }
      }
    }
  }) {
    clientMutationId
  }
}

Rollback Workflow added if Execute Workflow used Previously

All User Groups that had the Execute Workflow permission enabled will now have Rollback Workflow enabled, also. You can disable it if needed.

Review: Platform and Workflow Support

Rollback Deployment currently supports the following platforms and strategies:

  • Kubernetes deployments: Basic, Blue/Green, Canary, Rolling Workflows.
  • SSH deployments: Blue/Green, Canary, and Basic Workflows.
  • PCF (Pivotal Cloud Foundry) deployments: Blue/Green, Canary, and Basic Workflows.
  • WinRM (IIS and .NET) deployments: Blue/Green, Canary, and Basic Workflows.
  • ECS deployments: all Workflow types, and both EC2 and Fargate clusters.
  • AMI/ASG deployments: Blue/Green, Canary, and Basic Workflows.

Harness anticipates expanding this feature to other deployment platforms.

Step 1: Rollback a Production Deployment

Before you begin, please review Requirements and Warnings.

To initiate a post-deployment rollback:

  1. Open your Services Dashboard.
  2. In the Current Deployment Status panel, click the More Options ⋮ menu beside the most-recent deployment. Then select Rollback Deployment.
    The Rollback Deployment option appears only for the current deployment.
  3. In the resulting confirmation dialog, verify the deployment's details. If everything looks correct, click Rollback.
    Harness then invokes the Workflow's configured Rollback Strategy, executing the same Rollback Steps as if the deployment had failed.
    Once the rollback completes, your deployed instances will be returned to the state they were in before the most-recent deployment.

Requirements and Warnings

Rollback Deployment will execute Rollback Steps on your deployment according to the Workflow's current configuration. Make sure the Workflow (including any variables) has not been reconfigured since this most-recent deployment, or the rollback can have unpredictable results.
A deployment's Rerun option will be unavailable during, and following, a post-deployment rollback.

In order to use Post-Deployment rollback, the following requirements must be met:

  • The Workflow must use a Production Environment. It will not work on Non-Production Environments.
  • There must be at least two successful deployments of the Workflow.
  • The Workflow Type cannot be a Multi-Service Deployment.
  • A user's ability to invoke the Rollback Deployment option is based on their User Group membership, and on corresponding role-based permissions.

How did we do?