Traffic Splitting Without Istio

Updated 1 day ago by Michael Cretzman

This topic describes how to progressively increase traffic to new application versions using Ingress resources, Harness annotations, and the Apply step.

Kubernetes Annotations are used to ignore the Ingress manifests during the main deployment, and to specify weights for each Ingress resource (0, 25, 50). Each Ingress resource is then applied using a separate Apply strep.

This technique can be use with Blue/Green and Canary deployments. For this topic, we will modify standard Harness Kubernetes Blue/Green deployment.

For applications that are implemented using service mesh, such as Istio, see Set Up Kubernetes Traffic Splitting.

In this topic:

Before You Begin

Visual Summary

Here's a recording of what your completed deployment will look like, including approval steps:

Step 1: Review Blue/Green Service Swap

When you create a Harness Service for a Blue/Green deployment, you need to include a manifest for each of the Kubernetes services used in Blue/Green.

Harness refers to the two services as primary and stage Kubernetes services, distinguished using the following mandatory annotations:

  • Primary - annotations: harness.io/primary-service: true
  • Stage - annotations: harness.io/stage-service: true

When the Workflow is deployed, Harness modifies the selector at runtime to add harness.io/color with values blue and green.

Harness uses these to redirect traffic from the stage service to the primary service (current version).

After the route update where the primary and stage service is swapped, the primary service routes requests to the new app version based on the harness.io/color selector. Here is a log of the swap:

Begin execution of command Kubernetes Swap Service Selectors

Selectors for Service One : [name:harness-example-svc]
app: harness-example
harness.io/color: green

Selectors for Service Two : [name:harness-example-svc]
app: harness-example
harness.io/color: green

Swapping Service Selectors..

Updated Selectors for Service One : [name:harness-example-svc]
app: harness-example
harness.io/color: green

Updated Selectors for Service Two : [name:harness-example-svc]
app: harness-example
harness.io/color: green

Done

With this method, only one version of the application is servicing the requests at anytime.

For more information, see Create a Kubernetes Blue/Green Deployment.

Step 2: Review Traffic Splitting with Ingress

In Traffic Splitting Without Istio deployment, first we deploy the new version of the app and use the stage service to send traffic to it.

Next we increase the stage service traffic to 25% and then 50% so it receives half the traffic. Now the new and old versions are sharing traffic equally.

Once the swap occurs, and the stage service is now routing to the old app version, we will decrease the traffic to the stage service (and old version) to 25% and then 0%.

To increase and decrease traffic, we are using the weight-based traffic splitting of the NGINX Ingress controller. To control the weights, we use this controller's nginx.ingress.kubernetes.io/canary-weight annotation in our Ingress manifests.

Here's how the Ingress weights will be used on the Kubernetes stage service:

  1. Before the swap:
    1. Stage service receives weight of 25%.
    2. Stage service receives weight of 50%.
  2. After the swap:
    1. Stage service receives weight of 25%.
    2. Stage service receives weight of 0%.

Step 3: Create the Harness Service

  1. Create the Harness Kubernetes Service you will use for your artifact and manifests.
  2. Add a container artifact using the steps in Add a Docker Artifact Source.

In the following steps we will supplement the default files generated by Harness. The following files will be used but will not be changed:

  • deployment.yaml
  • namespace.yaml
  • values.yaml
  1. Remove the default service.yaml file. We will be replacing this file in the steps below.

Step 4: Create the Primary Service Manifest

  1. In the templates folder, create a file named service-primary.yaml.
  2. Add the following YAML to service-primary.yaml:
apiVersion: v1
kind: Service
metadata:
name: {{.Values.name}}-svc
annotations:
harness.io/primary-service: true
spec:
type: {{.Values.serviceType}}
ports:
- port: {{.Values.servicePort}}
targetPort: {{.Values.serviceTargetPort}}
protocol: TCP
selector:
app: {{.Values.name}}

Note the harness.io/primary-service: true annotation.

Next, we'll create the Kubernetes manifest for the stage service.

Step 5: Create the Stage Service Manifest

  1. In the templates folder, create a file named service-stage.yaml.
  2. Add the following YAML to service-primary.yaml:
apiVersion: v1
kind: Service
metadata:
name: {{.Values.name}}-svc
annotations:
harness.io/stage-service: true
spec:
type: {{.Values.serviceType}}
ports:
- port: {{.Values.servicePort}}
targetPort: {{.Values.serviceTargetPort}}
protocol: TCP
selector:
app: {{.Values.name}}

Note the harness.io/stage-service: true annotation.

Step 6: Add the Ingress Manifests

There are three Ingress manifests to add. Harness will ignore them in the main deployment step (Stage Development step) because they start with the comment:

# harness.io/skip-file-for-deploy

See Ignore a Manifest File During Deployment for more information on ignoring manifests.

Each Ingress manifest will also contain the nginx.ingress.kubernetes.io/canary-weight annotation. Here is an example from one Ingress manifest:

# Enable canary and send 0% of traffic to version 2
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "0"

Copy and paste the following three Ingress manifests into three new files in templates.

  1. For each new file, click the templates folder, and then click Add File:

Add the following three files:

ingress-traffic-split0.yaml

# harness.io/skip-file-for-deploy
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: {{.Values.name}}-stage

labels:
app: {{.Values.name}}
annotations:
kubernetes.io/ingress.class: "nginx"

# Enable canary and send 0% of traffic to version 2
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "0"
spec:
rules:
- host: {{.Values.name}}.com
http:
paths:
- backend:
serviceName: {{.Values.name}}-stage
servicePort: 80

ingress-traffic-split25.yaml

# harness.io/skip-file-for-deploy
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: {{.Values.name}}-stage

labels:
app: {{.Values.name}}
annotations:
kubernetes.io/ingress.class: "nginx"

# Enable canary and send 25% of traffic to version 2
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "25"
spec:
rules:
- host: {{.Values.name}}.com
http:
paths:
- backend:
serviceName: {{.Values.name}}-stage
servicePort: 80

ingress-traffic-split50.yaml

# harness.io/skip-file-for-deploy
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: {{.Values.name}}-stage

labels:
app: {{.Values.name}}
annotations:
kubernetes.io/ingress.class: "nginx"

# Enable canary and send 50% of traffic to version 2
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
rules:
- host: {{.Values.name}}.com
http:
paths:
- backend:
serviceName: {{.Values.name}}-stage
servicePort: 80

That's all the configuration needed in the Harness Service.

Step 7: Define Your Kubernetes Target Infrastructure

There are no Harness Infrastructure Definition settings specific to Kubernetes Blue/Green deployment. Create or use the Infrastructure Definition that targets your cluster, as described in Define Your Kubernetes Target Infrastructure.

Ensure that the Kubernetes cluster includes an Ingress controller that supports traffic splitting, such as NGINX Ingress controller we are using as an example.

Step 8: Create the Blue/Green Workflow

  1. In your Application, click Workflows, and then click Add Workflow. The Workflow settings appear. Enter the following settings:

  • Name: Enter a name for your Workflow.
  • Workflow Type: Select Blue/Green Deployment.
  • Environment: Select the Environment that contains your target Infrastructure Definition.
  • Service: Select the Service containing your Ingress and service manifests.
  • Infrastructure Definition: Select the Infrastructure Definition for your target Kubernetes cluster.

The Workflow is created with the default Blue/Green steps.

When we are done with the following steps the Workflow will look like this:

Step 9: Add Workflow Sections

Workflow sections help you organize your steps. We'll add one section before the Verify section, and one after the Route Update section.

To add a section, click the options button () next to any section and then click Add Section.

Enter the name Shift 50% Traffic Before Switch and click Submit.

The new section is added to the bottom of the Workflow. Use the Reorder option to move the section to right above Verify.

Add another section named Shift Remaining Traffic After Switch and move it to right before the Wrap Up step.

When you're done, the Workflow will look like this:

Step 10: Add Apply Steps

Next we'll add the Apply steps for the Ingress objects to defined in your Harness Service.

  1. In the new Shift 50% Traffic Before Switch section, click Add Step.
  2. Enter the following settings to apply the ingress-traffic-split25.yaml file from the Harness Service and click Submit:

  • Name: Enter Configure Stage 25%.
  • File Paths: Enter templates/ingress-traffic-split25.yaml.
  • For the rest of the settings, you can leave the defaults.

This step will increase the traffic routed to the stage service and the new app version by 25%.

Next, you will add a step to increase the traffic routed to the stage service and the new app version by 50%.

  1. Below this step, add another Apply step to apply the ingress-traffic-split50.yaml file from the Harness Service and click Submit:

  • Name: Enter Configure Stage 50%.
  • File Paths: Enter templates/ingress-traffic-split50.yaml.
  • For the rest of the settings, you can leave the defaults.

Now you can add the steps for decreasing the traffic routed to the stage service and the old version of the app. The first step decreases the traffic to 25%:

  1. In the Shift Remaining Traffic After Switch section, add an Apply step for the templates/ingress-traffic-split25.yaml file from the Harness Service and click Submit:

  • Name: Enter Configure Stage 25%.
  • File Paths: Enter templates/ingress-traffic-split25.yaml.
  • For the rest of the settings, you can leave the defaults.

Finally, you add a step to decrease the traffic routed to the stage service and the old app version to 0%:

  1. Below this step, add another Apply step to apply the ingress-traffic-split0.yaml file from the Harness Service and click Submit:

  • Name: Enter Configure Stage 0%.
  • File Paths: Enter templates/ingress-traffic-split0.yaml.
  • For the rest of the settings, you can leave the defaults.

You can actually run the deployment now. The following step add Approval steps between each traffic increase and decrease so you can approve of the changes.

Option 1: Add Approval Steps

Add Approve steps in between each Apply step to ensure that traffic is not increased or decreased without your approval.

Here is an example of the Approve step for the 25% increase:

  • Name: Enter Approve 25%.
  • Ticketing System: Select Harness UI.
  • User Groups: Select a group in which you are a member, such as Account Administrator.

Create Approve steps before each of the remaining Apply steps. When you're done the Workflow will look like this:

Step 11: Deploy the Workflow

Now that your Workflow is complete, you can deploy it. If you added the Approve steps, be sure to approve each one.

Here's a recording of what your deployment will look like:

Option 2: Create Workflow Steps Using YAML

Once you first create a new Blue/Green Workflow as described above, and before you add any steps, you can paste the YAML below and automatically create the Workflow steps you need.

  1. In the Workflow, click the </> button to display the YAML Editor, and then click Edit.

You simply need to update references to the values of entities such as the envName, infraDefinitionName, serviceName, and computeProviderName: K8s Tutorial keys with the names of the entities you have set up for your configuration:

harnessApiVersion: '1.0'
type: BLUE_GREEN
concurrencyStrategy: INFRA
description: No Istio
envName: Shift Example
failureStrategies:
- executionScope: WORKFLOW
failureTypes:
- APPLICATION_ERROR
repairActionCode: ROLLBACK_WORKFLOW
retryCount: 0
phases:
- type: KUBERNETES
computeProviderName: K8s Tutorial
daemonSet: false
infraDefinitionName: no Istio
name: Blue/Green
phaseSteps:
- type: K8S_PHASE_STEP
name: Deploy
steps:
- type: K8S_DELETE
name: Delete
properties:
deleteNamespacesForRelease: false
resources: Service/harness-example-svc
- type: K8S_BLUE_GREEN_DEPLOY
name: Stage Deployment
stepsInParallel: false
- type: K8S_PHASE_STEP
name: Shift 50% Traffic Before Switch
steps:
- type: K8S_APPLY
name: Apply 25%
properties:
filePaths: templates/ingress-traffic-split25.yaml
stateTimeoutInMinutes: 10
templateUuid: null
templateVariables: null
templateVersion: null
- type: K8S_APPLY
name: Apply 50%
properties:
filePaths: templates/ingress-traffic-split50.yaml
stateTimeoutInMinutes: 10
stepsInParallel: false
- type: K8S_PHASE_STEP
name: Verify
stepsInParallel: false
- type: K8S_PHASE_STEP
name: Route Update
steps:
- type: KUBERNETES_SWAP_SERVICE_SELECTORS
name: Swap Primary with Stage
properties:
service1: ${k8s.primaryServiceName}
service2: ${k8s.stageServiceName}
stepsInParallel: false
- type: K8S_PHASE_STEP
name: Shift Remaining Traffic After Switch
steps:
- type: K8S_APPLY
name: Apply 75%
properties:
filePaths: templates/ingress-traffic-split25.yaml
stateTimeoutInMinutes: 10
- type: K8S_APPLY
name: Apply 100%
properties:
filePaths: templates/ingress-traffic-split0.yaml
stateTimeoutInMinutes: 10
templateUuid: null
templateVariables: null
templateVersion: null
stepsInParallel: false
- type: K8S_PHASE_STEP
name: Wrap Up
stepsInParallel: false
provisionNodes: false
serviceName: nginx
statefulSet: false
rollbackPhases:
- type: KUBERNETES
computeProviderName: K8s Tutorial
daemonSet: false
infraDefinitionName: no Istio
name: Rollback Blue/Green
phaseNameForRollback: Blue/Green
phaseSteps:
- type: K8S_PHASE_STEP
name: Route Update
phaseStepNameForRollback: Route Update
statusForRollback: SUCCESS
steps:
- type: KUBERNETES_SWAP_SERVICE_SELECTORS
name: Swap Primary with Stage
properties:
service1: ${k8s.primaryServiceName}
service2: ${k8s.stageServiceName}
stepsInParallel: false
- type: K8S_PHASE_STEP
name: Wrap Up
phaseStepNameForRollback: Wrap Up
stepsInParallel: false
provisionNodes: false
serviceName: nginx
statefulSet: false
templatized: false

Notes

If you deployed the standard Blue/Green Workflow before configuring it with the Ingress steps, and then deployed it with the Ingress steps, you might get the following error:

Found conflicting service [harness-example-svc] in the cluster. For blue/green deployment, the label [harness.io/color] is required in service selector. Delete this existing service to proceed

This is because the services changed between deployments. You can delete the first service using a Delete step or you can use a different name for your app.

Next Steps


How did we do?