CloudWatch Verification

Updated 3 days ago by Michael Cretzman

The following sections describe how Harness integrates ​​CloudWatch into Harness Continuous Verification to monitor your live, production services and verify your deployments:

CloudWatch and Harness

Amazon CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, providing you with a unified view of AWS resources, applications and services that run on AWS, and on-premises servers.

Harness Continuous Verification integrates with CloudWatch to verify your deployments and live production applications using the following Harness features:

  • 24/7 Service Guard - Monitors your live, production applications.
  • Deployment Verification - Monitors your application deployments, and performs automatic rollback according to your criteria.

This document describes how to set up these Harness Continuous Verification features and monitor your deployments and production applications using its unsupervised machine-learning functionality.

Monitoring with CloudWatch

Harness Analysis

Verification is limited to EC2 instance and ELB-related metrics data.

Setup Preview

You set up CloudWatch and Harness in the following way:

  1. CloudWatch - Using CloudWatch, you monitor the EC2 and ELB used to run your microservice or application.
  2. Cloud Provider - In Harness, you connect Harness to your AWS account, adding AWS as a Cloud Provider.
  3. Harness Application - Create a Harness Application with a Service and an Environment. We do not cover Application set up in this article. See Application Checklist.
  4. ​24/7 Service Guard Setup- In the Environment, set up 24/7 Service Guard to monitor your live, production application.
  5. Verify Deployments:
    1. Add a Workflow to your Harness Application and deploy your microservice or application to the service infrastructure in your Environment.
    2. After you have run a successful deployment, you then add verification steps to the Workflow using your Verification Provider.
    3. Harness uses unsupervised machine-learning and CloudWatch analytics to analyze your future deployments, discovering events that might be causing your deployments to fail. Then you can use this information to set rollback criteria and improve your deployments.

For information on setting up CloudWatch to monitor EC2, ECS, and ELB, see Monitoring Your Instances Using CloudWatch from AWS. When you enable CloudWatch monitoring on EC2, you are prompted with the following dialog.

Click Yes, Enable, and then go to CloudWatch to view metrics.

Cloud Provider Setup

The first step in using CloudWatch with Harness is to set up an AWS Cloud Provider in Harness.

A Harness Cloud Provider is a connection to AWS and its monitoring tools, such as CloudWatch. Once Harness is connected, you can use Harness 24/7 Service Guard and Deployment Verification with your CloudWatch data and analysis.

AWS Permissions Required: Harness requires the IAM user to be able to make API requests to AWS. The User Access Type required is Programmatic access. This enables an access key ID and secret access key for the AWS API, CLI, SDK, and other development tools. For more information, see Creating an IAM User in Your AWS Account from AWS.

To add AWS as a Harness Cloud Provider, you need a Harness Delegate installed in your AWS VPC. For AWS, the Shell Script Delegate and ECS Delegate are most commonly used.

Once the Delegate in installed in your VPC, you can use its credentials to connect the AWS Cloud Provider. For example, see the Assume IAM Role on Delegate and Delegate Tag options and in the AWS Cloud Provider:

For more information on setting up an AWS Cloud Provider in Harness, see Installation Example: Amazon Web Services and ECS and Amazon Web Services (AWS) Cloud.

24/7 Service Guard Setup

Harness 24/7 Service Guard monitors your live applications, catching problems that surface minutes or hours following deployment. For more information, see 24/7 Service Guard.

You can add your CloudWatch monitoring to Harness 24/7 Service Guard in your Harness Application Environment. For a setup overview, see Setup Preview.

This section assumes you have a Harness Application set up and containing a Service and Environment. For steps on setting up a Harness Application, see Application Checklist.

To set up 24/7 Service Guard for CloudWatch, do the following:

  1. Ensure that you have added CloudWatch as a Harness Verification Provider, as described in Verification Provider Setup.
  2. In your Harness Application, ensure that you have added a Service, as described in Services. For 24/7 Service Guard, you do not need to add an Artifact Source to the Service, or configure its settings. You simply need to create a Service and name it. It will represent your application for 24/7 Service Guard.
  3. In your Harness Application, click Environments.
  4. In Environments, ensure that you have added an Environment for the Service you added. For steps on adding an Environment, see Environments.
  5. Click the Environment for your Service. Typically, the Environment Type is Production.
  6. In the Environment page, locate 24/7 Service Guard.
  7. In 24/7 Service Guard, click Add Service Verification, and then click CloudWatch. The CloudWatch dialog appears.

  1. Fill out the dialog. The dialog has the following fields.
For 24/7 Service Guard, the queries you define to collect logs are specific to the application or service you want monitored. Verification is application/service level. This is unlike Workflows, where verification is performed at the host/node/pod level.

Field

Description

Display Name

The name that will identify this service on the Continuous Verification dashboard. Use a name that indicates the environment and monitoring tool, such as CloudWatch.

Service

The Harness Service to monitor with 24/7 Service Guard.

CloudWatch Server

Select the CloudWatch Verification Provider to use.

Region

Select the AWS region where the ECS and/or ELB are located.

ELB Metrics

Click Add for each load balancer you want to monitor. For more information, see Elastic Load Balancing Metrics and Dimensions from AWS.

ECS Metrics

This Cluster drop-down menu contains the available ECS clusters. The Metrics drop-down contains the available metrics. Select the metrics to monitor.

You can see the available metrics in CloudWatch.

For more information, see Using Amazon CloudWatch Metrics from AWS.

Lambda

Select the Lambda function and metrics to monitor. The functions displayed are from the region you selected. Only functions that have been deployed are displayed.

Algorithm Sensitivity

Select the sensitivity level for flagging anomalies.

Enable 24/7 Service Guard

Enable this setting to turn on 24/7 Service Guard. If you simply want to set up 24/7 Service Guard, but not enable it, leave this setting disabled.

When you are finished, the dialog will look something like this:

  1. Click TEST. Harness verifies the settings you entered.
  2. Click SUBMIT. The CloudWatch 24/7 Service Guard to configured.

To see the running 24/7 Service Guard analysis, click Continuous Verification.

The 24/7 Service Guard dashboard displays the production verification results.

For information on using the dashboard, see Using 24/7 Service Guard.

Verify Deployments

The following procedure describes how to add CloudWatch as a verification step in a Harness workflow. For more information about workflows, see Add a Workflow.

Once you run a deployment and CloudWatch preforms verification, Harness' machine-learning verification analysis will assess the risk level of the deployment.

In order to obtain the names of the host(s), pod(s), or container(s) where your service is deployed, the verification provider should be added to your workflow after you have run at least one successful deployment.

To verify your deployment with CloudWatch, do the following:

  1. Ensure that you have added AWS as a cloud provider, as described above.
  2. In your workflow, under Verify Service, click Add Verification, and then click CloudWatch. The CloudWatch dialog appears.

The CloudWatch dialog has the following fields.

Field

Description

CloudWatch Server

Select the AWS cloud provider you set up earlier.

Region

Select the AWS region where the EC2 and/or ELB are located.

Lambda

You simply need to select this option and Harness will use the CloudWatch monitoring for the Lambda function(s) the Workflow is deploying.

You can select ELB Metrics (Load Balancers, Metric Names) but they are not required. For information on Lambda metrics, see AWS Lambda Metrics.

EC2 Metrics

This drop-down menu contains the available EC2 metrics. Select the metrics to monitor. For more information, see Using Amazon CloudWatch Metrics from AWS.You can see the available metrics in CloudWatch. Click Metrics, and then click All metrics.

ECS Metrics

Expand the ECS Metrics option and specify the Cluster and Metric Names for monitoring.

ELB Metrics

ELB Metrics are available for all of the CloudWatch types. Add each load balancer you want to monitor. For more information, see Elastic Load Balancing Metrics and Dimensions from AWS.

Load Balancers

Select the load balancer to monitor. The list of load balancers is populated according to the AWS cloud provider and region you selected.

Metrics Name

This drop-down menu contains the available ELB metrics. Select the metrics you want to monitor.

Analysis Time duration

Set the duration for the verification step. If a verification step exceeds the value, the workflow Failure Strategy is triggered. For example, if the Failure Strategy is Ignore, then the verification state is marked Failed but the workflow execution continues.

Baseline for Risk Analysis

Select Previous Analysis to have this verification use the previous analysis for a baseline comparison. If your workflow is a Canary workflow type, you can select Canary Analysis to have this verification compare old versions of nodes to new versions of nodes in real-time.

Execute with previous steps

Check this checkbox to run this verification step in parallel with the previous steps in Verify Service.

Failure Criteria

Specify the sensitivity of the failure criteria. When the criteria is met, the workflow Failure Strategy is triggered.

Here is an example of a completed CloudWatch verification step.

When you are finished, click SUBMIT. The CloudWatch verification step is added to your workflow.

Verification Results

Once you have deployed your workflow (or pipeline) using the CloudWatch verification step, you can automatically verify cloud application and infrastructure performance across your deployment. For more information, see Add a Workflow and Add a Pipeline.

Workflow Verification

To see the results of Harness machine-learning evaluation of your CloudWatch verification, in your workflow or pipeline deployment you can expand the Verify Service step and then click the CloudWatch step.

[screenshot]

Continuous Verification

You can also see the evaluation in the Continuous Verification dashboard. The workflow verification view is for the DevOps user who developed the workflow. The Continuous Verification dashboard is where all future deployments are displayed for developers and others interested in deployment analysis.

To learn about the verification analysis features, see the following sections.

Deployments

Harness supports Metrics from CloudWatch for Lambda, EC2, ECS, and ELB.

Deployment info - See the verification analysis for each deployment, with information on its service, environment, pipeline, and workflows.

Verification phases and providers - See the verification phases for each verification provider. Click each provider for logs and analysis.

Verification timeline - See when each deployment and verification was performed.

Web Transaction Analysis

Execution details - See the details of verification execution. Total is the total time the verification step took, and Analysis duration is how long the analysis took.

Risk level analysis - Get an overall risk level and view the cluster chart to see events.

Web Transaction-level summary - See a summary of each transaction with the query string, error values comparison, and a risk analysis summary.

Event Management

Event-level analysis - See the threat level for each event captured.

Tune event capture - Remove events from analysis at the service, workflow, execution, or overall level.

Event distribution - Click the chart icon to see an event distribution including the measured data, baseline data, and event frequency.

Next Steps


How did we do?