CloudWatch Verification

Updated 4 months ago by Michael Cretzman

Amazon CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, providing you with a unified view of AWS resources, applications and services that run on AWS, and on-premises servers.

You can add a CloudWatch verification step to your workflow and CloudWatch will be used by Harness to verify the performance and quality of your deployments using Harness machine-learning verification analysis.

Monitoring with CloudWatch

Harness Analysis

Verification is limited to EC2 instance and ELB-related metrics data.

Verification Setup Overview

You set up CloudWatch and Harness in the following way:

  1. Using CloudWatch, you monitor the EC2 and ELB used to run your microservice or application.
  2. In Harness, you connect Harness to your AWS account, adding AWS as a Cloud Provider.
  3. After you have built and run a successful deployment of your microservice or application in Harness, you then add CloudWatch verification steps to your Harness deployment workflow.
  4. Harness uses CloudWatch to verify your future microservice/application deployments.
  5. Harness Continuous Verification uses unsupervised machine-learning to analyze your deployments and CloudWatch analytics/logs, discovering events that might be causing your deployments to fail. Then you can use this information to improve your deployments.

For information on setting up CloudWatch to monitor EC2 and ELB, see Monitoring Your Instances Using CloudWatch from AWS. When you enable CloudWatch monitoring on EC2, you are prompted with the following dialog.

Click Yes, Enable, and then go to CloudWatch to view metrics.

Intended Audience

  • Developers
  • DevOps

Before You Begin

AWS Permissions Required

Harness requires the IAM user to be able to make API requests to AWS. The User Access Type required is Programmatic access. This enables an access key ID and secret access key for the AWS API, CLI, SDK, and other development tools.

For more information, see Creating an IAM User in Your AWS Account from AWS.

Connect to CloudWatch

Connect Harness to AWS to have Harness verify the success of your deployments. Harness will use your tools to verify deployments and use its machine learning features to identify sources of failures.

To add AWS as a Harness cloud provider, do the following (also documented in Amazon Web Services (AWS) Cloud).

  1. In Harness, click Setup, and then click Cloud Providers.
  2. Click Add Cloud Provider. The Cloud Provider dialog appears.
  3. Select Amazon Web Services in Type. The Cloud Provider dialog changes for the AWS settings.

The AWS dialog has the following fields.

Field

Description

Access Key

Enter the access key for your AWS account. For more information, see Access Keys (Access Key ID and Secret Access Key) from AWS.

Secret Key

Enter the secret key for your AWS account.

AWS Account Name

Choose a name for this provider. The name is to differentiate AWS providers in Harness. It is not the actual AWS account name.

Usage Scope

If you want to restrict the use of a provider to specific applications and environments, do the following:

In Usage Scope, click the drop-down under Applications, and click the name of the application.

In Environments, click the name of the environment.

Verify with CloudWatch

The following procedure describes how to add CloudWatch as a verification step in a Harness workflow. For more information about workflows, see Add a Workflow.

Once you run a deployment and CloudWatch preforms verification, Harness' machine-learning verification analysis will assess the risk level of the deployment.

In order to obtain the names of the host(s), pod(s), or container(s) where your service is deployed, the verification provider should be added to your workflow after you have run at least one successful deployment.

To verify your deployment with CloudWatch, do the following:

  1. Ensure that you have added AWS as a cloud provider, as described above.
  2. In your workflow, under Verify Service, click Add Verification, and then click CloudWatch. The CloudWatch dialog appears.

The CloudWatch dialog has the following fields.

Field

Description

CloudWatch Server

Select the AWS cloud provider you set up earlier.

Region

Select the AWS region where the EC2 and/or ELB are located.

EC2 Metrics

This drop-down menu contains the available EC2 metrics. Select the metrics to monitor. For more information, see Using Amazon CloudWatch Metrics from AWS.

You can see the available metrics in CloudWatch. Click Metrics, and then click All metrics.

ELB Metrics

Click Add for each load balancer you want to monitor. For more information, see Elastic Load Balancing Metrics and Dimensions from AWS.

Load Balancers

Select the load balancer to monitor. The list of load balancers is populated according to the AWS cloud provider and region you selected.

Metrics Name

This drop-down menu contains the available ELB metrics. Select the metrics you want to monitor.

Analysis Time duration

Set the duration for the verification step. If a verification step exceeds the value, the workflow Failure Strategy is triggered. For example, if the Failure Strategy is Ignore, then the verification state is marked Failed but the workflow execution continues.

Baseline for Risk Analysis

Select Previous Analysis to have this verification use the previous analysis for a baseline comparison. If your workflow is a Canary workflow type, you can select Canary Analysis to have this verification compare old versions of nodes to new versions of nodes in real-time.

Execute with previous steps

Check this checkbox to run this verification step in parallel with the previous steps in Verify Service.

Failure Criteria

Specify the sensitivity of the failure criteria. When the criteria is met, the workflow Failure Strategy is triggered.

Here is an example of a completed CloudWatch verification step.

When you are finished, click SUBMIT. The CloudWatch verification step is added to your workflow.

Verification Results

Once you have deployed your workflow (or pipeline) using the CloudWatch verification step, you can automatically verify cloud application and infrastructure performance across your deployment. For more information, see Add a Workflow and Add a Pipeline.

Workflow Verification

To see the results of Harness machine-learning evaluation of your CloudWatch verification, in your workflow or pipeline deployment you can expand the Verify Service step and then click the CloudWatch step.

[screenshot]

Continuous Verification

You can also see the evaluation in the Continuous Verification dashboard. The workflow verification view is for the DevOps user who developed the workflow. The Continuous Verification dashboard is where all future deployments are displayed for developers and others interested in deployment analysis.

To learn about the verification analysis features, see the following sections.

Deployments

Deployment info
See the verification analysis for each deployment, with information on its service, environment, pipeline, and workflows.

Verification phases and providers
See the vertfication phases for each vertfication provider. Click each provider for logs and analysis.

Verification timeline
See when each deployment and verification was performed.

Transaction Analysis

Execution details
See the details of verification execution. Total is the total time the verification step took, and Analysis duration is how long the analysis took.

Risk level analysis
Get an overall risk level and view the cluster chart to see events.

Transaction-level summary
See a summary of each transaction with the query string, error values comparison, and a risk analysis summary.

Execution Analysis

Event type
Filter cluster chart events by Unknown Event, Unexpected Frequency, Anticipated Event, Baseline Event, and Ignore Event.

Cluster chart
View the chart to see how the selected event contrast. Click each event to see its log details.

Event Management

Event-level analysis
See the threat level for each event captured.

Tune event capture
Remove events from analysis at the service, workflow, execution, or overall level.

Event distribution
Click the chart icon to see an event distribution including the measured data, baseline data, and event frequency.

Next Steps


How did we do?