3 - Verify Deployments with Datadog
Harness can analyze Datadog metrics to verify, rollback, and improve deployments. To apply this analysis to your deployments, you set up Datadog as a verification step in a Harness Workflow.
Once you run a deployment, and Datadog preforms verification, Harness' machine-learning verification analysis will assess the risk level of the deployment.
In this topic:
Deployment Verification Setup
You can use Datadog metrics or logs for verification. They are configured as separate Workflow steps, but most of their settings are the same.
To verify your deployment with Datadog logs, do the following:
- Ensure that you have added Datadog as a verification provider, as described above.
- In your workflow, under Verify Service, click Add Verification, and then click Datadog Metrics or Datadog Log. The Datadog Metrics or Datadog Log dialog appears.
- Both dialogs have the same fields, unless noted otherwise.
Datadog Metrics/Log Server
Select the Datadog verification provider you added, as described above.
Datadog Metrics only
Here you can select any of the Datadog API metrics. For a list of the API metrics, see Data Collected from Datadog.
Datadog Metrics only
Datadog Custom Metrics
Datadog Metrics only
Datadog Log only
Enter search keywords, such as
Expression for Host/Container name
Enter an expression that evaluates to the host/container/pod name tagged in the Datadog events.For example, in Datadog, a Kubernetes deployment might use the tag pod_name to identify the pod where the microservice is deployed. Find the where the same name is identified in the deployment environment, and use that path as the expression.For example, locate the pod name in the Datadog Event Stream page:
Analysis Time duration
Set the duration for the verification step. If a verification step exceeds the value, the workflow Failure Strategy is triggered. For example, if the Failure Strategy is Ignore, then the verification state is marked Failed but the workflow execution continues.
Baseline for Risk Analysis
Execute with previous steps
Check this checkbox to run this verification step in parallel with the previous steps in Verify Service.
- For Datadog Metrics, click TEST. Harness verifies the settings you entered.
- When you are finished, click SUBMIT. The Datadog verification step is added to your workflow.
Datadog and ECS
For ECS-based deployments, Datadog uses the container ID to fetch data for both metrics and logs. Harness can fetch the container ID if the Harness Delegate is running on same ECS cluster as the container or the Delegate must be in same AWS VPC and port 51678 must be open for incoming traffic.
Once you have deployed your workflow (or pipeline) using the Datadog verification step, you can automatically verify cloud application and infrastructure performance across your deployment. For more information, see Add a Workflow and Add a Pipeline.
To see the results of Harness machine-learning evaluation of your Datadog verification, in your workflow or pipeline deployment you can expand the Verify Service step and then click the Datadog step.
You can also see the evaluation in the Continuous Verification dashboard. The workflow verification view is for the DevOps user who developed the workflow. The Continuous Verification dashboard is where all future deployments are displayed for developers and others interested in deployment analysis.
To learn about the verification analysis features, see the following sections.
Deployment info: See the verification analysis for each deployment, with information on its service, environment, pipeline, and workflows.
Verification phases and providers: See the vertfication phases for each vertfication provider. Click each provider for logs and analysis.
Verification timeline: See when each deployment and verification was performed.
Execution details: See the details of verification execution. Total is the total time the verification step took, and Analysis duration is how long the analysis took.
Risk level analysis: Get an overall risk level and view the cluster chart to see events.
Transaction-level summary: See a summary of each transaction with the query string, error values comparison, and a risk analysis summary.
Event type: Filter cluster chart events by Unknown Event, Unexpected Frequency, Anticipated Event, Baseline Event, and Ignore Event.
Cluster chart: View the chart to see how the selected event contrast. Click each event to see its log details.
Event-level analysis: See the threat level for each event captured.
Tune event capture: Remove events from analysis at the service, workflow, execution, or overall level.
Event distribution: Click the chart icon to see an event distribution including the measured data, baseline data, and event frequency.