2 - 24/7 Service Guard for Stackdriver

Updated 1 month ago by Michael Cretzman

Harness 24/7 Service Guard monitors your live applications, catching problems that surface minutes or hours following deployment.

Once you have set up a Stackdriver Verification Provider in Harness, as described in 1- Stackdriver Connection Setup, you can add your Stackdriver Logs and Metrics monitoring to Harness 24/7 Service Guard in your Harness Application Environment.

For more information on 24/7 Service Guard, see 24/7 Service Guard.

In this topic:

24/7 Service Guard Setup

To set up 24/7 Service Guard for Stackdriver, do the following:

  1. Ensure that you have added Stackdriver as a Harness Verification Provider, as described in 1- Stackdriver Connection Setup.
  2. In your Harness Application, ensure that you have added a Service, as described in Services. For 24/7 Service Guard, you do not need to add an Artifact Source to the Service, or configure its settings. You simply need to create a Service and name it. It will represent your application for 24/7 Service Guard.
  3. In your Harness Application, click Environments.
  4. In Environments, ensure that you have added an Environment for the Service you added. For steps on adding an Environment, see Environments.
  5. Click the Environment for your Service. Typically, the Environment Type is Production.
  6. In the Environment page, locate 24/7 Service Guard.
  7. In 24/7 Service Guard, click Add Service Verification, and then click Stackdriver Log or Stackdriver Metric. The Stackdriver Log or Stackdriver Metric dialog appears.

  1. Fill out the dialog. The dialog has the fields described in the respective sections below.

Stackdriver Logs

For 24/7 Service Guard, the queries you define to collect logs are specific to the application or service you want monitored. Verification is application/service level. This is unlike Workflows, where verification is performed at the host/node/pod level.

The Stackdriver Logs dialog allows you to use your Stackdriver log queries in Harness as part of 24/7 Service Guard:

The following table describes the settings for Stackdriver Logs:

Field

Description

Display Name

The name that will identify this service on the Continuous Verification dashboard. Use a name that indicates the environment and monitoring tool, such as Stackdriver.

Service

The Harness Service to monitor with 24/7 Service Guard.

GCP Cloud Provider

Select the GCP Cloud Provider to use, as described in Stackdriver Connection Setup. If you currently connect to GCP via a Kubernetes Cluster Cloud Provider, you must set up a GCP Cloud Provider for access to the Stackdriver data on your cluster.

Search Keywords

Enter search keywords for your query. You can use the same filters you have in GCP Logs Viewer.

Simply copy a filer entry into Search Keywords:

To use multiple filter entries, place an AND between them or use multiline entries. For example:

For advanced filter examples, see Advanced filters library from GCP.

For information on the log entries used, see Viewing Logs from GCP.

Host Name Field

Enter the log field that contains the name of the host/pod/container for which you want logs. You can enter a pod ID or field name for example.

Harness uses this field to group data and perform analysis at the container-level.

For example, the query in Search Keywords looks for pods labelled nginx-deployment:

resource.type="container"
resource.labels.pod_id:"nginx-deployment-"

In Host Name Field, you would enter pod_id because it is the log field containing the pod name. In a log, this field will be in the resource section:

...
resource: {
labels: {
cluster_name: "doc-example"
container_name: "harness-delegate-instance"
instance_id: "1733097732247470454"
namespace_id: "harness-delegate"
pod_id: "harness-sample-k8s-delegate-wverks-0"
project_id: "exploration-161417"
zone: "us-central1-a"
}
type: "container"
}
...

Enable 24/7 Service Guard

Click the checkbox to enable 24/7 Service Guard.

Baseline

Select the baseline time unit for monitoring. For example, if you select For 4 hours, Harness will collect the logs for the last 4 hours as the baseline for comparisons with future logs. If you select Custom Range you can enter a Start Time and End Time.

When you are finished, the dialog will look something like this:

  1. Click TEST. Harness verifies the settings you entered.
  2. Click SUBMIT. The Stackdriver 24/7 Service Guard is configured.

To see the running 24/7 Service Guard analysis, click Continuous Verification. The 24/7 Service Guard dashboard displays the production verification results.

For more information, see 24/7 Service Guard Overview.

Stackdriver Metrics

The Stackdriver Metrics dialog allows you to use your existing Stackdriver metrics in Harness as part of 24/7 Service Guard.

The following table describes the settings for Stackdriver Metrics:

Field

Description

Display Name

The name that will identify this service on the Continuous Verification dashboard. Use a name that indicates the environment and monitoring tool, such as Stackdriver.

Service

The Harness Service to monitor with 24/7 Service Guard.

GCP Cloud Provider

Select the GCP Cloud Provider to use, as described in Stackdriver Connection Setup. If you currently connect to GCP via a Kubernetes Cluster Cloud Provider, you must set up a new GCP Cloud Provider for access to the Stackdriver data on your cluster.

Metrics To Monitor

In this section you define the Stackdriver metrics you want to monitor. For example, here is a Stackdriver Metrics Explorer configured to monitor Kubernetes container restarts, filtered by a cluster name and grouped by cluster.

To reproduce these settings in Metrics To Monitor, you simply copying its filter and group by details via its JSON.

Metric Name, Metric Type, Group Name

In Metric Name, enter a name to identify the metric in Harness, such as Restarts. This is not the Stackdriver-specific name of a metric.

In Metric Type, select the type of metric to monitor, such as Infra.

In Group Name, enter a name for grouping the metrics in Harness, such as PodRestarts. The Group Name is useful when you want Harness to monitor multiple metrics, and be able to group them.

JSON Query

Paste in the JSON query from Stackdriver Metrics Explorer.

In Stackdriver Metrics Explorer, once you have your metric query set up, click the View as JSON option.

Next, click COPY JSON.

In Harness Stackdriver Metrics, in JSON Query, paste in the JSON.

Algorithm Sensitivity

See CV Strategies, Tuning, and Best Practices.

Enable 24/7 Service Guard

Click the checkbox to enable 24/7 Service Guard.

When you are finished, the dialog will look something like this:

Click TEST. Harness verifies the settings you entered.

Click SUBMIT. The Stackdriver 24/7 Service Guard is configured.

To see the running 24/7 Service Guard analysis, click Continuous Verification. The 24/7 Service Guard dashboard displays the production verification results.

Next Step


How did we do?