2 - 24/7 Service Guard for Elasticsearch

Updated 2 weeks ago by Michael Cretzman

Harness 24/7 Service Guard monitors your live applications, catching problems that surface minutes or hours following deployment. For more information, see 24/7 Service Guard Overview.

You can add your Elasticsearch monitoring to Harness 24/7 Service Guard in your Harness Application Environment. For a setup overview, see .

This section assumes you have a Harness Application set up and containing a Service and Environment. For steps on setting up a Harness Application, see Application Checklist.

24/7 Service Guard Setup

To set up 24/7 Service Guard for Elasticsearch, do the following:

  1. Ensure that you have added ELK Elasticsearch as a Harness Verification Provider, as described in Verification Provider Setup.
  2. In your Harness Application, ensure that you have added a Service, as described in Services. For 24/7 Service Guard, you do not need to add an Artifact Source to the Service, or configure its settings. You simply need to create a Service and name it. It will represent your application for 24/7 Service Guard.
  3. In your Harness Application, click Environments.
  4. In Environments, ensure that you have added an Environment for the Service you added. For steps on adding an Environment, see Environments.
  5. Click the Environment for your running microservice. Typically, the Environment Type is Production.
  6. In the Environment page, locate 24/7 Service Guard.
  7. In 24/7 Service Guard, click Add Service Verification, and then click ELK. The ELK dialog appears.
  8. Fill out the dialog. The dialog has the following fields.
For 24/7 Service Guard, the queries you define to collect logs are specific to the application or service you want monitored. Verification is application/service level. This is unlike Workflows, where verification is performed at the host/node/pod level.

Field

Description

Display Name

The name that will identify this service on the Continuous Verification dashboard. Use a name that indicates the environment and monitoring tool, such as ELK.

Service

The Harness Service to monitor with 24/7 Service Guard.

ELK Server

Select the ELK Verification Provider to use.

Search Keywords

Enter search keywords for your query, such as error or exception.

Query Type

Select TERM to finds documents that contain the exact term specified in the inverted index. MATCH queries accept text, numerics, and dates, analyze them, and construct a query. If you want the query analyzed, then use MATCH.

Index

Enter the the index to search. This field is automatically populated from the index templates, if available.

Message Field

Enter the field by which the messages are usually indexed. Typically, a log field.

To find the field in Kibana and enter it in Harness, do the following:

  1. In Kibana, click Discover.
  2. In the search field, search for error or exception.
  3. In the results, locate a log for the host/container/pod ELK is monitoring. For example, in the following Kubernetes results in Kibana, the messages are indexed under the log field.
  4. In Harness, in the ELK dialog, next to Message Field, click Guide From Example. The Message Field popover appears.
  5. In the JSON response, click on the name of the label that maps to the log in your Kibana results. Using our Kubernetes example, you would click the log label.

    The label is added to the Message Field.

Timestamp Field

Enter the timestamp field in the Elasticsearch record, such as @timestamp.

Timestamp Format

Enter the format for the timestamp field in the Elasticsearch record. Use Kibana to determine the format.

In Kibana, use the Filter feature in Discover to construct your timestamp range:

Format Examples:

Timestamp: 2018-08-24T21:40:20.123Z. Format: yyyy-MM-dd'T'HH:mm:ss.SSSX

Timestamp: 2018-08-30T21:57:23+00:00. Format: yyyy-MM-dd'T'HH:mm:ss.SSSXXX

For more information, see Data Math from Elastic.

Algorithm Sensitivity

Select the Algorithm Sensitivity.

Enable 24/7 Service Guard

Click the checkbox to enable 24/7 Service Guard.

Baseline

Select the baseline time unit for monitoring. For example, if you select For 4 hours, Harness will collect the logs for the last 4 hours as the baseline for comparisons with future logs. If you select Custom Range you can enter a Start Time and End Time.

When you are finished, the dialog will look something like this:

  1. Click TEST. Harness verifies the settings you entered.
  2. Click SUBMIT. The ELK 24/7 Service Guard is configured.

To see the running 24/7 Service Guard analysis, click Continuous Verification.

The 24/7 Service Guard dashboard displays the production verification results.

For more information, see 24/7 Service Guard Overview.

Next Step


How did we do?