Bugsnag Verification

Updated 2 months ago by Michael Cretzman

Bugsnag provides error reporting libraries for every major software platform, detecting and reporting errors in your apps, and capturing diagnostic data for each error. Bugsnag captures your app’s exceptions or events and groups them into errors according to their root causes.

Bugsnag provides unique browser-focused reporting. See Browser-Based Benefits below.

You can add a Bugsnag verification step to your workflow and Bugsnag will be used by Harness to verify the performance and quality of your deployments using Harness machine-learning verification analysis.

Reporting with Bugsnag

Harness Analysis

Browser-Based Benefits

Bugsnag is particularly useful for browser-based apps, as it collects browser information as part of its exception and error capture. This can helpful in determining if a new version of a browser is causing problems for users. Here is an example from Bugsnag:

Once you have deployed your app via Harness, you can add host/node-focused verification to your Harness workflow using another Verification Provider, and use Bugsnag to focus on browser-based issues. Here is an example of a Harness verification where other verification tools have been used to verify host/node issues, and Bugsnag is added as the last verification step to capture browser-based issues:

When you set up Bugsnag as a verification step in a Harness workflow, you can indicate if your app is browser-based. When Harness arrives at the Bugsnag verification step, Harness will ignore deployment host or node information and focus on browser-based data. This browser focus enables you to capture browser issues on their own after you have already ensured that the deployment host/node environment is running correctly.

Harness can now use this browser data with the machine-learning in its Continuous Verification and determine what events are causing errors or have to the potential to cause errors in the future.

For information about advanced browser event capturing in Bugsnag, see React integration guide from Bugsnag.

Verification Setup Overview

You set up Bugsnag and Harness in the following way:

  1. Using Bugsnag, you monitor your microservice or application.
  2. In Harness, you connect Harness to your Bugsnag account, adding Bugsnag as a Harness Verification Provider.
  3. After you have built and run a successful deployment of your microservice or application in Harness, you then add Bugsnag verification steps to your Harness deployment workflow.
    1. If you like, you can use Bugsnag to focus solely on browser-based exceptions and errors.
  4. Harness uses Bugsnag to verify your future microservice/application deployments.
  5. Harness Continuous Verification uses unsupervised machine-learning to analyze your deployments and Bugsnag analytics/logs, discovering events that might be causing your deployments to fail. Then you can use this information to improve your deployments.

Intended Audience

  • Developers
  • DevOps

Before You Begin

Connect to Bugsnag

Connect Harness to Bugsnag to have Harness verify the success of your deployments. Harness will use your tools to verify deployments and use its machine learning features to identify sources of failures.

To add Bugsnag as a verification provider, do the following:

  1. Click Setup.
  2. Click Connectors.
  3. Click Verification Providers.
  4. Click Add Verification Provider, and select Bugsnag. The Add Bugsnag Verification Provider dialog for your provider appears.

The Add Bugsnag Verification Provider dialog has the following fields.

Field

Description

Bugsnag URL

Enter https://api.bugsnag.com/. This is the URL for the Bugsnag API.

Auth Token

Enter the Data Access API personal auth token for your Bugsnag account. Here is how you get the token in Bugsnag:

  1. In Bugsnag, click your account icon in the top-right of any page, and then click Settings.
  2. In the My account page, next to Data Access API, click Personal auth tokens.
  3. In Personal auth tokens, click GENERATE NEW TOKEN.


    The Generate new auth token dialog appears.
  4. In Token description, enter a name, such as Harness, and click GENERATE. The auth token is generated.
  5. Click Copy to clipboard.
  6. In Harness, in the Add Bugsnag Verification Provider dialog, paste the token in the Auth Token field.

Display Name

The name for this Bugsnag verification provider connection in Harness. If you will have multiple Bugsnag connections, enter a unique name. You will use this name to select this connection when integrating Bugsnag with the Verify Steps of your workflows, described below.

Usage Scope

If you want to restrict the use of a provider to specific applications and environments, do the following:

In Usage Scope, click the drop-down under Applications, and click the name of the application.

In Environments, click the name of the environment.

Verify with Bugsnag

The following procedure describes how to add Bugsnag as a verification step in a Harness workflow. For more information about workflows, see Add a Workflow.

Once you run a deployment and Bugsnag preforms verification, Harness' machine-learning verification analysis will assess the risk level of the deployment.

In order to obtain the useful comparison data, the verification provider should be added to your workflow after you have run at least one successful deployment.

To verify your deployment with Bugsnag, do the following:

  1. Ensure that you have added Bugsnag as a verification provider, as described above.
  2. In your workflow, under Verify Service, click Add Verification, and then click Bugsnag. The Bugsnag dialog appears.

The Bugsnag dialog has the following fields.

Field

Description

Bugsnag Server

Select the Bugsnag verification provider you added to Harness, as described above. Harness will immediately use the connection to obtain organization and project information from Bugsnag.

Organization ID

Select the Organization ID for your Bugsnag account. In Bugsnag, this is the Organization name in the account's Organization page:

Project ID

Select the Project ID for the Bugsnag project you want to use. In Bugsnag, this is the Project Name in the Projects page:

Browser Application

Click the checkbox to have Harness ignore host/node deployment events and focus on the browser events Bugsnag captures.

Expression for Host/Container name

If you do not enable the Browser Application checkbox, Harness will use the host/node/container event data Bugsnag captures.

Add a variable that evaluates to the hostname value in the host field of event messages. For example, in a Bugsnag message in a Harness deployment verification, if you look at a event message, you will see a hosts field:

Next, look in the JSON for the host/container/pod in the deployment environment and identify the label containing the same hostname. The path to that label is what the expression should be in Expression for Host/Container name. The default variable is ${host.hostName}. In most cases, this expression will work.

Analysis Time duration

Set the duration for the verification step. If a verification step exceeds the value, the workflow Failure Strategy is triggered. For example, if the Failure Strategy is Ignore, then the verification state is marked Failed but the workflow execution continues.

Baseline for Risk Analysis

Select Previous Analysis to have this verification use the previous analysis for a baseline comparison. If your workflow is a Canary workflow type, you can select Canary Analysis to have this verification compare old versions of nodes to new versions of nodes in real-time.

Execute with previous steps

Check this checkbox to run this verification step in parallel with the previous steps in Verify Service.

Failure Criteria

Specify the sensitivity of the failure criteria. When the criteria is met, the workflow Failure Strategy is triggered.

Include instances from previous phases

If you are using this verification step in a multi-phase deployment, select this checkbox to include instances used in previous phases when collecting data. Do not apply this setting to the first phase in a multi-phase deployment.

When you are finished, click SUBMIT. The Bugsnag verification step is added to your workflow.

Verification Results

Once you have deployed your workflow (or pipeline) using the Bugsnag verification step, you can automatically verify cloud application and infrastructure performance across your deployment. For more information, see Add a Workflow and Add a Pipeline.

Workflow Verification

To see the results of Harness machine-learning evaluation of your Bugsnag verification, in your workflow or pipeline deployment you can expand the Verify Service step and then click the Bugsnag step.

Continuous Verification

You can also see the evaluation in the Continuous Verification dashboard. The workflow verification view is for the DevOps user who developed the workflow. The Continuous Verification dashboard is where all future deployments are displayed for developers and others interested in deployment analysis.

To learn about the verification analysis features, see the following sections.

Deployments

Deployment info
See the verification analysis for each deployment, with information on its service, environment, pipeline, and workflows.

Verification phases and providers
See the vertfication phases for each vertfication provider. Click each provider for logs and analysis.

Verification timeline
See when each deployment and verification was performed.

Transaction Analysis

Execution details
See the details of verification execution. Total is the total time the verification step took, and Analysis duration is how long the analysis took.

Risk level analysis
Get an overall risk level and view the cluster chart to see events.

Transaction-level summary
See a summary of each transaction with the query string, error values comparison, and a risk analysis summary.

Execution Analysis

Event type
Filter cluster chart events by Unknown Event, Unexpected Frequency, Anticipated Event, Baseline Event, and Ignore Event.

Cluster chart
View the chart to see how the selected event contrast. Click each event to see its log details.

Event Management

Event-level analysis
See the threat level for each event captured.

Tune event capture
Remove events from analysis at the service, workflow, execution, or overall level.

Event distribution
Click the chart icon to see an event distribution including the measured data, baseline data, and event frequency.

Next Steps


How did we do?