PagerDuty alert method

Reading time: 0 minute(s) (0 words)

The PagerDuty alert method sends alerts through PagerDuty to notify Nobl9 users whenever an incident is triggered.

Authentication

PagerDuty requires specifying the Integration Key that allows it to send you an incident alert. For more details on how to obtain the Integration Key, go to Services and Integrations | PagerDuty documentation.

Adding the PagerDuty alert method

You can add the PagerDuty alert method directly in the Nobl9 Web application.

Alternatively, utilize the SLOs-as-code approach by:

• Applying an alert method YAML configuration with the sloctl apply command.
• Configuring the PagerDuty alert method definition programmatically using the Nobl9 Terraform provider.

Nobl9 Web

To set up the alert method on the Nobl9 Web, follow these steps:

1. Navigate to Integrations > Alert methods.
2. Click .
3. Select PagerDuty.

4. Enter the Integration Key (mandatory).
   For more information, refer to the Authentication section.

5. Select a Project (mandatory).
   Project is a way to organize your Nobl9 resources and manage access to them.
   When Project is skipped, Nobl9 uses the default project.
6. Enter a Display Name (optional).
   Spaces are allowed.
7. Enter a Name (mandatory).
   The name is mandatory and can only contain lowercase, alphanumeric characters, and dashes (for example, my-alert-method-1). Nobl9 duplicates the display name here, transforming it into the supported format, but you can edit the result.
8. Enter a Description (optional).
   Provide extra details about it, its purpose, responsible persons, etc.
   Up to 1050 characters.

9. Configure Alert resolution (optional) to receive a closing event after the Cooldown period is over.

   • The optional message will be sent in the closing event notification.
     
10. Click Add alert method.

PagerDuty alerts severity mapping

PagerDuty returns alert severity levels, which Nobl9 maps to its own severity levels for consistency in handling alerts. Below is the mapping of N9 severity levels to the severities returned by PagerDuty:

PagerDuty severity	Nobl9 severity
Warning	Low
Error	Medium
Critical	High

YAML configuration for PagerDuty alerts

Example

General YAML sample for PagerDuty alert method

apiVersion: n9/v1alpha
kind: AlertMethod
metadata:
  name: string # Mandatory
  displayName: string # Optional
  project: default # Mandatory
  # Optional
  # The key value: only alphanumeric characters, `-`, `_`, or `.` are allowed.
  # The key value must start and end with an alphanumeric character.
  # An optional DNS subdomain prefix followed by `/` is allowed.
  annotations:
    key: value
spec:
  description: string # Optional
  pagerduty:
    integrationKey: # PagerDuty integration key

Example

Working YAML sample for PagerDuty alert method

apiVersion: n9/v1alpha
kind: AlertMethod
metadata:
  name: pagerduty
  displayName: PagerDuty Alert Method
  project: default
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example PagerDuty Alert Method
  pagerduty:
    integrationKey: "123456789"
    sendResolution:
      message: Alert is now resolved

Configuring sendResolution for PagerDuty - YAML

It is possible to configure the PagerDuty alert method to send additional messages to PagerDuty to resolve the incident. This can be configured by adding an optional section sendResolution to your PagerDuty YAML configuration.

sendResolution feature for PagerDuty adheres to the following rules:

• If you added the section sendResolution in your YAML configuration, then notification is sent to resolve an incident in PagerDuty.

• By default sendResolution is nil. This means that the all-clear notification is not sent by default.

• sendResolution[ ]:message - string, optional.

  • If you provide value for this field, it is attached to the summary of incident resolution.
  note

  Only PagerDuty supports sending all-clear to the external system. For other alert method types, this section is ignored.

• Here's an example of the sendResolution YAML configuration for PagerDuty:

  apiVersion: n9/v1alpha
  kind: AlertMethod
  metadata:
    name: pager-duty-notification-sending-resolution
    displayName: PagerDuty notification
  spec:
    description: Open new Incident with "Nobl9 Test" service
    pagerDuty:
      integrationKey: "[secret]"
      sendResolution:
        message: Alert is now resolved

Sending notification for a resolved alert to PagerDuty

If you configure the PagerDuty alert method to send all-clear, an additional message is sent to PagerDuty service to resolve the incident.

• PagerDuty UI:
  

• PagerDuty events API:

note

The all-clear message is sent for both alert statuses: Canceled and Resolved.

Testing the PagerDuty alert method in the UI

You can test alert methods for all notification services supported by the Nobl9 platform through the UI.

To verify the correctness of your alert method configuration:

1. Navigate to Integrations > Alert methods.
2. Click the required alert method.
   Its Details page opens.
3. Click Test under the alert method name:
   
   
   • When the configuration is correct, Nobl9 displays Test completed successfully! under the button.
   • Otherwise, the Test failed message appears. It contains the details of incorrect settings.

Useful links

For a more in-depth look, consult additional resources:

PagerDuty alert methodTerraform provider