Skip to main content

Alert policies

Reading time: 0 minute(s) (0 words)

Setting up effective alert policies is critical for proactive reliability management and ensuring error budgets are managed efficiently. Nobl9 makes this process easier with alert presets—predefined templates designed for a variety of use cases.

Whether you're addressing fast error budget burns, monitoring long-term trends, or customizing alerts for specific thresholds, alert presets let you create policies quickly and intuitively.

This article dives into the available alert policy presets, customization options, and how they integrate seamlessly with your existing SLOs, helping you optimize your alerting strategy.

Overview​

The following presets are available in Nobl9:

Fast burn
Alert when a high momentary burn rate risks exhausting the budget (any time window).
Slow burn for long window SLOs
Alert when the budget burns slowly and is not recovering (time windows over a week).
Slow burn for short window SLOs
Alert when the budget burns slowly and is not recovering (time windows up to one week).
Budget almost exhausted
Alert when the error budget is nearly exhausted (any time window).
Fast exhaustion above budget
Alert when the error budget is exhausting significantly and there’s still some budget remaining (any time window).
Fast exhaustion below budget
Alert when the error budget is exhausting significantly and there’s no remaining budget left (any time window).
Slow exhaustion for long window SLOs
Alert when the error budget is exhausting slowly and is not recovering (time windows over a week).
Slow exhaustion for short window SLOs
Alert when the error budget is exhausting slowly and is not recovering (time windows up to one week).
Fast budget drop
Alert me when the budget dropped by 10% over the last 15 minutes and is not recovering.
Slow budget drop
Alert me when the budget dropped by 5% over the last 1 hour and is not recovering.

Configuring alert presets​

To access alert presets, go to Alerts and click . The alert policy wizard opens.

Here, you can see the available alert policy presets and tailor any of them to your needs. Also, you can create an alert policy from scratch using the Custom option at the bottom of the preset list.

Creating alert policy from preset
Image 1: Creating new Alert Policy from a Preset

To start adjusting the preset, click Use Preset.

Customizing the preset​

You may need to customize the alert preset to suit your needs better. There are two ways to create a custom alert policy in Nobl9:

  • Select the Custom option in step 1 of the alert policy wizard
  • Select an existing preset and rewrite its default settings

Once you're done editing, you can apply the preset to create an alert policy with your custom values. Your edited preset will then be accessible from the alert policies view.

tip

Nobl9 won't persist the customized version of an alert preset. Opening the alert preset view again will restore the original preset definitions.

If you want to re-use the same custom configuration of a preset for a different alert policy, you can duplicate the alert policy created using the customized Alert Preset. The easiest way to do that is to copy the YAML configuration of your customized preset by running thesloctl get alertpolicies [alert_policy_name] command and copying the specs.[...] block.

Alert policy presets in YAML​

You can also use presets in YAML format. Use a configuration template that fits your needs, updating the project field to match your setup, and specifying your preferred alertMethod.

Fast burn
apiVersion: n9/v1alpha
kind: AlertPolicy
metadata:
  name: fast-burn
  displayName: Fast burn (20x5min)
  project: default
spec:
  alertMethods: []
  conditions:
    - alertingWindow: 5m
      measurement: averageBurnRate
      value: 20
      op: gte
  coolDown: 5m
  description: There’s been a significant spike in burn rate over a brief period.
  severity: High
Same project for alert policies and SLOs

Alert policies are limited to a specific project, so they must be assigned to the same project as the SLO they are linked to. The templates above create policies in the default project.

Alert presets in Terraform​

Alert presets are also available in Terraform. They can be treated as an example, to set up your own alert policies using predefined, Nobl9 recommended configuration.

You can access them in our Terraform Documentation.

Last updated on
On this page