Skip to main content

Prometheus

Reading time: 0 minute(s) (0 words)

Prometheus is an open-source software application used for event monitoring and alerting. It records real-time metrics in a time series database built using an HTTP pull model, with flexible query language and real-time alerting.

Prometheus parameters and supported features in Nobl9
General support:
Release channel: Stable, Beta
Connection method: Agent
Replay and SLI Analyzer: Historical data limit 30 days
Event logs: Not supported
Query checker: Not supported
Query parameters retrieval: Supported
Timestamp cache persistence: Supported
Query parameters:
Query interval: 1 min
Query delay: 0
Jitter: 15 sec
Timeout: 30 sec
Agent details and minimum required versions for supported features:
Plugin name: n9prometheus
Query delay environment variable: PROM_QUERY_DELAY
Replay and SLI Analyzer: 0.65.0
Query parameters retrieval: 0.73.2
Timestamp cache persistence: 0.65.0
Custom HTTP headers: 0.83.0-beta

Creating SLOs with Prometheus​

Nobl9 Web​

Follow the instructions below to create your SLOs with Prometheus on the Nobl9 Web:

  1. Navigate to Service Level Objectives.

  2. Click .

Step 1: Select the service the SLO will be associated with.

Step 2:

  1. Select your Prometheus data source.
  2. Specify Metric and enter the query:

The threshold metric evaluates a single time series against a threshold value you set.

3. Enter the query. For example: myapp_server_requestMsec{host="*",job="nginx"}

Step 3: define a Time Window for your SLO.

  • Rolling time windows are better for tracking the recent user experience of a service.
  • Calendar-aligned windows are best suited for SLOs that are intended to map to business metrics measured on a calendar-aligned basis, such as every calendar month or every quarter.

Step 4: specify the Error Budget Calculation Method and your Objective(s).

  • Occurrences method counts good attempts against the count of total attempts.
  • Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
  • You can define up to 12 objectives for an SLO.

See the use case example and the SLO calculations guide for more information on the error budget calculation methods.

Step 5: add the Display name, Name, and other settings for your SLO:

  • Set notification on data, if this option is available for your data source.
    When activated, Nobl9 notifies you if your SLO hasn't received data or received incomplete data for more than 15 minutes.
  • Add alert policies, labels, and links, if required.
    You can add up to 20 links per SLO.

Click Create SLO.

sloctl​

Sample Prometheus threshold SLO
apiVersion: n9/v1alpha
kind: SLO
metadata:
  name: api-server-slo
  displayName: API Server SLO
  project: default
  labels:
    area:
      - latency
      - slow-check
    env:
      - prod
      - dev
    region:
      - us
      - eu
    team:
      - green
      - sales
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example Prometheus SLO
  indicator:
    metricSource:
      name: prometheus
      project: default
      kind: Agent
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good response (200)
      value: 200
      name: ok
      target: 0.95
      rawMetric:
        query:
          prometheus:
            promql: api_server_requestMsec{host="*",job="nginx"}
      op: lte
      primary: true
  service: api-server
  timeWindows:
    - unit: Month
      count: 1
      isRolling: false
      calendar:
        startTime: '2022-12-01 00:00:00'
        timeZone: UTC
  alertPolicies:
    - fast-burn-5x-for-last-10m
  attachments:
    - url: https://docs.nobl9.com
      displayName: Nobl9 Documentation
  anomalyConfig:
    noData:
      alertMethods:
        - name: slack-notification
          project: default

Specification for metric from Prometheus always has one mandatory field:

  • promql – a Prometheus query in the language called PromQL | Prometheus documentation (Prometheus Query Language) that lets the user select and aggregate time series data in real time.

Querying the Prometheus server​

The Nobl9 agent leverages the Prometheus API parameters. It pulls data at a per-minute interval from the Prometheus server.

For a more in-depth look, consult additional resources:
Add Prometheus as a data sourceAdding data sourcesSetting up a Prometheus SLO with Nobl9VideoCreating SLOs using TerraformTerraform
Last updated on
On this page