Skip to main content

Prometheus

Reading time: 0 minute(s) (0 words)

Prometheus is an open-source software application used for event monitoring and alerting. It records real-time metrics in a time series database built using an HTTP pull model, with flexible query language and real-time alerting.

Prometheus parameters and supported features in Nobl9
General support:
Release channel: Stable, Beta
Connection method: Agent
Replay and SLI Analyzer: Historical data limit 30 days
Event logs: Not supported
Query checker: Not supported
Query parameters retrieval: Supported
Timestamp cache persistence: Supported
Query parameters:
Query interval: 1 min
Query delay: 0 sec
Jitter: 15 sec
Timeout: 30 sec
Agent details and minimum required versions for supported features:
Plugin name: n9prometheus
Query delay environment variable: PROM_QUERY_DELAY
Replay and SLI Analyzer: 0.65.0
Query parameters retrieval: 0.73.2
Timestamp cache persistence: 0.65.0
Custom HTTP headers: 0.88.0 / 0.83.0-beta

Creating SLOs with Prometheus​

Nobl9 Web​

Follow the instructions below to create your SLOs with Prometheus on the Nobl9 Web:

  1. Navigate to Service Level Objectives.

  2. Click .

Step 1: Select the service the SLO will be associated with.

Step 2:

  1. Select your Prometheus data source.
  2. Specify Metric and enter the query:

The threshold metric evaluates a single time series against a threshold value you set.

3. Enter the query. For example: myapp_server_requestMsec{host="*",job="nginx"}

Step 3: define a Time Window for your SLO.

  • Rolling time windows are better for tracking the recent user experience of a service.
  • Calendar-aligned windows are best suited for SLOs that are intended to map to business metrics measured on a calendar-aligned basis, such as every calendar month or every quarter.

Step 4: specify the Error Budget Calculation Method and your Objective(s).

  • Occurrences method counts good attempts against the count of total attempts.
  • Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
  • You can define up to 12 objectives for an SLO.

See the use case example and the SLO calculations guide for more information on the error budget calculation methods.

Step 5: add the Display name, Name, and other settings for your SLO:

  • Set no data anomaly detection.
    When activated, Nobl9 notifies you if your SLO hasn't received data for the period you set.
  • Add alert policies, labels, and links, if required.
    You can add up to 20 links per SLO.

Click Create SLO.

YAML​

Sample Prometheus threshold SLO
apiVersion: n9/v1alpha
kind: SLO
metadata:
  name: api-server-slo
  displayName: API Server SLO
  project: default
  labels:
    area:
      - latency
      - slow-check
    env:
      - prod
      - dev
    region:
      - us
      - eu
    team:
      - green
      - sales
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example Prometheus SLO
  indicator:
    metricSource:
      name: prometheus
      project: default
      kind: Agent
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good response (200)
      value: 200
      name: ok
      target: 0.95
      rawMetric:
        query:
          prometheus:
            promql: api_server_requestMsec{host="*",job="nginx"}
      op: lte
      primary: true
  service: api-server
  timeWindows:
    - unit: Month
      count: 1
      isRolling: false
      calendar:
        startTime: '2022-12-01 00:00:00'
        timeZone: UTC
  alertPolicies:
    - fast-burn-5x-for-last-10m
  attachments:
    - url: https://docs.nobl9.com
      displayName: Nobl9 Documentation
  anomalyConfig:
    noData:
      alertMethods:
        - name: slack-notification
          project: default
      alertAfter: 1h

Specification for metric from Prometheus always has one mandatory field:

  • promql – a Prometheus query in the language called PromQL | Prometheus documentation (Prometheus Query Language) that lets the user select and aggregate time series data in real time.

Querying the Prometheus server​

The Nobl9 agent leverages the Prometheus API parameters. It pulls data at a per-minute interval from the Prometheus server.

Check out these related guides and references:
Add Prometheus as a data sourceAdding data sourcesSetting up a Prometheus SLO with Nobl9VideoCreating SLOs using TerraformTerraform
On this page