Skip to main content

Azure Monitor managed service for Prometheus

Reading time: 0 minute(s) (0 words)

Azure Monitor managed service for Prometheus is a part of Azure Monitor Metrics. It allows collecting Prometheus metrics and analyzing them with Azure Monitor tools.

Integration with Nobl9 lets you collect metrics from Azure Monitor managed service for Prometheus and create SLOs based on them.

Azure Monitor managed service for Prometheus parameters and supported features in Nobl9
General support:
Release channel: Beta
Connection method: Agent, Direct
Replay and SLI Analyzer: Supported
Event logs: Supported
Query checker: Not supported
Query parameters retrieval: Supported
Timestamp cache persistence: Supported
Query parameters:
Query interval: 1 min
Query delay: 0
Jitter: 15 sec
Timeout: 30 sec
Agent details and minimum required versions for supported features:
Environment variable: PROM_QUERY_DELAY
Plugin name: n9prometheus
Replay and SLI Analyzer: 0.78.0-beta
Maximum historical data retrieval period: 30 days
Query parameters retrieval: 0.78.0-beta
Timestamp cache persistence: 0.78.0-beta
Custom HTTP headers: 0.83.0-beta
Additional notes:
Support for Prometheus metrics
Learn more Opens in a new tab

Authentication​

To query an Azure Monitor workplace, authenticate with your Microsoft Entra ID with client_id and client_secret. For this:

  1. Register an Azure application with Microsoft Entra ID.
  2. Assign your application the Monitoring Data Reader role to your required Azure Monitor workspace.
    This role meets the Nobl9 requirements for metric collection.

You can also use sloctl. This way, you can configure SLOs for your Azure Cloud application without the resource and metric autocompletion.

Azure IAM

We recommend granting the Monitoring Data Reader role on the subscription or resource group level rather than a specific resource. A broader scope provides a more comprehensive choice of subscriptions, resource groups, resources, and metrics in the Nobl9 platform.

URL​

The Azure Monitor managed service for Prometheus agent requests the Range queries API endpoint in the /api/v1/query_range form. For example:

GET /api/v1/query_range
POST /api/v1/query_range

Omit the /api/v1/query_range API path from the URL. Specify only the base URL for your Prometheus server.
For example, if your Prometheus server is available under <http://prometheus.example.com> and you access API via <http://prometheus.example.com/api/v1>, then use only the <http://prometheus.example.com> part.

Other APIs or Web UIs have similar path endings. Omit them in the URL: for example, the /graph part of the path.

This integration focuses on querying the Prometheus servers, not fetching metrics directly from services. Avoid using URLs pointing to service endpoints that expose data in the Prometheus format (often under the /metrics path).

Learn about how to find a query endpoint.

Adding Azure Monitor managed service for Prometheus as a data source​

You can connect the Azure Monitor managed service for Prometheus data source using the direct or agent connection methods.

Direct connection method​

Nobl9 Web​

  1. Navigate to Integrations > Sources.
  2. Click .
  3. Click the required Source button.
  4. Choose Direct.
release channel
Currently, Nobl9 integration with Azure Monitor managed service for Prometheus is available in the Beta release channel only.

  1. Enter the URL of your required base Prometheus server.

  2. Enter your Azure Tenant ID.
    It is an 8-4-4-4-12-character code containing digits 0-9 and letters Aa-Ff.

  3. Enter your Microsoft Entra Client ID and Client Secret.

  1. Select a Project.
    Specifying a project is helpful when multiple users are spread across multiple teams or projects. When the Project field is left blank, Nobl9 uses the default project.
  2. Enter a Display Name.
    You can enter a user-friendly name with spaces in this field.
  3. Enter a Name.
    The name is mandatory and can only contain lowercase, alphanumeric characters, and dashes (for example, my-project-1). Nobl9 duplicates the display name here, transforming it into the supported format, but you can edit the result.
  4. Enter a Description.
    Here you can add details such as who is responsible for the integration (team/owner) and the purpose of creating it.
  5. Specify the Query delay to set a customized delay for queries when pulling the data from the data source.
    • The default value in Azure Monitor managed service for Prometheus integration for Query delay is 0 minutes.
    info
    Changing the Query delay may affect your SLI data. For more details, check the Query delay documentation.
  6. Enter a Maximum Period for Historical Data Retrieval.
    • This value defines how far back in the past your data will be retrieved when replaying your SLO based on this data source.
    • The maximum period value depends on the data source.
      Find the maximum value for your data source.
    • A greater period can extend the loading time when creating an SLO.
      • The value must be a positive integer.
  7. Enter a Default Period for Historical Data Retrieval.
    • It is used by SLOs connected to this data source.
    • The value must be a positive integer or 0.
    • By default, this value is set to 0. When you set it to >0, you will create SLOs with Replay.
  8. Click Add Data Source.

sloctl​

  1. Create a YAML resource definition for your Azure Monitor managed service for Prometheus using the provided template.

  2. Run sloctl apply to proceed.

- apiVersion: n9/v1alpha
  kind: Direct
  metadata:
    # Optional
    displayName: Azure Prometheus data source (direct connection)
    name: azure-prometheus-data-source
    project: my-project
  spec:
    azurePrometheus:
      # Replace the clientId and clientSecret placeholders with your Microsoft Entra ID credentials
      clientId: "<YOUR_CLIENT_ID>" # secret
      clientSecret: "<YOUR_CLIENT_SECRET>" # secret
      tenantId: <YOUR_AZURE_TENANT_ID>
      url: "<YOUR_PROMETHEUS_SERVER_URL>"
    # Replay configuration
    historicalDataRetrieval:
      maxDuration:
        # Numeric, up to 30 days for Azure monitor managed service for Prometheus
        value: 30
        unit: Day
      defaultDuration:
        value: 15
        unit: Day
    # The default query delay value for Azure Monitor managed service for Prometheus is 0 seconds
    queryDelay:
      value: 0
      unit: Second
FieldTypeDescription
queryDelay.unit
mandatory
enumSpecifies the unit for the query delay. Possible values: Second | Minute.
• Check query delay documentation for default unit of query delay for each source.
queryDelay.value
mandatory
numericSpecifies the value for the query delay.
• Must be a number less than 1440 minutes (24 hours).
• Check query delay documentation for default unit of query delay for each source.
releaseChannel
mandatory
enumSpecifies the release channel. Accepted values: beta | stable.
Source-specific fields
azurePrometheus.clientId
mandatory
string, secretYour Microsoft Entra ID client ID.
azurePrometheus.clientSecret
mandatory
string, secretYour Microsoft Entra ID client secret.
azurePrometheus.tenantID
mandatory
stringThe identifier of your Microsoft Entra tenant.
azurePrometheus.url
mandatory
stringBase URL to Prometheus server. See authentication section above for more details.
Replay-related fields
historicalDataRetrieval
optional
n/aOptional structure related to configuration related to Replay.
❗ Use only with supported sources.
• If omitted, Nobl9 uses the default values of value: 0 and unit: Day for maxDuration and defaultDuration.
maxDuration.value
optional
numericSpecifies the maximum duration for historical data retrieval. Must be integer ≥ 0. See Replay documentation for values of max duration per data source.
maxDuration.unit
optional
enumSpecifies the unit for the maximum duration of historical data retrieval. Accepted values: Minute | Hour | Day.
defaultDuration.value
optional
numericSpecifies the default duration for historical data retrieval. Must be integer ≥ 0 and ≤ maxDuration.
defaultDuration.unit
optional
enumSpecifies the unit for the default duration of historical data retrieval. Accepted values: Minute | Hour | Day.

Agent connection method​

Use the agent method to run an agent alongside your Prometheus server. Once connected, the agent will periodically connect to Nobl9 using an outbound connection.

Nobl9 Web​

To connect Azure Monitor managed service for Prometheus, do the following:

  1. Navigate to Integrations > Sources.
  2. Click .
  3. Click the required Source button.
  4. Choose Agent.
release channel
Currently, Nobl9 integration with Azure Monitor managed service for Prometheus is available in the Beta release channel only.

  1. Enter the URL of your required base Prometheus server.

  2. Enter your Azure Tenant ID.
    It is an 8-4-4-4-12-character code containing digits 0-9 and letters Aa-Ff.

  1. Select a Project.
    Specifying a project is helpful when multiple users are spread across multiple teams or projects. When the Project field is left blank, Nobl9 uses the default project.
  2. Enter a Display Name.
    You can enter a user-friendly name with spaces in this field.
  3. Enter a Name.
    The name is mandatory and can only contain lowercase, alphanumeric characters, and dashes (for example, my-project-1). Nobl9 duplicates the display name here, transforming it into the supported format, but you can edit the result.
  4. Enter a Description.
    Here you can add details such as who is responsible for the integration (team/owner) and the purpose of creating it.
  5. Specify the Query delay to set a customized delay for queries when pulling the data from the data source.
    • The default value in Azure Monitor managed service for Prometheus integration for Query delay is 0 minutes.
    info
    Changing the Query delay may affect your SLI data. For more details, check the Query delay documentation.
  6. Enter a Maximum Period for Historical Data Retrieval.
    • This value defines how far back in the past your data will be retrieved when replaying your SLO based on this data source.
    • The maximum period value depends on the data source.
      Find the maximum value for your data source.
    • A greater period can extend the loading time when creating an SLO.
      • The value must be a positive integer.
  7. Enter a Default Period for Historical Data Retrieval.
    • It is used by SLOs connected to this data source.
    • The value must be a positive integer or 0.
    • By default, this value is set to 0. When you set it to >0, you will create SLOs with Replay.
  8. Click Add Data Source.

sloctl​

  1. Create a YAML resource definition for your Azure Monitor managed service for Prometheus using the provided template.

  2. Run sloctl apply to proceed.

- apiVersion: n9/v1alpha
  kind: Agent
  metadata:
    name: azure-prometheus-data-source
    # Optional
    displayName: Azure Prometheus data source (agent connection)
    project: my-project
  spec:
    # Optional
    description: My sample Azure Prometheus data source (agent connection)
    # Enum: stable || beta. Currently, available in beta only
    releaseChannel: beta
    azurePrometheus:
      url: "<YOUR_PROMETHEUS_SERVER_URL>"
      tenantId: "<YOUR_AZURE_TENANT_ID>"
    # Replay configuration
    historicalDataRetrieval:
      maxDuration:
        # Numeric, up to 30 days for Azure monitor managed service for Prometheus
        value: 30
        unit: Day
      defaultDuration:
        value: 15
        unit: Day
    # The default query delay value for Azure Monitor managed service for Prometheus is 0 seconds
    queryDelay:
      value: 0
      unit: Second
FieldTypeDescription
queryDelay.unit
mandatory
enumSpecifies the unit for the query delay. Possible values: Second | Minute.
• Check query delay documentation for default unit of query delay for each source.
queryDelay.value
mandatory
numericSpecifies the value for the query delay.
• Must be a number less than 1440 minutes (24 hours).
• Check query delay documentation for default unit of query delay for each source.
releaseChannel
mandatory
enumSpecifies the release channel. Accepted values: beta | stable.
Source-specific fields
azurePrometheus.url
mandatory
stringBase URL to Prometheus server. See authentication section above for more details.
azurePrometheus.tenantID
mandatory
stringThe identifier of your Microsoft Entra tenant.
Replay-related fields
historicalDataRetrieval
optional
n/aOptional structure related to configuration related to Replay.
❗ Use only with supported sources.
• If omitted, Nobl9 uses the default values of value: 0 and unit: Day for maxDuration and defaultDuration.
maxDuration.value
optional
numericSpecifies the maximum duration for historical data retrieval. Must be integer ≥ 0. See Replay documentation for values of max duration per data source.
maxDuration.unit
optional
enumSpecifies the unit for the maximum duration of historical data retrieval. Accepted values: Minute | Hour | Day.
defaultDuration.value
optional
numericSpecifies the default duration for historical data retrieval. Must be integer ≥ 0 and ≤ maxDuration.
defaultDuration.unit
optional
enumSpecifies the unit for the default duration of historical data retrieval. Accepted values: Minute | Hour | Day.

Agent deployment​

When you add a data source, Nobl9 automatically generates a Kubernetes configuration and a Docker command line for you to use to deploy the agent. Both of these are available on the Nobl9 Web, under the Agent Configuration section.

Swap in your credentials (e.g., replace the <YOUR_AZURE_APPLICATION_CLIENT_ID> and <YOUR_AZURE_APPLICATION_CLIENT_SECRET> with your client ID and client secret).

To deploy the created agent to a Kubernetes cluster, do the following:

  1. Create a YAML config file using the provided template.
  2. Run sloctl apply to proceed.
    The agent facilitates Nobl9 to import your service metrics.
# DISCLAIMER: This deployment description contains only the fields necessary for the purpose of this demo.
# It is not a ready-to-apply k8s deployment description.

apiVersion: v1
kind: Secret
metadata:
  name: azure-prometheus-data-source
  namespace: default
type: Opaque
stringData:
  azure_client_id: "<AZURE_MONITOR_CLIENT_ID>"
  azure_client_secret: "<AZURE_MONITOR_CLIENT_SECRET>"
  client_id: "<NOBL9_CLIENT_ID>"
  client_secret: "<NOBL9_CLIENT_SECRET>"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: azure-prometheus-data-source
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      nobl9-agent-name: "azure-prometheus-agent"
      nobl9-agent-project: "my-project"
      nobl9-agent-organization: "my-organization"
  template:
    metadata:
    spec:
      containers:
        - name: agent-container
          image: nobl9/agent:0.88.0-beta
          resources:
            requests:
              memory: "700Mi"
              cpu: "0.2"
          env:
            - name: N9_CLIENT_ID
              valueFrom:
                secretKeyRef:
                  key: client_id
                  name: azure-prometheus-data-source
            - name: N9_AZURE_MONITOR_CLIENT_ID
              valueFrom:
                secretKeyRef:
                  key: azure_client_id
                  name: azure-prometheus-data-source
            - name: N9_AZURE_MONITOR_CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  key: azure_client_secret
                  name: azure-prometheus-data-source
            - name: N9_INTAKE_URL
              value: "<YOUR_VALUE>"
            - name: N9_QUERYENGINE_URL
              value: "<YOUR_VALUE"
            - name: N9_CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  key: client_secret
                  name: azure-prometheus-data-source
            - name: N9_AUTH_SERVER
              value: "<YOUR_VALUE>"
            - name: N9_OKTA_ORG_URL
              value: "<YOUR_VALUE>"
            - name: N9_METRICS_PORT
              value: "9090"
            - name: N9_NATS_URL
              # A wss:// URL
              value: "<YOUR_VALUE>"

Important notes:

  • The N9_METRICS_PORT is a variable specifying the port to which the /metrics and /health endpoints are exposed.
  • 9090 is the default value and can be changed.
  • If you don’t want the metrics to be exposed, comment out or delete the N9_METRICS_PORT variable.
  • N9_DIAGNOSTIC_QUERY_LOG_SAMPLE_INTERVAL_MINUTES sets the frequency of log emission for SLI Analyzer. The default value is 10. To deactivate log emission, set this value to 0.

Creating SLOs with Azure Monitor managed service for Prometheus​

Nobl9 integration with Azure Monitor managed service for Prometheus supports Prometheus metrics.

You can create SLOs based on Azure Monitor managed service for Prometheus using the Nobl9 Terraform provider or applying a YAML definition with sloctl.

Nobl9 Web​

Follow the instructions below to create your SLOs with Azure Monitor managed service for Prometheus on the Nobl9 Web:

  1. Navigate to Service Level Objectives.

  2. Click .

Step 1: Select the service the SLO will be associated with.

Step 2:

  1. Select your Azure Monitor managed service for Prometheus data source.
  2. Configure Replay: set the Period for historical data retrieval.
    It can be 0 or a positive integer up to 30.
  3. Specify Metric and enter the PromQL query:

The threshold metric evaluates a single time series against a threshold value you set.

4. Enter the query. For example: sum(rate(prometheus_http_requests_total{code=~"^2.*"}[1h]))

Step 3: define a Time Window for your SLO.

  • Rolling time windows are better for tracking the recent user experience of a service.
  • Calendar-aligned windows are best suited for SLOs that are intended to map to business metrics measured on a calendar-aligned basis, such as every calendar month or every quarter.

Step 4: specify the Error Budget Calculation Method and your Objective(s).

  • Occurrences method counts good attempts against the count of total attempts.
  • Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
  • You can define up to 12 objectives for an SLO.

See the use case example and the SLO calculations guide for more information on the error budget calculation methods.

Step 5: add the Display name, Name, and other settings for your SLO:

  • Set notification on data, if this option is available for your data source.
    When activated, Nobl9 notifies you if your SLO hasn't received data or received incomplete data for more than 15 minutes.
  • Add alert policies, labels, and links, if required.
    You can add up to 20 links per SLO.

Click Create SLO.

sloctl​

Azure Monitor managed service for Prometheus is case-insensitive.
Refer to the YAML SLO reference for details.

Here’s an example of Azure Monitor managed service for Prometheus Metrics using a rawMetric (the threshold metric):

# Metric type: threshold
# Budgeting method: Occurrences
# Time window type: Calendar
- apiVersion: n9/v1alpha
  kind: SLO
  metadata:
    name: my-threshold-slo
    # Optional
    displayName: My threshold SLO based on Azure Monitor managed service for Prometheus
    project: my-project
    # Optional
    labels:
      key-1:
        - value-1
        - value-2
      key-2:
        - value-1
        - value-2
    # Optional
    annotations:
      key-1: value-1
      key-2: value-1
  spec:
    # Optional
    description: My sample threshold SLO based on Azure Monitor managed service for Prometheus
    indicator:
      metricSource:
        name: azure-prometheus-data-source
        project: my-project
        kind: Direct
    # Enum: Occurrences || Timeslices
    budgetingMethod: Occurrences
    objectives:
      # Optional
      - displayName: My objective 1
        value: 200.0
        name: my-objective
        target: 0.95
        rawMetric:
          query:
            azurePrometheus:
              promql: |-
                sum((rate(container_cpu_usage_seconds_total{container!="POD",container!=""}[30m])
                - on (namespace,pod,container) group_left avg by (namespace,pod,container)(kube_pod_container_resource_requests{resource="cpu"}))
                * -1 >0)
        op: lte
        primary: true
    service: my-service
    timeWindows:
      - unit: Month
        count: 1
        isRolling: false
        calendar:
          startTime: 2022-12-01 00:00:00
          timeZone: UTC
    # Optional. Up to 20 alert policies per SLO
    alertPolicies:
      - my-alert-policy
    # Optional
    attachments:
      - url: https://my-url.com
        displayName: My URL
    # Optional, beta functionality
    anomalyConfig:
      noData:
        alertMethods:
          - name: my-alert-method
            project: my-project

Querying the Azure Monitor managed service for Prometheus API​

The Nobl9 agent leverages the Prometheus API parameters. It pulls data at a per-minute interval from the Prometheus server.

For a more in-depth look, consult additional resources:
Agent metricsAgentCreating SLOs via TerraformTerraformCreating agents via TerraformTerraformHow to find a tenant IDAzure documentationGetting ID values for Azure MonitorAzure documentationAzure Monitor managed service for PrometheusAzure documentationQuery Prometheus metrics using the API and PromQLAzure documentationQuerying Prometheus: BasicsPrometheus documentationPrometheus metricsAzure documentation
On this page