Replay
Beta

Reading time: 0 minute(s) (0 words)

Replay provides the ability to retrieve SLI data collected by your data source in the past.

You can use Replay when you need to recover missing or corrupted SLI data from your data source. It also allows you to create Service Level Objectives (SLOs) by retrieving historical data from the start, eliminating the need to wait for data to accumulate naturally over time. Additionally, Replay can help you backfill your SLO reporting. If you have a backlog of SLI data from the past few days or weeks, you can retrieve it using Replay and recalculate your remaining error budget.

How far back in time Replay can retrieve SLI data depends on the maximum period for historical data retrieval initially allowed by your data source.

Since using Replay can be resource-intensive, you might want to further limit this maximum duration at the integration level to avoid overloading. To do this, specify your preferred Maximum Period for Historical Data Retrieval when adding a data source to your Nobl9 organization.

The value for the maximum period for historical data retrieval must be less or equal to the value initially allowed by your data source.

When you need to auto-replay SLOs on creation, configure this option for your data source.

Data retrieval period

Replay always retrieves SLI data up to the time it's been launched—you select only how far back the data will be retrieved.

Every organization in Nobl9 is limited to two concurrent Replays.
Subsequent Replays are queued and processed in order as Replay slots open up.

Replay at a glance

With Replay, you can access historical data for new and existing SLOs.
Replay fetches historical data while your SLO collects new data in real-time. The historical and current data are merged, producing an error budget calculated for the entire period.
Data sources limit the data retrieval period. This period can be limited even more for your data source in Nobl9.
Replay always retrieves data until now.¹
You can run two Replays simultaneously; further Replays are queued.
You can track Replays in the Job Status widget.
You can cancel queued Replays in the Job Status widget. However, cancelling ongoing Replays isn't possible.
SLOs with ongoing Replays display no data in the tiles and charts over the retrieved period until Replay is complete. Queued Replays don't affect SLO's tiles and charts.
¹"Now" means the moment of triggering Replay

Scope of support

Supported data sources for Replay, their minimum required agent versions, and the maximum period for historical data retrieval:

Data source name	Nobl9 agent minimum version	Maximum period for historical data retrieval
Amazon CloudWatch	`0.65.0`	15 days
Amazon Prometheus	`0.65.0`	30 days
AppDynamics	`0.68.0`	30 days
Azure Monitor	`0.69.0-beta01`	30 days
Azure Monitor managed service for Prometheus	`0.78.0-beta`	30 days
Datadog	`0.65.0`	30 days
Dynatrace	`0.66.0`	28 days
Elasticsearch	`0.85.0-beta`	30 days
Google Cloud Monitoring	`0.79.0-beta`	30 days
Graphite	`0.65.0`	30 days
LogicMonitor	`0.81.0-beta`	30 days
New Relic	`0.65.0`	30 days
Prometheus	`0.65.0`	30 days
ServiceNow Cloud Observability	`0.65.0`	30 days
Splunk	`0.65.0`	30 days

Amazon CloudWatch: only the Configurations queries are supported.
Dynatrace: when you retrieve historical data for the maximum period (28), Dynatrace limitations can cause a one-hour interval of a degraded resolution.
Google Could Monitoring: the historical data retrieval period can be less for some metrics, depending on Google's data retention period.

Replay configuration

Nobl9 Web
sloctl

In the Data source wizard, define the Advanced settings > Historical data retrieval:

The Maximum period for historical data retrieval value sets the limits for how far back to the past Nobl9 can query data from your data source.
The maximum period must be less or equal to the period initially allowed by your data source.
The Default period for historical data retrieval defines the period applied to replay SLOs based on your data source. This value is suggested by default as Period for historical data retrieval for SLOs created based on this data source. This value must be less or equal to the Maximum period for historical data retrieval set for this data source.

Requirements:

The values must be the whole positive numbers or zero.
Default period for historical data retrieval must be up to the maximum period set for this data source. You can override this value when creating SLOs.
Maximum period for historical data retrieval must be up to the maximum period allowed initially by your data source.
To replay SLOs based on this data source immediately on creation, set Maximum period for historical data retrieval to a non-zero value.

In your data source's YAML definition for agent or direct, define the values for historicalDataRetrieval:

maxDuration sets the limits for how far back to the past Nobl9 can query data from your data source.
The maximum period must be less than or equal to the period initially allowed by your data source.
defaultDuration defines the period applied to replay SLOs based on your data source.

Example of a data source YAML definition with the auto-replay objects
- apiVersion: n9/v1alpha
  # Oneof: Agent, Direct
  kind: Direct
  metadata:
    displayName: My data source with auto-replay
    name: my-data-source
    project: my-project
  spec:
    # Your data source-specific fields
    # ...
    historicalDataRetrieval:
      defaultDuration:
        unit: Day
        value: 0
      maxDuration:
        unit: Day
        value: 30
    # Query parameters configuration: query interval, query delay, jitter, and timeont
    interval:
      unit: Minute
      value: 1
    queryDelay:
      unit: Minute
      value: 5
    jitter:
      unit: Second
      value: 45
    timeout:
      unit: Second
      value: 50
    # Event logs available for direct connections only
    logCollectionEnabled: true
    # Oneof: beta, stable. Use beta for Replay
    releaseChannel: beta

Run sloctl apply to proceed.

Learn more about YAML configuration for data sources connected using the agent or direct methods.

Requirements:

The values must be the whole positive numbers or zero.
The value for defaultPeriod must be up to the maximum period set for this data source.
The maxDuration value must be up to the maximum period allowed initially by your data source.
To replay SLOs based on this data source immediately on creation, set maxDuration to a non-zero value.

You can also set up replaying SLOs immediately on creation with the Nobl9 Terraform provider. For this, specify the value for retrieve_historical_data_from in your SLO definition.