Replay restrictions and impact

Reading time: 0 minute(s) (0 words)

Prerequisite

To ensure Replay operation for your required data source, set the data source's maximum period for historical data retrieval to five minutes or more.

• Two slots are allocated for every organization. This means you can run two Replays at a time.
  Track job progress in the Job Status widget.
• Required roles for handling Replays:
  • Organization admin
  • Project owner or editor
• Cancelling a Replay is available only at the data retrieval step. Once Replay proceeds with data processing, you cannot terminate the job.
• Replaying a standard SLO can take a considerable time. It depends on the following:
  • The length of the replayed period
  • The number of objectives in your SLO
  • The number of unique queries used in your SLO

Data sources can alter data from the past

Metric gathering systems usually downsample older data using different aggregate functions like mean or sum or simply by dropping data points. This is aimed at saving space and can affect the result of a query made against a time range in the past. Refer to the documentation of your data source for more details.

Running a Replay for an existing SLO has important consequences for SLI data, reports, and alerts.

Although SLOs continue to collect data during a Replay, data processing is paused. As a result, SLO visualizations are also paused—the Active alerts and Active data anomalies tiles do not update live, and SLO charts display gaps. As soon as the Replay is complete, tiles resume live updates, and charts fill in the gaps.

SLO type support

Replay is not supported for composite SLOs.

SLI data

Replay impacts SLI data by rewriting it for the selected period with data retrieved from the data source. This process can lead to changes that may affect your SLO's reporting accuracy and visualization.

Here are the possible effects of Replay on SLI data:

1. Overwritten data
   • During a Replay, existing data for the selected period is completely replaced with data retrieved from the source.
   • SLO charts might look different after a Replay, reflecting the recalculated data. This can include shifts in trends or altered visualizations.
2. Downsampled or lower-resolution data
   • Data retrieved by Replay may already be downsampled by the data source, reducing its granularity.
   • Imported data may have a lower resolution than the original, which could affect the precision of charts and calculations.
3. Unaddressed or additional data gaps
   • Replay does not fill data points that are missing from the data source itself, introducing new gaps in the data stream.

Editing SLOs during Replay

Adding objectives to an SLO, while this SLO is being replayed, results in the following:

• Replay continues for the existing objectives of this SLO.
• New objectives' historical data won't be considered in calculating the SLO error budget. The new objectives are considered in error budget calculations without historical data, from the moment of saving changes to the SLO.

Reports and dashboards

Reports and dashboards depend on accurate SLI data to provide insights into service performance. Since Replay overrides previously collected data, it can alter data resolution or introduce gaps. As a result, calculations and insights in reports and dashboards might differ from previous versions, impacting trend analysis.

Alerts

• Alerting is suspended for the entire Replay duration and resumes once the Replay is complete.
• Once the Replay finishes:
  • Alerts that were already sent for the historical period will not be sent again.
  • Missed alerts—those that would have triggered while the Replay was running—are sent based on the recalculated data.

Impact on alerts

• Alerting is suspended for the entire Replay duration and resumed once Replay is complete.
• Once Replay is complete:
  • You won't receive already received alerts for the recalculated historical period again.
  • You receive missed alerts: the alerts triggered when Replay was running.
    These alerts are triggered based on the recalculated data.

Impact on composite SLOs

Currently, you can't replay a composite SLO, but only its components.

An ongoing Replay on component SLOs impacts any composites that contain them, as the components temporarily stop reporting data until the process is complete.

The exact impact depends on several factors:

• The duration of the Replay
• The composite's max delay setting
• Whether other components in the composite SLO are reporting data without interruption

Table: Composite SLO calculations during a component Replay

Are other components reporting data?	Replay duration vs. max delay	Composite SLO calculations
Yes	Replay < max delay	The composite SLO's calculations pause for the duration of the Replay. After it finishes, new data from the component is included as normal.
Yes	Replay > max delay	The component is treated as delayed according to the max delay setting. New data is processed normally after Replay is complete.
No	Any ratio	The composite SLO's calculations pause. Once Replay is complete, the historical data from the replayed component is used to fill the data gap.

Useful links

Check out these related guides and references:

ReplayReplayReplaying SLOsReplaysloctl replayReplayReplaying SLOs from other SLOsReplay