Skip to main content

sloctl replay

Reading time: 0 minute(s) (0 words)

You can replay SLOs using sloctl commands. With sloctl, you can replay SLOs in bulk and remove queued Replays from the queue.

🧪sloctl version requirements

We recommend updating sloctl to these versions or higher for the following functionality:

Subcommands​

The available sloctl replay subcommands are as follows:

SubcommandDescriptionAvailable flags
cancelCancel an ongoing Replay
Works only during data import		-p, --project: cancel Replays from a specified project
deleteRemove the Replay from the queue--all: delete all queued Replays
-p, --project: delete Replays from a specified project
listGet the list of all Replays in an organizationGlobal flags only

Usage​

path to your target directory

sloctl executes commands in the %UserProfile%\.config\nobl9\ directory on Windows and ~/.config/nobl9/ on other operating systems.

Provide the path to your SLO definition when it's located in a different directory.

To replay an SLO, run:

sloctl replay -p [project-name] --from=[YYYY-MM-DDThh:mm:ssZ] [slo-name]

Provide the following:

ParameterDescriptionDefault / alternative
[project-name]The name identifier of your required SLO's parent projectYour context's default project
[YYYY-MM-DDThh:mm:ssZ]The start time of the required period for historical data retrieval in the RFC 3339 formatSpecify the historical data retrieval start in the YAML definition you're
[slo-name]The name identifier of your required SLON/A

You can replay SLOs providing YAML definitions. For this, run one of the following commands:

CommandDescription
sloctl replay -f [my-slo.yaml]Replay an SLO providing its YAML definition
sloctl replay <./[my-slo.yaml]Replay an SLO using its definition YAML from stdin

To replay several SLOs in bulk, add as many SLOs to the YAML definition as you need. In this case, the first two Replays start immediately, provided all Replay slots are free. For further Replays, sloctl reports if they are queued successfully. Failures can occur because of the following:

ReasonTroubleshooting
A given SLO is currently being replayed or queuedWait until Replay is complete, or
remove Replay from the queue
Not enough permissions to replay SLOsTo replay SLOs, you role must be Project owner, Project editor, or Organization admin
No access to parent projects of the involved SLOsAsk the required project owner for access
Replay period for an SLO exceeds the maxDuration period set for the connected data sourceNarrow down Replay period.
Find maximum period for historical data retrieval allowed by your data source

List Replays​

To list Replays, run:

sloctl replay list

sloctl returns the list of all ongoing and queued Replays in your organization.

FieldDescriptionNotes
sloThe name identifier of an SLO being replayedRequires access to SLO's parent project
projectThe name identifier of the replayed SLO's parent projectIndicates the project holding a replayed or queued SLO
elapsedTimeHow long Replay is runningRequires access to SLO's parent project
retrievedScopeThe replayed timeframeRequires access to SLO's parent project
retrievedFromThe timeframe start timeRequires access to SLO's parent project
statusReplay status
  • in progres: the SLO is being replayed
  • queued: Replay is in queue
    • cancellationThe possibility to cancel data import
  • possible: data is being imported, you can cancel Replay
  • requested: Replay cancellation requested, waiting for the response from the backend
  • denied: data import is over, cancellation request cannot be completed
  • blocked: data is being imported, you cannot cancel Replay

    • When no Replays are queued or in progress at the moment when you run sloctl replay list, the command returns Replay not found.

    Remove from queue​

    You can also clear the queue in your organization. For this, remove Replays one by one.

    When you remove a Replay from the queue, Replay proceeds with the next SLO in the queue.

    To remove a Replay from the queue, run one of the following commands:

    CommandDescription
    sloctl replay delete [slo-name] -p [project_name]Removes an individual Replay from the queue
    sloctl replay delete -allRemoves all Replays from the queue

    Cancel Replay​

    You can cancel Replay at the data import step. For this, run

    sloctl replay cancel <slo-name> [flags]

    Available flags:

    FlagShorthandDescription
    --project-pThe name identifier of a project holding your required SLO

    Applying SLOs with --replay​

    You can apply SLOs and replay them immediately. For this, apply the required SLO YAML definitions followed by --replay:

    sloctl apply -f [SLO YAML definition] --replay --from=[time in the RFC 3339 format]

    This way, you apply the YAML and replay your SLO at the same time.

    Replaying SLOs from other SLOs​

    🧪Experimental feature

    This is an experimental feature currently under development. It is available only with sloctl version 0.6.0 or higher.

    You can also replay SLOs using other SLOs as the historical data source. For this, run sloctl replay providing a configuration file.

    In the configuration file, provide your required source SLO name with the sourceSLO field. When no source SLO is provided, Replay re-imports historical data from the SLO's original data source.

    To replay an SLO using historical data of another SLO, do the following:

    1. Prepare the configuration YAML.
      You can provide more than one target SLO in this configuration. All target SLOs provided in such configuration will be replayed using a source SLO.
      For example,
      Sample YAML configuration
      # Specifies the SLO to replay
      - slo: target-slo-1
        # The project where the target SLO is located
        project: target-project-1
        # The timestamp indicating when to begin the replay
        from: 2023-02-02T15:02:05Z
        # Defines the SLO to be used as the data source for the replay
        sourceSLO:
          slo: source-slo-1
          project: source-project-1
          # The objectivesMap field defines how objectives from the source SLO map to those of the target SLO
          objectivesMap:
            # A list of mappings, where each mapping defines a source objective from sourceSLO to a target objective in the target SLO
            # All objectives in the target SLO must be mapped to a source objective
            # The same source objective can be mapped to multiple target objectives
            # Not all source objectives must be used
            - source: objective-1
              target: objective-1
            - source: objective-2
              target: objective-2
            - source: objective-3
              target: objective-3
          # DISCLAIMER: The sourceSLO configurations do not affect the target SLO
          # Only the SLI is copied, and no changes are made to the configuration of the target SLO

      # Specifies the second SLO to replay
      - slo: target-slo-2
        project: target-project-2
        from: 2023-02-02T15:02:05+02:00
        sourceSLO:
          slo: source-slo-2
          project: source-project-2
          objectivesMap:
            - source: objective-1
              target: objective-1
    2. Run: 
      sloctl replay -f ./your-configuration.yaml
    mixed replay configuration

    A single configuration file can contain settings for replaying from both—the original data source and another SLO.

    Functional requirements​

    • If sourceSLO is omitted, Replay uses the original data source of the target SLO.
    • sourceSLO and its objectives must exist before replaying.
    • slo and sourceSLO must use the same metric type (threshold or ratio). For ratio metrics, both SLOs must have the same numerator type (good or bad).
    • slo and sourceSLO must use the same subtype of ratio metric (incremental or non-incremental) to avoid incorrect results or overwriting replayed SLOs with invalid data.
    • slo and sourceSLO can belong to different projects and use different data sources, time windows, queries, budgeting methods, targets, and values.
    • You can provide the same SLO as slo and sourceSLO. This replays your SLO using its own existing SLI data. This way, no new data is imported from the data source.

    Constraints​

    • You cannot replay an SLO while it's used as a sourceSLO.
    • You can modify or delete a sourceSLO while it's in use. Any changes made after Replay has started won't affect the ongoing Replay process.
    • RBAC: to trigger Replay, you must have at least view permission for the project holding the source SLO.
    • Both the target SLO's data source and the sourceSLO's data source must support Replay.

    Key takeaways​

    Replaying SLOs is a permanent action

    Replaying SLOs is a permanent action and cannot be rolled back.

    Learn more about Replay impact on connected resources.

    • Replay can fail in the following cases:
      • Your data source is unsupported
      • The maximum period for historical data retrieval for your data source is set to 0
      • A given SLO is currently being replayed
      • Replay for a given SLO is queued
    • When your SLO has alert policies attached, alerting is suspended for the entire Replay duration. It resumes once the Replay process is complete.
    • If the project is not set, it is inferred from Nobl9 config.toml for the current context.
    • If from is not provided in the config file, you must specify it with --from=[time] flag.
    • Setting the --project or --from=[] flags don't override the configuration values.

    Minimum configuration with parameters can be set globally with flags:

    cat <<EOF | sloctl replay -p my-project --from 2024-10-20T15:00:00Z
    - slo: my-slo
    - slo: my-another-slo
    EOF
    For a more in-depth look, consult additional resources:
    ReplayReplayReplay restrictionsReplayReplaying SLOsReplay
    Last updated on
    On this page