Skip to main content

SLI Analyzer
Beta

Reading time: 0 minute(s) (0 words)

Realizing which reliability target to choose is difficult as it requires understanding your system's performance over the past month or year. To do so, you usually need to create and adjust targets every few weeks.

With SLI Analyzer, you can retrieve your historical data, set reliability targets, and see how your system performs. The SLI Analyzer queries for a specific time series from a specified data source. It then analyzes the data and provides metrics for that particular SLI.

You can quickly adjust the targets and view updated error budgets and error budget burn down. Once you've determined the right target using your existing data, you can easily create a new SLO.

Leveraging SLI Analyzer, you can:

  • Create and instantly adjust your first SLO on the Nobl9 platform for more robust monitoring of your reliability.
  • Choose a meaningful reliability target by analyzing how your SLOs will behave before you even create them.
  • Understand the reliability of your services without waiting and adjusting them every few days or weeks.

Scope of Support for SLI Analyzer​

The SLI Analyzer supports the following data sources:

  • Datadog
  • Prometheus
  • AMS Prometheus
  • Splunk
  • New Relic
  • Amazon CloudWatch
  • Graphite
  • Dynatrace
    beta
    (available only in beta channel)
  • Lightstep

To use SLI Analyzer for specific data sources, you may need to update the version of the Nobl9 Agent you use. Check the table to determine the minimum Agent version required to use this feature.

note

You can use the beta version of SLI Analyzer only for new SLOs based on the historical data that you can fetch from the supported data sources.

Once you've determined the right target using your existing data, you can create an SLO using those parameters.

SLI Analyzer and RBAC​

The following list presents the overview of permissions required to use SLI Analyzer:

  • Only Organization Admins or users with Project Owner, or Project Editor permissions can view, create, edit, and delete analyses.

    note

    A project that is used by SLI Analyzer is inherited from the data source. This has important implications on what users can view and whether they can create and edit analyses.

  • Users having Organization Viewer or Responder, Project Viewer, Responder, or Integrations User permissions can preview existing analyses in the SLI Analyzer tab.

    note

    If you're an Organization or Project Viewer, make sure you have the necessary project-level permissions to create and edit analyses for a specific data source.

  • Users having Organization User permissions cannot create, delete, or preview analyses in the SLI Analyzer tab.

    caution

    Make sure the data source has the Maximum Period for Data Retrieval set to greater than or equal to 5 minutes.

    Otherwise, you won't be able to run an analysis for that data source:

    too small period for max data retrieval

Limitations​

A quota limits the number of ongoing analyses. Those limits differ for two parts of the analysis - importing the data and analyzing (calculating) the data:

  • There can only be 1 analysis at a time in Importing status per organization.
  • There can only be 1 analysis at a time in Analyzing status per organization
tip

You can reset these limits:

  • For importing data: by deleting an analysis with the Importing... status in the main SLI Analyzer tab.
  • For analyzing data: by deleting an analysis with the Analyzing... status in the main SLI Analyzer tab.
caution

Replay and SLI Analyzer share the same mechanism for fetching historical data.

Effectively, if you run a Replay process, imports for SLI analyses might be delayed until that Replay process is finished (and vice versa).

Using SLI Analyzer​

To create an analysis, you must complete two steps:

  1. Import the SLI data.
  2. Adjust settings and create an analysis.

Import the SLI Data​

To create an analysis, you must first import the data for analysis. To do so, follow these steps:

  1. Go to the SLI Analyzer tab in Nobl9 UI.

  2. Click the Create Analysis button.

Then, the SLI Analysis creation window is displayed:

create sli analysis
Image 1: Create SLI Analysis - Step 1

Configure Data Import​

To import the data, configure the following settings:

  1. Enter the Analysis Name.
    You can enter a friendly name with spaces. The default name consists of the current date and time.

  2. Select the previously defined data source.

    • To create an analysis leveraging SLI Analyzer, you must be connected to one of the supported data sources.
    • Each data source has its specific configuration. You can find the Integration-specific details in the Sources section in the Nobl9 Documentation.

  3. Specify the Metric and provide a query.
    You can choose either a Threshold Metric, where a single time series is evaluated against a threshold, or a Ratio Metric, which allows you to enter two time series to compare (for example, a count of good requests and total requests).

    tip

    Make sure the points in your query are not too sparse - there should be at least one point per minute.

  4. Specify the Graph Time Window.
    The Graph Time Window is preset with the data source's maximum historical data retrieval time.

    note

    For the Graph Time Window duration, set a specific period for which you want to calculate the error budget. Nobl9 does not support rolling time windows for analysis.

    • You can set a maximum 30-day time window for the graph (unless the maximum period set for the specific data source is lower).
    • The start date cannot exceed the maximum time window.
    • We recommend setting the time window to at least 5 minutes as SLI Analyzer may not be able to calculate error budgets properly otherwise.

    tip

    If you select a short Graph Time Window, SLI Analyzer will display the results much faster. Remember that by doing so, SLI Analyzer can miss events you wanted to see in the results.

    Accordingly, if you select a long Graph Time Window, it might take longer for the chart to appear as there is more data to downsample.

  5. Select the Time Zone for your analysis.

  6. Click the Import Data button.

The process of data import can have three statuses regarding the data import:
Status NameStatus Description
Importing...Indicates that the process of importing data is in progress.
Import completeIndicates that Nobl9 has imported the historical data for the analysis, but the analysis has not yet been run.
Import failedIndicates that something went wrong with importing data. For more detailed troubleshooting information, refer to the Troubleshooting section of the documentation.
note

Depending od the data source’s API, data import can take between 2 to 45 minutes. If the process takes longer, contact support@nobl9.com.

Access the List of all Imports/Analyses​

When the data is successfully imported, you can see raw statistical data calculated for the SLI as well as the SLI chart and SLI Values Distribution chart displaying the value buckets that occurred across the defined data set. For more detailed information, refer to the Overview of the SLI Analysis section.

The following video shows an overview of all SLI analyses available in the main SLI Analyzer tab:

Video 1: Create SLI Analysis after data import
caution

You cannot adjust data source and metrics settings (data source, metric, query) after the data import has started. To change those settings, create a new analysis.

Create Analysis​

When the data has already been imported, you can create an analysis. Follow these steps to configure the settings and start an analysis using the SLI Analyzer:

  1. Open the already created analysis that is in the Import complete status.

  2. Select the Error Budget Calculation Method.
    Nobl9 supports two methods of calculating error budgets: Occurrences and Time Slices. For more detailed information, refer to the SLO Calculations guide.

    • For the Time Slices method, enter the Time Slice Allowance and Target to define the reliability target for time slice and SLO.

    • For the Occurrences method, enter the Target and Values to define the reliability target for SLO.

    note

    If you selected a Threshold Metric as a metric type when configuring the data import, you can also set target Values to compare the values gathered from the metric source. For example, if Values less than 200 is the objective for a Good experience, values below 200 are considered good.

  3. Click the Analyze button to see how your error budget performs with the selected values.

The process of analysis can have three different statuses:
Status NameStatus Description
Analyzing...Indicates that the analysis is in progress.
Analysis completeIndicates that Nobl9 has successfully run the analysis.
Analysis failedIndicates that something went wrong with the analysis. For more detailed troubleshooting information, refer to the Troubleshooting section of the documentation.

Analysis Results​

When the analysis is successfully performed, you can see its results and investigate the SLI chart, SLI Values Distribution chart, and Reliability Burn Down chart. For more detailed information, refer to the Overview of the SLI Analysis section.

analysis results
Image 2: Results of an SLI analysis

You can now adjust the data as much as you want until you understand your reliability.

Create SLO from Analysis​

Once you've determined the right target using your existing data, create an SLO using those parameters that make sense for your business. You can do so in the Nobl9 UI or through sloctl.

Create SLO in the UI​

Follow these steps to create an SLO from an analysis in the Nobl9 UI:

  1. Open the already created, successfull analysis that is in the Analysis complete status.

    caution

    You can't create an SLO from an analysis that has no data available. Create a new analysis and select a different Graph Time Window to get the analysis results and create an SLO.

  2. Click the Create SLO button.
create-slo-from-analysis
Image 3: Create SLO based on an analysis

  1. The SLO Wizard is displayed. Select the following:

    • In step 1, select a Service from the drop-down list that this SLO applies to.

    • In step 2, the Data Source and the Metric settings are prefilled with the values selected in the analysis.

    • In step 3, define a Time Window, that is, choose a Rolling or Calendar-Aligned Time Window.

    • In step 4, the Error Budget Calculation Method, Target, Values are populated from the analysis as a first Objective in the SLO.

      • You must provide unique SLO Objective's Name.

    • In step 5 of the SLO wizard, you must define the SLO Name and other SLO attributes.

      note

      Nobl9 automatically adds a link in the SLO to the SLI Analysis.

    • Click the Create SLO button.
note

Only Organization Admins or users with Project Owner or Editor permissions can create SLO from an analysis.

Create SLO through sloctl​

Follow these steps to create an SLO from an analysis through sloctl from an automatically generated YAML configuration:

  1. Open the already created, successfull analysis that is in the Analysis complete status.

  2. Click the View YAML button.

create-slo-from-analysis
Image 4: View SLO YAML configuration
note

Nobl9 generates the SLO YAML configuration with settings populated from the last analysis.

  1. Copy the YAML configuration and replace the <SLO_NAME> and <SERVICE_NAME> placeholders in the following YAML with the name of your SLO and Service.
create-slo-from-analysis
Image 5: SLO YAML configuration
  1. Then, apply that YAML file using sloctl to create the SLO that you’ve configured.

Troubleshooting​

I Can't Create an Analysis for a Data Source​

SLI Analyses are connected to a Project via the data source used in the analysis.

It means that:

  • Users with the Organization Viewer role can see all analyses in the organization.
  • However, if they don't have any Project roles assigned, they won't be able to create any analyses.

Remember that on top of project permissions, it's important that the data source is configured correctly to perform an analysis. Let's go over two examples:

  • Suppose a user has some Project roles assigned (i.e., Project Owner or Project Editor role) AND there are some data sources with the Maximum Period for Historical Data Retrieval set to ≥ 5 minutes in this project. In that case, this user will be able to create an analysis.
  • If the project that this user owns or can edit doesn't have any data sources available for historical data retrieval (i.e., the Maximum Period for Historical Data Retrieval is set to < 5 minutes), they won't be able to create any analyses.

Missing Data​

Missing data might be caused by the data source's retention policy. If you see missing data, ensure that the data source you use for the SLI analysis contains the data in the requested time range. Check your source's data retention policy.

Empty SLI Values Distribution Chart​

SLI Analyzer might display an empty SLI Values Distribution chart after an analysis has been completed. In such a case, your data is either too sparse or too homogenous, so SLI Analyzer can't create the SLI Values Distribution chart.

To display the chart, change the Graph Time Window and make sure that your data source contains the data in the selected time range.

Importing Failed​

If your SLI Analysis shows an Import failed status, check your data source logs to spot the probable cause of failure.

The Reliability Burn Down Chart is Over 100%​

When creating an SLI Analysis using ratio metric, ensure you correctly set your data count method as incremental or non-incremental. For some data sources, "incremental" is set by default, and you need to change it before you click the Import data button. You can't change the data count method afterward.

The problem with the reliability burn down chart over 100% may occur when your data is marked as incremental, but in fact, it is not. The following chart shows the reliability burn down chart for a non-incremental ratio metric that was marked as incremental:

reliability over 100%
Image 4: Reliability burn down chart displaying >100% values

What Reliability Target Should I Choose

Choose Reliability Targets and Create Meaningful SLOs

Use Cases of SLI Analyzer - Threshold and Ratio | Nobl9 Documentation

Use Case of SLI Analyzer - Post-Incident Target Adjustment | Nobl9 Documentation

Last updated on