SLI Analyzer

Beta

Reading time: 0 minute(s) (0 words)

Realizing which reliability target to choose is difficult as it requires understanding your system's performance over the past month or year. To do so, you usually need to create and adjust targets every few weeks.

With SLI Analyzer, you can retrieve your historical data, set reliability targets, and see how your system performs. The SLI Analyzer queries for a specific time series from a specified data source. It then analyzes the data and provides metrics for that particular SLI.

You can quickly adjust the targets and view updated error budgets and error budget burn down. Once you've determined the right target using your existing data, you can easily create a new SLO.

Leveraging SLI Analyzer, you can:

• Create and instantly adjust your first SLO on the Nobl9 platform for more robust monitoring of your reliability.
• Choose a meaningful reliability target by analyzing how your SLOs will behave before you even create them.
• Understand the reliability of your services without waiting and adjusting them every few days or weeks.

Scope of support for SLI Analyzer

The SLI Analyzer supports the following data sources:

• Amazon CloudWatch
• AMS Prometheus
• AppDynamics (including the bad-over-total ratio metric)
• Azure Monitor
  beta
• Datadog
• Dynatrace
• Graphite
• New Relic
• Prometheus
• ServiceNow Cloud Observability
• Splunk

To use SLI Analyzer for specific data sources, you may need to update the version of the Nobl9 agent you use. Check the table to determine the minimum agent version required to use this feature.

note

You can use the beta version of SLI Analyzer only for new SLOs based on the historical data that you can fetch from the supported data sources.

Once you've determined the right target using your existing data, you can create an SLO using those parameters.

SLI Analyzer and RBAC

The following list presents the overview of permissions required to use SLI Analyzer:

• Only Organization Admins, Integration Users or users with Project Owner, or Project Editor permissions can view, create, edit, and delete analyses.

  note

  A project that is used by SLI Analyzer is inherited from the data source. This has important implications on what users can view and whether they can create and edit analyses.

• Users having Organization Viewer or Responder, Project Viewer, Responder, or Integrations User permissions can preview existing analyses in the SLI Analyzer tab.

  note

  If you're an Organization or Project Viewer, make sure you have the necessary project-level permissions to create and edit analyses for a specific data source.

• Users having Organization User permissions cannot create, delete, or preview analyses in the SLI Analyzer tab.

  caution

  Make sure the data source has the Maximum Period for Data Retrieval set to greater than or equal to 5 minutes.

  Otherwise, you won't be able to run an analysis for that data source:

  

Limitations

A quota limits the number of ongoing analyses. Those limits differ for two parts of the analysis—importing the data and analyzing (calculating) the data:

• There can only be two analyses at a time in Importing status per organization
• There can only be two analyses at a time in Analyzing status per organization

You can track the progress of all ongoing Data import processes for analyses and empty slots available in your organization using the Job status widget.

Click next to the top right corner in any tab to access the widget:

Image 1: Job status widget

tip

You can reset these limits:

• For importing data: delete an analysis with the Importing... status in the main SLI Analyzer tab
• For analyzing data: delete an analysis with the Analyzing... status in the main SLI Analyzer tab

caution

Replay and SLI Analyzer share the same mechanism for fetching historical data.

Effectively, if you run a Replay process, imports for SLI analyses might be delayed until that Replay process is finished (and vice versa).

• Currently, historical data imports for the bad-over-total ratio metric in Amazon CloudWatch are not supported. Although, you can build SLOs based on this metric.

Using SLI Analyzer

To create an analysis, you must complete two steps:

1. Import the SLI data.
2. Adjust settings and create an analysis.

Import the SLI data

To create an analysis, you must first import the data for analysis. To do so, follow these steps:

1. Go to the SLI Analyzer tab in Nobl9 UI.

2. Click Create Analysis.

Then, the SLI Analysis creation window is displayed:

Image 2: Create SLI Analysis - Step 1

Configure data import

To import the data, configure the following settings:

1. Enter the analysis Name.
   You can enter a friendly name with spaces. The default name consists of the current date and time.

2. Select the previously defined data source.

   • To create an analysis leveraging SLI Analyzer, you must be connected to one of the supported data sources.
   • Each data source has its specific configuration. You can find the integration-specific details in the Data sources section in the Nobl9 Documentation.

3. Specify the Metric and provide a query.
   You can choose either a Threshold Metric, where a single time series is evaluated against a threshold, or a Ratio Metric, which allows you to enter two time series to compare (for example, a count of good requests and total requests).

   tip

   Make sure the points in your query are not too sparse—there should be at least one point per minute.

4. Specify the Graph Time Window.
   The graph time window is preset with the data source's maximum historical data retrieval time.

   note

   For the graph time window duration, set a specific period for which you want to calculate the error budget. Nobl9 does not support rolling time windows for analysis.

   • You can set a maximum 30-day time window for the graph (unless the maximum period set for the specific data source is lower).
   • The start date cannot exceed the maximum time window.
   • We recommend setting the time window to at least 5 minutes as SLI Analyzer may not be able to calculate error budgets properly otherwise.
     
     
   tip

   If you select a short graph time window, SLI Analyzer will display the results much faster. Remember that by doing so, SLI Analyzer can miss events you wanted to see in the results.

   Accordingly, if you select a long graph time window, it might take longer for the chart to appear as there is more data to downsample.

5. Select the Time Zone for your analysis.

6. Click Import Data.

The process of data import can have three statuses regarding the data import:

Status name	Status description
Importing...	Indicates that the process of importing data is in progress.
Import complete	Indicates that Nobl9 has imported the historical data for the analysis, but the analysis has not yet been run.
Import failed	Indicates that something went wrong with importing data. For more detailed troubleshooting information, refer to the Troubleshooting section of the documentation.

note

Depending on the data source’s API, data import can take 2-45 minutes. If the process takes longer, contact support@nobl9.com.

Access the list of all imports/analyses

When the data is successfully imported, you can see raw statistical data calculated for the SLI as well as the SLI chart and SLI values distribution chart displaying the value buckets that occurred across the defined data set. For more detailed information, refer to the Overview of the SLI analysis section.

The following video shows an overview of all SLI analyses available in the main SLI Analyzer tab:

Video 1: Create SLI analysis after data import

caution

You cannot adjust data source and metrics settings (data source, metric, query) after the data import has started. To change those settings, create a new analysis.

Create analysis

When the data has already been imported, you can create an analysis. Follow these steps to configure the settings and start an analysis using the SLI Analyzer:

1. Open the already created analysis that is in the Import complete status.

2. Select the Error Budget Calculation Method.
   Nobl9 supports two methods of calculating error budgets: occurrences and time slices. For more detailed information, refer to the SLO calculations guide.

   • For the Time slices method, enter the Time Slice Allowance and Target to define the reliability target for time slice and SLO.

   • For the Occurrences method, enter the Target and Values to define the reliability target for SLO.

   note

   If you selected a threshold metric when configuring the data import, you can also set target Values to compare the values gathered from the metric source. For example, if Values less than 200 is the objective for a Good experience, values below 200 are considered good.

3. Click Analyze to see how your error budget performs with the selected values.

The process of analysis can have three different statuses:

Status name	Status description
Analyzing...	Indicates that the analysis is in progress.
Analysis complete	Indicates that Nobl9 has successfully run the analysis.
Analysis failed	Indicates that something went wrong with the analysis. For more detailed troubleshooting information, refer to the Troubleshooting section of the documentation.

Analysis results

When the analysis is successfully performed, you can see its results and investigate the SLI chart, SLI values distribution chart, and Reliability burn down chart. For more detailed information, refer to the Overview of the SLI Analysis section.

Image 3: Results of an SLI analysis

You can now adjust the data as much as you want until you understand your reliability.

Create SLO from analysis

Once you've determined the right target using your existing data, create an SLO using those parameters that make sense for your business. You can do so in the Nobl9 UI or through sloctl.

Create SLO in the UI

Follow these steps to create an SLO from an analysis in the Nobl9 UI:

1. Open a successful analysis. Its status must be Analysis complete.

   :::warning caution You can't create an SLO from an analysis that has no data available. Create a new analysis and select a different graph time window to get the analysis results and create an SLO. :::

2. Click CREATE SLO.

Image 4: Create SLO based on an analysis

3. The SLO wizard opens. Follow these steps:

   • Step 1: select a Service from the drop-down list that this SLO applies to.

   • Step 2: the Data Source and the Metric settings are prefilled with the values selected in the analysis.

   • Step 3: define a Time Window, that is, choose a Rolling or Calendar-Aligned Time Window.

   • Step 4: the Error Budget Calculation Method, Target, Values are populated from the analysis as a first objective in the SLO.

     • You must provide unique SLO objective's Name.

   • Step 5: define the SLO Name and other SLO attributes.
     

     note

     Nobl9 automatically adds a link in the SLO to the SLI Analysis.

   • Click Create SLO.

note

Only Organization Admins or users with Project Owner or Editor permissions can create SLO from an analysis.

Create SLO through sloctl

Follow these steps to create an SLO from an analysis through sloctl from an automatically generated YAML configuration:

1. Open a successful analysis. Its status must be Analysis complete.

2. Click VIEW YAML.

Image 5: View SLO YAML configuration

note

Nobl9 generates the SLO YAML configuration with settings populated from the last analysis.

3. Copy the YAML configuration and replace the <SLO_NAME> and <SERVICE_NAME> placeholders in the following YAML with the name of your SLO and Service.

Image 6: SLO YAML configuration

4. Apply this YAML file using sloctl to create the SLO you configured.

Troubleshooting

I can't create an analysis for a data source

SLI analyses are connected to a project via the data source used in the analysis.

It means that:

• Users with the Organization Viewer role can see all analyses in the organization.
• However, if they don't have any Project roles assigned, they won't be able to create any analyses.

Remember that on top of project permissions, it's important that the data source is configured correctly to perform an analysis. Let's go over two examples:

• Suppose a user has some Project roles assigned (i.e., Project Owner or Project Editor role) AND there are some data sources with the Maximum Period for Historical Data Retrieval set to ≥ 5 minutes in this project. In that case, this user will be able to create an analysis.
• If the project that this user owns or can edit doesn't have any data sources available for historical data retrieval (i.e., the Maximum Period for Historical Data Retrieval is set to < 5 minutes), they won't be able to create any analyses.

Missing data

Missing data might be caused by the data source's retention policy. If you see missing data, ensure that the data source you use for the SLI analysis contains the data in the requested time range. Check your source's data retention policy.

Empty SLI values distribution chart

SLI Analyzer might display an empty SLI values distribution chart after an analysis has been completed. In such a case, your data is either too sparse or too homogenous, so SLI Analyzer can't create the SLI values distribution chart.

To display the chart, change the graph time window and make sure that your data source contains the data in the selected time range.

Importing failed

If your SLI Analysis shows the Import failed status, check your data source logs to spot the probable cause of failure.

The Reliability burn down chart is over 100%

When creating an SLI Analysis using ratio metric, ensure you correctly set your data count method as incremental or non-incremental. For some data sources, "incremental" is set by default, and you need to change it before you click the Import data button. You can't change the data count method afterward.

The problem with the reliability burn down chart over 100% may occur when your data is marked as incremental, but in fact, it is not. The following chart shows the reliability burn down chart for a non-incremental ratio metric that was marked as incremental:

Image 7: Reliability burn down chart displaying >100% values

Useful links

What reliability target should I choose?

Choose reliability targets and create meaningful SLOs

Use cases of SLI Analyzer - threshold and ratio

Use case of SLI Analyzer - post-incident target adjustment