SLI Analyzer Beta
Realizing which reliability target to choose is difficult as it requires understanding your system's performance over the past month or year. To do so, you usually need to create and adjust targets every few weeks.
With SLI Analyzer, you can retrieve your historical data, set reliability targets, and see how your system performs. The SLI Analyzer queries for a specific time series from a specified data source. It then analyzes the data and provides metrics for that particular SLI.
You can quickly adjust the targets and view updated error budgets and error budget burn down. Once you've determined the right target using your existing data, you can easily create a new SLO.
Leveraging SLI Analyzer, you can:
- Create and instantly adjust your first SLO on the Nobl9 platform for more robust monitoring of your reliability.
- Choose a meaningful reliability target by analyzing how your SLOs will behave before you even create them.
- Understand the reliability of your services without waiting and adjusting them every few days or weeks.
Scope of support for SLI Analyzer
The SLI Analyzer supports the following data sources:
- Amazon CloudWatch
- AMS Prometheus
- AppDynamics (including the
- Azure Monitor beta
- New Relic
- ServiceNow Cloud Observability
SLI Analyzer and RBAC
The following list presents the overview of permissions required to use SLI Analyzer:
Only Organization Admins, Integration Users or users with Project Owner, or Project Editor permissions can view, create, edit, and delete analyses.note
A project that is used by SLI Analyzer is inherited from the data source. This has important implications on what users can view and whether they can create and edit analyses.
Users having Organization Viewer or Responder, Project Viewer, Responder, or Integrations User permissions can preview existing analyses in the SLI Analyzer tab.note
If you're an Organization or Project Viewer, make sure you have the necessary project-level permissions to create and edit analyses for a specific data source.
Users having Organization User permissions cannot create, delete, or preview analyses in the SLI Analyzer tab.caution
Make sure the data source has the Maximum Period for Data Retrieval set to greater than or equal to 5 minutes.
Otherwise, you won't be able to run an analysis for that data source:
A quota limits the number of ongoing analyses. Those limits differ for two parts of the analysis—importing the data and analyzing (calculating) the data:
- There can only be two analyses at a time in Importing status per organization
- There can only be two analyses at a time in Analyzing status per organization
You can track the progress of all ongoing Data import processes for analyses and empty slots available in your organization using the Job status widget.
Click next to the top right corner in any tab to access the widget:
You can reset these limits:
- For importing data: delete an analysis with the
Importing...status in the main SLI Analyzer tab
- For analyzing data: delete an analysis with the
Analyzing...status in the main SLI Analyzer tab
Replay and SLI Analyzer share the same mechanism for fetching historical data.
Effectively, if you run a Replay process, imports for SLI analyses might be delayed until that Replay process is finished (and vice versa).
- Currently, historical data imports for the
bad-over-totalratio metric in Amazon CloudWatch are not supported. Although, you can build SLOs based on this metric.
Using SLI Analyzer
To create an analysis, you must complete two steps:
Import the SLI data
To create an analysis, you must first import the data for analysis. To do so, follow these steps:
Go to the SLI Analyzer tab in Nobl9 UI.
Click Create Analysis.
Then, the SLI Analysis creation window is displayed:
Configure data import
To import the data, configure the following settings:
Enter the analysis Name.
You can enter a friendly name with spaces. The default name consists of the current date and time.
Select the previously defined data source.
Specify the Metric and provide a query.
You can choose either a Threshold Metric, where a single time series is evaluated against a threshold, or a Ratio Metric, which allows you to enter two time series to compare (for example, a count of good requests and total requests).tip
Make sure the points in your query are not too sparse—there should be at least one point per minute.
Specify the Graph Time Window.
The graph time window is preset with the data source's maximum historical data retrieval time.note
For the graph time window duration, set a specific period for which you want to calculate the error budget. Nobl9 does not support rolling time windows for analysis.
- You can set a maximum 30-day time window for the graph (unless the maximum period set for the specific data source is lower).
- The start date cannot exceed the maximum time window.
- We recommend setting the time window to at least 5 minutes as SLI Analyzer may not be able to calculate error budgets properly otherwise.
If you select a short graph time window, SLI Analyzer will display the results much faster. Remember that by doing so, SLI Analyzer can miss events you wanted to see in the results.
Accordingly, if you select a long graph time window, it might take longer for the chart to appear as there is more data to downsample.
Select the Time Zone for your analysis.
Click Import Data.
|Status name||Status description|
|Indicates that the process of importing data is in progress.|
|Indicates that Nobl9 has imported the historical data for the analysis, but the analysis has not yet been run.|
|Indicates that something went wrong with importing data. For more detailed troubleshooting information, refer to the Troubleshooting section of the documentation.|
Depending on the data source’s API, data import can take 2-45 minutes. If the process takes longer, contact email@example.com.
Access the list of all imports/analyses
When the data is successfully imported, you can see raw statistical data calculated for the SLI as well as the SLI chart and SLI values distribution chart displaying the value buckets that occurred across the defined data set. For more detailed information, refer to the Overview of the SLI analysis section.
The following video shows an overview of all SLI analyses available in the main SLI Analyzer tab:
You cannot adjust data source and metrics settings (data source, metric, query) after the data import has started. To change those settings, create a new analysis.
When the data has already been imported, you can create an analysis. Follow these steps to configure the settings and start an analysis using the SLI Analyzer:
Open the already created analysis that is in the Import complete status.
Select the Error Budget Calculation Method.
Nobl9 supports two methods of calculating error budgets: occurrences and time slices. For more detailed information, refer to the SLO calculations guide.
For the Time slices method, enter the Time Slice Allowance and Target to define the reliability target for time slice and SLO.
For the Occurrences method, enter the Target and Values to define the reliability target for SLO.
If you selected a threshold metric when configuring the data import, you can also set target Values to compare the values gathered from the metric source. For example, if
Values less than 200is the objective for a
Goodexperience, values below 200 are considered good.
Click Analyze to see how your error budget performs with the selected values.
|Status name||Status description|
|Indicates that the analysis is in progress.|
|Indicates that Nobl9 has successfully run the analysis.|
|Indicates that something went wrong with the analysis. For more detailed troubleshooting information, refer to the Troubleshooting section of the documentation.|
When the analysis is successfully performed, you can see its results and investigate the SLI chart, SLI values distribution chart, and Reliability burn down chart. For more detailed information, refer to the Overview of the SLI Analysis section.
You can now adjust the data as much as you want until you understand your reliability.
Create SLO from analysis
Create SLO in the UI
Follow these steps to create an SLO from an analysis in the Nobl9 UI:
Open a successful analysis. Its status must be Analysis complete.
:::warning caution You can't create an SLO from an analysis that has no data available. Create a new analysis and select a different graph time window to get the analysis results and create an SLO. :::
Click CREATE SLO.
The SLO wizard opens. Follow these steps:
Step 1: select a Service from the drop-down list that this SLO applies to.
Step 2: the Data Source and the Metric settings are prefilled with the values selected in the analysis.
Step 3: define a Time Window, that is, choose a Rolling or Calendar-Aligned Time Window.
Step 4: the Error Budget Calculation Method, Target, Values are populated from the analysis as a first objective in the SLO.
- You must provide unique SLO objective's Name.
Step 5: define the SLO Name and other SLO attributes.note
Nobl9 automatically adds a link in the SLO to the SLI Analysis.
- Click Create SLO
Only Organization Admins or users with Project Owner or Editor permissions can create SLO from an analysis.
Create SLO through
Follow these steps to create an SLO from an analysis through
sloctl from an automatically generated YAML configuration:
Open a successful analysis. Its status must be Analysis complete.
Click VIEW YAML.
Nobl9 generates the SLO YAML configuration with settings populated from the last analysis.
- Copy the YAML configuration and replace the
<SERVICE_NAME>placeholders in the following YAML with the name of your SLO and Service.
- Apply this YAML file using
sloctlto create the SLO you configured.
I can't create an analysis for a data sourceSLI analyses are connected to a project via the data source used in the analysis.
It means that:
- Users with the Organization Viewer role can see all analyses in the organization.
- However, if they don't have any Project roles assigned, they won't be able to create any analyses.
Remember that on top of project permissions, it's important that the data source is configured correctly to perform an analysis. Let's go over two examples:
- Suppose a user has some Project roles assigned (i.e., Project Owner or Project Editor role) AND there are some data sources with the Maximum Period for Historical Data Retrieval set to
≥ 5 minutesin this project. In that case, this user will be able to create an analysis.
- If the project that this user owns or can edit doesn't have any data sources available for historical data retrieval (i.e., the Maximum Period for Historical Data Retrieval is set to
< 5 minutes), they won't be able to create any analyses.
Missing data might be caused by the data source's retention policy. If you see missing data, ensure that the data source you use for the SLI analysis contains the data in the requested time range. Check your source's data retention policy.
Empty SLI values distribution chart
SLI Analyzer might display an empty SLI values distribution chart after an analysis has been completed. In such a case, your data is either too sparse or too homogenous, so SLI Analyzer can't create the SLI values distribution chart.
To display the chart, change the graph time window and make sure that your data source contains the data in the selected time range.
If your SLI Analysis shows the Import failed status, check your data source logs to spot the probable cause of failure.
The Reliability burn down chart is over 100%
When creating an SLI Analysis using ratio metric, ensure you correctly set your data count method as incremental or non-incremental. For some data sources, "incremental" is set by default, and you need to change it before you click the Import data button. You can't change the data count method afterward.
The problem with the reliability burn down chart over 100% may occur when your data is marked as incremental, but in fact, it is not. The following chart shows the reliability burn down chart for a non-incremental ratio metric that was marked as incremental: