LogicMonitor
LogicMonitor is a SaaS-based monitoring platform that automates IT system monitoring. It provides a centralized view of various IT components, including networks, cloud environments, servers, and applications. LogicMonitor offers automated monitoring and data insights.
Nobl9 integrates with LogicMonitor,
enabling you
to create threshold and ratio SLOs for tracking the reliability of your LogicMonitor devices and websites.
To do this, select the required query typeโdevice_metrics
or website_metricsโin Nobl9 and provide the necessary parameters for your LogicMonitor device or website.
LogicMonitor parameters and supported features in Nobl9
- General support:
- Release channel: Beta
- Connection method: Agent, Direct
- Replay and SLI Analyzer: Historical data limit 30 days
- Event logs: Supported
- Query checker: Not supported
- Query parameters retrieval: Supported
- Timestamp cache persistence: Supported
- Query parameters:
- Query interval: 1 min
- Query delay: 2 min
- Jitter: 15 sec
- Timeout: 30 sec
- Agent details and minimum required versions for supported features:
- Plugin name: n9logic_monitor
- Query delay environment variable: LOGIC_MONITOR_QUERY_DELAY
- Replay and SLI Analyzer: 0.81.0-beta
- Query parameters retrieval: 0.76.0-beta
- Timestamp cache persistence: 0.76.0-beta
- Additional notes:
- Create SLOs to track your LogicMonitor devices and websites
- Bad over total ratio metrics
Creating SLOs with LogicMonitorโ
You can create SLOs using the device or website LogicMonitor metrics.
Nobl9 Webโ
- Navigate to Service Level Objectives.
- Click
.
- Select a Service.
It will be the location for your SLO in Nobl9. - Select your LogicMonitor data source.
- Modify Period for Historical Data Retrieval, if necessary.
- This value defines how far back in the past your data will be retrieved when replaying your SLO based on LogicMonitor.
- A longer period can extend the data loading time for your SLO.
- Must be a positive whole number up to the maximum period value you've set when adding the LogicMonitor data source.
Non-editable Replay periodNon-editable Replay period indicates that the maximum period for historical data retrieval set for your LogicMonitor data source is set to zero.
Adjust the data source settings to create the SLO with Replay. - Metric refers to the way of calculating and interpreting calculate and interpret data from your data source.
- Threshold metric is defined by a single numerical value (the threshold) that separates satisfactory performance from unsatisfactory performance. It's represented by a single time series evaluated against the threshold.
- Ratio metric expresses the performance as a fraction or proportion, typically by dividing the number of successful or unsuccessful events by the total number of potential events (successes + failures). It's represented by two-time series for comparison for good or bad events and total events.
For ratio metrics, select the Data count method.SLI values for good and totalWhen choosing the query for the ratio SLI (countMetrics), keep in mind that the values โโresulting from that query for both good and total:- Must be positive.
- While we recommend using integers, fractions are also acceptable.
- If using fractions, we recommend them to be larger than
1e-4=0.0001. - Shouldn't be larger than
1e+20.
- A LogicMonitor query must follow the below patters:
- Device metrics
- Website metrics
- Query Type: device_metrics
- Device Data Source Instance ID: the identifier of a monitoring rule configuration or occurrence applied to your LogicMonitor device.
- Graph ID: a 5-symbol graph configuration.
- Line: the Datapoint value.
Enter the Line value in uppercase.
Find out how to retrieve these values in LogicMonitor.
- Query type: website_metrics
- Website ID: the numeric identifier of a website you want to monitor
- Checkpoint ID: the identifier of a checkpoint associated with your required website
- Graph Name: the name of your required graph.
The available values: status, performance (only for individual checkpoints), responsetime. - Line: the label of your required graph's line.
Enter the Line value in uppercase.
Find out how to retrieve these values in LogicMonitor.
Tips on retrieving SLI data in LogicMonitor provide instructions on retrieving the required values.
- Define the Time window for your SLO:
- Rolling time windows constantly move forward as time passes. This type can help track the most recent events.
- Calendar-aligned time windows are usable for SLOs intended to map to business metrics measured on a calendar-aligned basis.
- Configure the Error budget calculation method and Objectives:
- Occurrences method counts good attempts against the count of total attempts.
- Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
- You can define up to 12 objectives for an SLO.
Similar threshold values for objectivesTo use similar threshold values for different objectives in your SLO, we recommend differentiating them by setting varying decimal points for each objective.
For example, if you want to use threshold value1for two objectives, set it to1.0000001for the first objective and to1.0000002for the second one. - Add the Display name, Name, and other settings for your SLO:
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
Use only lowercase letters, numbers, and dashes. - Select No data anomaly alert to receive notifications when your SLO stops reporting data for a specified period:
- Choose up to five supported Alert methods.
- Specify the delay period before Nobl9 sends an alert about the missing data.
From 5 minutes to 31 days. Default: 15 minutes
- Add alert policies, labels, and links, if required.
Limits per SLO: 20 alert policies or links, 30 labels.
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
- Click CREATE SLO
YAMLโ
To create an SLO based on LogicMonitor, pass the required configuration with the sloctl apply -f command.
Your configuration must follow the pattern either for the threshold (raw) or ratio (count) metrics.
Refer to the YAML guide > SLO for more information about the fields.
Nobl9 agent 0.87.0-beta and later require quotation marks around these fields:
-
LogicMonitor
deviceSLOs:deviceDataSourceInstanceIdgraphId
-
LogicMonitor
websiteSLOs:websiteIdcheckpointIdgraphName
Omitting quotation marks in these fields with Nobl9 agent 0.87.0-beta or later results in an error. Earlier agent versions do not require quotation marks.
Review and correct your LogicMonitor device and website SLOs as needed.
Device metricsโ
- Threshold (rawMetric) device metrics
- Ratio good over total (countMetric) device metrics
- Ratio bad over total (countMetric) device metrics
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example LogicMonitor SLO
indicator:
metricSource:
name: logic-monitor
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 200
name: ok
target: 0.95
rawMetric:
query:
logicMonitor:
queryType: device_metrics
line: CONNECTIONSESTABLISHED
deviceDataSourceInstanceId: 933147615
graphId: 11436
op: lte
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: '2022-12-01 00:00:00'
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example LogicMonitor SLO
indicator:
metricSource:
name: logic-monitor
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1
name: ok
target: 0.95
countMetrics:
incremental: true
good:
logicMonitor:
queryType: device_metrics
line: CONNECTIONSUCCESSES
deviceDataSourceInstanceId: 933147615
graphId: 11438
total:
logicMonitor:
queryType: device_metrics
line: CONNECTIONSESTABLISHED
deviceDataSourceInstanceId: 933147615
graphId: 11436
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: '2022-12-01 00:00:00'
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example LogicMonitor SLO
indicator:
metricSource:
name: logic-monitor
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1
name: ok
target: 0.95
countMetrics:
incremental: true
bad:
logicMonitor:
queryType: device_metrics
line: CONNECTIONFAILURES
deviceDataSourceInstanceId: 933147615
graphId: 11437
total:
logicMonitor:
queryType: device_metrics
line: CONNECTIONSESTABLISHED
deviceDataSourceInstanceId: 933147615
graphId: 11436
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: '2022-12-01 00:00:00'
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
Click to open field reference
| Field | Type | Description |
|---|---|---|
apiVersion mandatory | string | API version. Use n9/v1alpha |
kind mandatory | string | The resource type. Use SLO |
| Metadata | ||
metadata.name mandatory | string | Name identifier for the SLO. Use only lowercase alphanumeric characters |
metadata.displayName | string | User-friendly SLO name |
metadata.project mandatory | string | The name identifier of the project where you need to host your SLO |
metadata.labels | object (map: string[]) | Grouping labels for filtering or viewing |
metadata.annotations | object (map: string) | Flat string annotations |
| Spec | ||
spec.description | string | SLO description |
spec.indicator.metricSource.name mandatory | string | Data source name |
spec.indicator.metricSource.project mandatory | string | Project containing the data source |
spec.indicator.metricSource.kind mandatory | string | Data source connection method. Can be Agent or Direct |
spec.budgetingMethod mandatory | enum | Error budget calculation method. Can be Occurrences or Time slices |
spec.objectives mandatory | array | Your SLO objective definition, up to 12 objectives per SLO. |
spec.objectives[].displayName | string | User-friendly objective name |
spec.objectives[].value mandatory | number | Data point values that is considered "good" (e.g., 200.0).In SLOs with two or more objectives, keep each objective's value unique. In ratio ( count) metrics, value is retained for legacy purposes. |
spec.objectives[].name mandatory | string | Name identifier for this objective |
spec.objectives[].op mandatory | string (enum) | Operator for objective. One of:lte (less than or equal to)lt (less than)gte (greater than or equal to)gt (greater than) |
spec.objectives[].target mandatory | float | The percentage of the good minutes or occurrences that must meet the desired performance (e.g., is the target is 0.95, the good performance is expected to be observed in at least 95% of the time window) |
spec.objectives[].rawMetric/.countMetric mandatory | object | The metric type indicator. Set:rawMetric for a threshold metriccountMetric for a ratio metric.A ratio metric requires the additional fields: countMetric.incremental (boolean) the data count methodcountMetric.good/.bad and countMetric.total a numerator and denominator queries |
spec.objectives[].countMetric.incremental mandatory | boolean | The data count method for a ratio (countMetric) metric type |
spec.objectives[].primary | boolean | The indicator of a primary SLO objective |
spec.service mandatory | string | The name identifier of a service to host this SLO. The service must exist in the project specified in metadata.project |
spec.timeWindows mandatory | array | Defines SLO time window for error budget calculation. Set: isRolling: true for the rolling time window typeisRolling: false for the calendar-aligned type |
spec.timeWindows.unit mandatory | integer | The time window units. One of:Day | Hour | Minute for the rolling time windowYear | Quarter | Month | Week | Day for the calendar-aligned time window |
spec.timeWindows.count mandatory | integer | The number of units in a time window |
spec.timeWindows.startTime | string | Mandatory for calendar-aligned time windows. Date and time in the format YYYY-MM-DDTHH:mm:ss |
spec.timeWindows.timeZone | string | Mandatory for calendar-aligned time-windows. A valid IANA Time Zone Database name |
spec.timeWindows.isRolling mandatory | boolean | true for the rolling time window typefalse for the calendar-aligned type |
spec.alertPolicies | array | The name identifiers of alert policies to be linked to this SLO (must be from the same project as the SLO). Up to 20 alert policies per SLO. |
spec.attachments | array | Links to any additional attributes of this SLO |
spec.anomalyConfig | object | Settings for a manual no data anomaly detection rule |
spec.noData.alertMethods | array | List of alert methods for no-data anomaly. Up to five alert methods per SLO. Every alert method must have the name and project fields |
spec.noData.alertAfter | string | Waiting time before sending a no-data notification. Must be 5m to 31d.Default: 15m |
| Source-specific fields | ||
logicmonitor.queryTypemandatory | string | The type of the query. One of: device_metrics | website_metrics |
logicmonitor.linemandatory | string | The datapoint of your required graph. The value must be uppercase |
logicmonitor.deviceDataSourceInstanceIdmandatory | string | The identifier of a monitoring rule configuration or occurrence applied to a LogicMonitor device to be monitored |
logicmonitor.graphIdmandatory | string | The graph configuration associated with your required device and defines how monitoring data collected by this device is visualized and presented in LogicMonitor |
Website metricsโ
- Threshold (rawMetric) website metrics
- Ratio good over total (countMetric) website metrics
- Ratio bad over total (countMetric) website metrics
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example LogicMonitor SLO
indicator:
metricSource:
name: logic-monitor
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 200.0
name: ok
target: 0.95
rawMetric:
query:
logicMonitor:
queryType: website_metrics
line: MIN RTT
websiteId: "1"
checkpointId: "1044712023"
graphName: responseTime
op: lte
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: "2022-12-01 00:00:00"
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example LogicMonitor SLO
indicator:
metricSource:
name: logic-monitor
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1
name: ok
target: 0.95
countMetrics:
incremental: true
good:
logicMonitor:
queryType: website_metrics
checkpointId: "7615"
graphName: "responseTime"
websiteId: "123213"
line: CONNECTIONFAILURES
total:
logicMonitor:
queryType: website_metrics
checkpointId: "7615"
graphName: "responseTime"
websiteId: "123213"
line: CONNECTIONFAILURES
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: 2022-12-01T00:00:00.000Z
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
countMetrics), keep in mind that the values โโresulting from that query for both good and total:- Must be positive.
- While we recommend using integers, fractions are also acceptable.
- If using fractions, we recommend them to be larger than
1e-4=0.0001. - Shouldn't be larger than
1e+20.
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example LogicMonitor SLO
indicator:
metricSource:
name: logic-monitor
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1
name: ok
target: 0.95
countMetrics:
incremental: true
bad:
logicMonitor:
queryType: website_metrics
checkpointId: "7615"
graphName: "responseTime"
websiteId: "123213"
line: CONNECTIONFAILURES
total:
logicMonitor:
queryType: website_metrics
checkpointId: "7615"
graphName: "responseTime"
websiteId: "123213"
line: CONNECTIONFAILURES
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: 2022-12-01T00:00:00.000Z
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
countMetrics), keep in mind that the values โโresulting from that query for both good and total:- Must be positive.
- While we recommend using integers, fractions are also acceptable.
- If using fractions, we recommend them to be larger than
1e-4=0.0001. - Shouldn't be larger than
1e+20.
Click to open field reference
| Field | Type | Description |
|---|---|---|
apiVersion mandatory | string | API version. Use n9/v1alpha |
kind mandatory | string | The resource type. Use SLO |
| Metadata | ||
metadata.name mandatory | string | Name identifier for the SLO. Use only lowercase alphanumeric characters |
metadata.displayName | string | User-friendly SLO name |
metadata.project mandatory | string | The name identifier of the project where you need to host your SLO |
metadata.labels | object (map: string[]) | Grouping labels for filtering or viewing |
metadata.annotations | object (map: string) | Flat string annotations |
| Spec | ||
spec.description | string | SLO description |
spec.indicator.metricSource.name mandatory | string | Data source name |
spec.indicator.metricSource.project mandatory | string | Project containing the data source |
spec.indicator.metricSource.kind mandatory | string | Data source connection method. Can be Agent or Direct |
spec.budgetingMethod mandatory | enum | Error budget calculation method. Can be Occurrences or Time slices |
spec.objectives mandatory | array | Your SLO objective definition, up to 12 objectives per SLO. |
spec.objectives[].displayName | string | User-friendly objective name |
spec.objectives[].value mandatory | number | Data point values that is considered "good" (e.g., 200.0).In SLOs with two or more objectives, keep each objective's value unique. In ratio ( count) metrics, value is retained for legacy purposes. |
spec.objectives[].name mandatory | string | Name identifier for this objective |
spec.objectives[].op mandatory | string (enum) | Operator for objective. One of:lte (less than or equal to)lt (less than)gte (greater than or equal to)gt (greater than) |
spec.objectives[].target mandatory | float | The percentage of the good minutes or occurrences that must meet the desired performance (e.g., is the target is 0.95, the good performance is expected to be observed in at least 95% of the time window) |
spec.objectives[].rawMetric/.countMetric mandatory | object | The metric type indicator. Set:rawMetric for a threshold metriccountMetric for a ratio metric.A ratio metric requires the additional fields: countMetric.incremental (boolean) the data count methodcountMetric.good/.bad and countMetric.total a numerator and denominator queries |
spec.objectives[].countMetric.incremental mandatory | boolean | The data count method for a ratio (countMetric) metric type |
spec.objectives[].primary | boolean | The indicator of a primary SLO objective |
spec.service mandatory | string | The name identifier of a service to host this SLO. The service must exist in the project specified in metadata.project |
spec.timeWindows mandatory | array | Defines SLO time window for error budget calculation. Set: isRolling: true for the rolling time window typeisRolling: false for the calendar-aligned type |
spec.timeWindows.unit mandatory | integer | The time window units. One of:Day | Hour | Minute for the rolling time windowYear | Quarter | Month | Week | Day for the calendar-aligned time window |
spec.timeWindows.count mandatory | integer | The number of units in a time window |
spec.timeWindows.startTime | string | Mandatory for calendar-aligned time windows. Date and time in the format YYYY-MM-DDTHH:mm:ss |
spec.timeWindows.timeZone | string | Mandatory for calendar-aligned time-windows. A valid IANA Time Zone Database name |
spec.timeWindows.isRolling mandatory | boolean | true for the rolling time window typefalse for the calendar-aligned type |
spec.alertPolicies | array | The name identifiers of alert policies to be linked to this SLO (must be from the same project as the SLO). Up to 20 alert policies per SLO. |
spec.attachments | array | Links to any additional attributes of this SLO |
spec.anomalyConfig | object | Settings for a manual no data anomaly detection rule |
spec.noData.alertMethods | array | List of alert methods for no-data anomaly. Up to five alert methods per SLO. Every alert method must have the name and project fields |
spec.noData.alertAfter | string | Waiting time before sending a no-data notification. Must be 5m to 31d.Default: 15m |
| Source-specific fields | ||
logicmonitor.queryTypemandatory | string | The type of the query. One of: website_metrics | device_metrics |
logicmonitor.linemandatory | string | The label of your required graph line. The value must be uppercase |
logicmonitor.websiteIdmandatory | string | The identifier of the monitored website. This value uniquely identifies the website you track within LogicMonitor. Put the value in quotation marks |
logicmonitor.checkpointIdmandatory | string | The identifier of a checkpoint associated with a website. A checkpoint is a specific monitoring location or probe used by LogicMonitor to test website availability and performance. Put the value in quotation marks |
logicmonitor.graphNamemandatory | string | The identifier of the specific graph to be monitored |
Find more SLO samples in the Nobl9 SDK.
Tips on retrieving SLI data in LogicMonitorโ
For correct monitoring, Nobl9 SLOs based on LogicMonitor require specific data, depending on a resource you need to monitorโa device or website. To retrieve the required values, do the following:
- I want to monitor a device
- I want to monitor a website
- Log in to your LogicMonitor dashboard.
- In the navigation sidebar, click Resource Tree.
All your monitored devices are listed here. - Select the required Resource > DataSource > instance.
Device Data Source Instance ID
The identifier of a monitoring rule configuration or occurrence applied to a LogicMonitor device you need to monitor.
GET /device/devices/{deviceId}/devicedatasources/{hdsId}/instancesdeviceDataSourceInstanceIdGraph ID
Graph configuration associated with your required device and defines how monitoring data collected by this device is visualized and presented in LogicMonitor
XXXXX in ...exchangeDataSourceGraphs-XXXXXGET /device/devices/{deviceId}/devicedatasources/{id}graphIdLine
A datapoint of your required graph
line.label valueGET /device/devicedatasourceinstances/{instanceId}/graphs/{graphId}/dataline- Log in to your LogicMonitor dashboard.
- In the navigation sidebar, click ellipses (More) > Websites.
- Select your required website in the navigation tree.
Website ID
The identifier of the monitored website. This value uniquely identifies the website you track within LogicMonitor
XXX in .../treeNodes#websiteGroups-5*,websites-XXXGET /website/websitesitems.id valuewebsiteIdGraph Name
The identifier of the specific graph to be monitored
status, performance (for some graphs), responsetimegraphNameCheckpoint ID
The identifier of a checkpoint associated with a website. A checkpoint is a specific monitoring location or probe used by LogicMonitor to test website availability and performance
GET website/websites/{websiteID}checkpoints.id valuecheckpointIdLine
The label of your required graph line
GET /website/websites/{websiteID}/checkpoints/{checkpointID}/graphs/{graphName}/dataline.label valuelineLogicMonitor API rate limitsโ
LogicMonitor limits GET requests to 500 per minute.
Read more about rate limiting in LogicMonitor.
You can optimize the rate limit usage, which is particularly beneficial when you have hundreds of SLIs. For this, keep an individual data source in Nobl9 per LogicMonitor instance, ensuring a single rate limit.
It makes possible batching queries by graphId and instanceId and
reading time series from responses per graphId, instanceId, and line (an SLI identifier).
As a result, Nobl9 can query for unique lines only once, even if graphId and instanceId include multiple lines.
The maximum number of lines graphId and instanceId can return depends on the way of metric organization.