Sumo Logic
Sumo Logic is an observability platform that provides visibility into AWS, Azure, and GCP cloud applications and infrastructure.
Sumo Logic parameters and supported features in Nobl9
- General support:
- Release channel: Stable, Beta
- Connection method: Agent, Direct
- Replay and SLI Analyzer: Historical data limit 30 days
- Event logs: Supported
- Query checker: Not supported
- Query parameters retrieval: Supported
- Timestamp cache persistence: Supported
- Query parameters:
- Query interval: 2 min
- Query delay: 4 min
- Jitter: 30 sec
- Timeout: 30 sec
- Agent details and minimum required versions for supported features:
- Plugin name: n9sumologic
- Query delay environment variable: SUMOLOGIC_QUERY_DELAY
- Replay and SLI Analyzer: 0.102.0-beta
- Query parameters retrieval: 0.73.2
- Timestamp cache persistence: 0.65.0
- Additional notes:
- Supported authentication using <accessId>:<accessKey>
Creating SLOs with Sumo Logicβ
You can create Sumo Logic SLOs using the Metrics or Logs types.
Nobl9 Webβ
- Navigate to Service Level Objectives.
- Click
.
- Select a Service.
It will be the location for your SLO in Nobl9. - Select your Sumo Logic data source.
- Modify Period for Historical Data Retrieval, if necessary.
- This value defines how far back in the past your data will be retrieved when replaying your SLO based on Sumo Logic.
- A longer period can extend the data loading time for your SLO.
- Must be a positive whole number up to the maximum period value you've set when adding the Sumo Logic data source.
Non-editable Replay periodNon-editable Replay period indicates that the maximum period for historical data retrieval set for your Sumo Logic data source is set to zero.
Adjust the data source settings to create the SLO with Replay. - Metric refers to the way of calculating and interpreting calculate and interpret data from your data source.
- Threshold metric is defined by a single numerical value (the threshold) that separates satisfactory performance from unsatisfactory performance. It's represented by a single time series evaluated against the threshold.
- Ratio metric expresses the performance as a fraction or proportion, typically by dividing the number of successful events by the total number of potential events (successes + failures). It's represented by two-time series for comparison for good events and total events.
For ratio metrics, select the Data count method.SLI values for good and totalWhen choosing the query for the ratio SLI (countMetrics), keep in mind that the values ββresulting from that query for both good and total:- Must be positive.
- While we recommend using integers, fractions are also acceptable.
- If using fractions, we recommend them to be larger than
1e-4=0.0001. - Shouldn't be larger than
1e+20.
- Choose the query type: Metrics or Logs.
- Metrics
- Logs
- Quantization refers to aggregating metric data points for time series over an interval of time.
Must be 15s or greater. - Rollup is the aggregation function applied to the metric (none by default).
- Enter the query.
Click to open query samples
metric=CPU_usage
metric=Mem_Used
metric=Mem_Total
- Define the Time window for your SLO:
- Rolling time windows constantly move forward as time passes. This type can help track the most recent events.
- Calendar-aligned time windows are usable for SLOs intended to map to business metrics measured on a calendar-aligned basis.
- Configure the Error budget calculation method and Objectives:
- Occurrences method counts good attempts against the count of total attempts.
- Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
- You can define up to 12 objectives for an SLO.
Similar threshold values for objectivesTo use similar threshold values for different objectives in your SLO, we recommend differentiating them by setting varying decimal points for each objective.
For example, if you want to use threshold value1for two objectives, set it to1.0000001for the first objective and to1.0000002for the second one. - Add the Display name, Name, and other settings for your SLO:
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
Use only lowercase letters, numbers, and dashes. - Select No data anomaly alert to receive notifications when your SLO stops reporting data for a specified period:
- Choose up to five supported Alert methods.
- Specify the delay period before Nobl9 sends an alert about the missing data.
From 5 minutes to 31 days. Default: 15 minutes
- Add alert policies, labels, and links, if required.
Limits per SLO: 20 alert policies or links, 30 labels.
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
- Click CREATE SLO
- Specify your query. Logs queries must contain the
timeslicekeyword.
Click to open query samples
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| if (status_code matches "20*" or status_code matches "30*",1,0) as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| if (status_code matches "20*" or status_code matches "30*",1,0) as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| count(*) as n9_value by n9_time
| sort by n9_time asc
- Define the Time window for your SLO:
- Rolling time windows constantly move forward as time passes. This type can help track the most recent events.
- Calendar-aligned time windows are usable for SLOs intended to map to business metrics measured on a calendar-aligned basis.
- Configure the Error budget calculation method and Objectives:
- Occurrences method counts good attempts against the count of total attempts.
- Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
- You can define up to 12 objectives for an SLO.
Similar threshold values for objectivesTo use similar threshold values for different objectives in your SLO, we recommend differentiating them by setting varying decimal points for each objective.
For example, if you want to use threshold value1for two objectives, set it to1.0000001for the first objective and to1.0000002for the second one. - Add the Display name, Name, and other settings for your SLO:
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
Use only lowercase letters, numbers, and dashes. - Select No data anomaly alert to receive notifications when your SLO stops reporting data for a specified period:
- Choose up to five supported Alert methods.
- Specify the delay period before Nobl9 sends an alert about the missing data.
From 5 minutes to 31 days. Default: 15 minutes
- Add alert policies, labels, and links, if required.
Limits per SLO: 20 alert policies or links, 30 labels.
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
- Click CREATE SLO
YAMLβ
Metrics queryβ
- Threshold (rawMetric)
- Ratio (countMetric)
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 200.0
name: ok
target: 0.95
rawMetric:
query:
sumoLogic:
type: metrics
query: metric=CPU_Usage
quantization: 15s
rollup: Avg
op: lte
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: "2022-12-01 00:00:00"
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1.0
name: ok
target: 0.95
countMetrics:
incremental: true
good:
sumoLogic:
type: metrics
query: metric=Mem_Used
quantization: 15s
rollup: Avg
total:
sumoLogic:
type: metrics
query: metric=Mem_Total
quantization: 15s
rollup: Avg
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: "2022-12-01 00:00:00"
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
Click to open field reference
| Field | Type | Description |
|---|---|---|
apiVersion mandatory | string | API version. Use n9/v1alpha |
kind mandatory | string | The resource type. Use SLO |
| Metadata | ||
metadata.name mandatory | string | Name identifier for the SLO. Use only lowercase alphanumeric characters |
metadata.displayName | string | User-friendly SLO name |
metadata.project mandatory | string | The name identifier of the project where you need to host your SLO |
metadata.labels | object (map: string[]) | Grouping labels for filtering or viewing |
metadata.annotations | object (map: string) | Flat string annotations |
| Spec | ||
spec.description | string | SLO description |
spec.indicator.metricSource.name mandatory | string | Data source name |
spec.indicator.metricSource.project mandatory | string | Project containing the data source |
spec.indicator.metricSource.kind mandatory | string | Data source connection method. Can be Agent or Direct |
spec.budgetingMethod mandatory | enum | Error budget calculation method. Can be Occurrences or Time slices |
spec.objectives mandatory | array | Your SLO objective definition, up to 12 objectives per SLO. |
spec.objectives[].displayName | string | User-friendly objective name |
spec.objectives[].value mandatory | number | Data point values that is considered "good" (e.g., 200.0).In SLOs with two or more objectives, keep each objective's value unique. In ratio ( count) metrics, value is retained for legacy purposes. |
spec.objectives[].name mandatory | string | Name identifier for this objective |
spec.objectives[].op mandatory | string (enum) | Operator for objective. One of:lte (less than or equal to)lt (less than)gte (greater than or equal to)gt (greater than) |
spec.objectives[].target mandatory | float | The percentage of the good minutes or occurrences that must meet the desired performance (e.g., is the target is 0.95, the good performance is expected to be observed in at least 95% of the time window) |
spec.objectives[].rawMetric/.countMetric mandatory | object | The metric type indicator. Set:rawMetric for a threshold metriccountMetric for a ratio metric.A ratio metric requires the additional fields: countMetric.incremental (boolean) the data count methodcountMetric.good/.bad and countMetric.total a numerator and denominator queries |
spec.objectives[].countMetric.incremental mandatory | boolean | The data count method for a ratio (countMetric) metric type |
spec.objectives[].primary | boolean | The indicator of a primary SLO objective |
spec.service mandatory | string | The name identifier of a service to host this SLO. The service must exist in the project specified in metadata.project |
spec.timeWindows mandatory | array | Defines SLO time window for error budget calculation. Set: isRolling: true for the rolling time window typeisRolling: false for the calendar-aligned type |
spec.timeWindows.unit mandatory | integer | The time window units. One of:Day | Hour | Minute for the rolling time windowYear | Quarter | Month | Week | Day for the calendar-aligned time window |
spec.timeWindows.count mandatory | integer | The number of units in a time window |
spec.timeWindows.startTime | string | Mandatory for calendar-aligned time windows. Date and time in the format YYYY-MM-DDTHH:mm:ss |
spec.timeWindows.timeZone | string | Mandatory for calendar-aligned time-windows. A valid IANA Time Zone Database name |
spec.timeWindows.isRolling mandatory | boolean | true for the rolling time window typefalse for the calendar-aligned type |
spec.alertPolicies | array | The name identifiers of alert policies to be linked to this SLO (must be from the same project as the SLO). Up to 20 alert policies per SLO. |
spec.attachments | array | Links to any additional attributes of this SLO |
spec.anomalyConfig | object | Settings for a manual no data anomaly detection rule |
spec.noData.alertMethods | array | List of alert methods for no-data anomaly. Up to five alert methods per SLO. Every alert method must have the name and project fields |
spec.noData.alertAfter | string | Waiting time before sending a no-data notification. Must be 5m to 31d.Default: 15m |
| Source-specific fields | ||
sumoLogic.typemandatory | string | The query type. One of: metrics | logs. Specify metrics for your metrics SLO |
sumoLogic.quantizationmandatory | string | Metric data point aggregation for time series over an interval of time (e.g., s, h). Must be 15s or greater |
sumoLogic.rollupmandatory | string | The metric aggregation function. One of: avg | sum | min | max | count | none |
sumoLogic.querymandatory | string | Your metrics query |
Logs queryβ
- Threshold (rawMetric)
- Ratio (countMetric)
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 200
name: ok
target: 0.95
rawMetric:
query:
sumoLogic:
type: logs
query: >-
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1" * * " as (status_code, size, tail)
| if (status_code matches "20" or status_code matches "30*",1,0)
as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc
op: lte
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: '2022-12-01 00:00:00'
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1
name: ok
target: 0.95
countMetrics:
incremental: true
good:
sumoLogic:
type: logs
query: |-
_collector="app-cluster" _source="logs"
| json "log"
| timeslice 15s as n9_time
| parse "level=* *" as (log_level, tail)
| if (log_level matches "error" ,0,1) as log_level_not_error
| sum(log_level_not_error) as n9_value by n9_time
| sort by n9_time asc
total:
sumoLogic:
type: logs
query: |-
_collector="app-cluster" _source="logs"
| json "log"
| timeslice 15s as n9_time
| parse "level=* *" as (log_level, tail)
| count(*) as n9_value by n9_time
| sort by n9_time asc
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: '2022-12-01 00:00:00'
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
Click to open field reference
| Field | Type | Description |
|---|---|---|
apiVersion mandatory | string | API version. Use n9/v1alpha |
kind mandatory | string | The resource type. Use SLO |
| Metadata | ||
metadata.name mandatory | string | Name identifier for the SLO. Use only lowercase alphanumeric characters |
metadata.displayName | string | User-friendly SLO name |
metadata.project mandatory | string | The name identifier of the project where you need to host your SLO |
metadata.labels | object (map: string[]) | Grouping labels for filtering or viewing |
metadata.annotations | object (map: string) | Flat string annotations |
| Spec | ||
spec.description | string | SLO description |
spec.indicator.metricSource.name mandatory | string | Data source name |
spec.indicator.metricSource.project mandatory | string | Project containing the data source |
spec.indicator.metricSource.kind mandatory | string | Data source connection method. Can be Agent or Direct |
spec.budgetingMethod mandatory | enum | Error budget calculation method. Can be Occurrences or Time slices |
spec.objectives mandatory | array | Your SLO objective definition, up to 12 objectives per SLO. |
spec.objectives[].displayName | string | User-friendly objective name |
spec.objectives[].value mandatory | number | Data point values that is considered "good" (e.g., 200.0).In SLOs with two or more objectives, keep each objective's value unique. In ratio ( count) metrics, value is retained for legacy purposes. |
spec.objectives[].name mandatory | string | Name identifier for this objective |
spec.objectives[].op mandatory | string (enum) | Operator for objective. One of:lte (less than or equal to)lt (less than)gte (greater than or equal to)gt (greater than) |
spec.objectives[].target mandatory | float | The percentage of the good minutes or occurrences that must meet the desired performance (e.g., is the target is 0.95, the good performance is expected to be observed in at least 95% of the time window) |
spec.objectives[].rawMetric/.countMetric mandatory | object | The metric type indicator. Set:rawMetric for a threshold metriccountMetric for a ratio metric.A ratio metric requires the additional fields: countMetric.incremental (boolean) the data count methodcountMetric.good/.bad and countMetric.total a numerator and denominator queries |
spec.objectives[].countMetric.incremental mandatory | boolean | The data count method for a ratio (countMetric) metric type |
spec.objectives[].primary | boolean | The indicator of a primary SLO objective |
spec.service mandatory | string | The name identifier of a service to host this SLO. The service must exist in the project specified in metadata.project |
spec.timeWindows mandatory | array | Defines SLO time window for error budget calculation. Set: isRolling: true for the rolling time window typeisRolling: false for the calendar-aligned type |
spec.timeWindows.unit mandatory | integer | The time window units. One of:Day | Hour | Minute for the rolling time windowYear | Quarter | Month | Week | Day for the calendar-aligned time window |
spec.timeWindows.count mandatory | integer | The number of units in a time window |
spec.timeWindows.startTime | string | Mandatory for calendar-aligned time windows. Date and time in the format YYYY-MM-DDTHH:mm:ss |
spec.timeWindows.timeZone | string | Mandatory for calendar-aligned time-windows. A valid IANA Time Zone Database name |
spec.timeWindows.isRolling mandatory | boolean | true for the rolling time window typefalse for the calendar-aligned type |
spec.alertPolicies | array | The name identifiers of alert policies to be linked to this SLO (must be from the same project as the SLO). Up to 20 alert policies per SLO. |
spec.attachments | array | Links to any additional attributes of this SLO |
spec.anomalyConfig | object | Settings for a manual no data anomaly detection rule |
spec.noData.alertMethods | array | List of alert methods for no-data anomaly. Up to five alert methods per SLO. Every alert method must have the name and project fields |
spec.noData.alertAfter | string | Waiting time before sending a no-data notification. Must be 5m to 31d.Default: 15m |
| Logs query keywords | ||
timeslicemandatory | string | Groups log messages into fixed time intervals. Used to aggregate data points into consistent time buckets for time-series analysis. The result timestamp marks the beginning of each interval. Time unit value: [number][unit], where unit is one of: s (seconds) | m (minutes) | h (hours) | d (days). Must be 15s or greater |
n9timemandatory | string | The timestamp field used for time-based analytics. Represents the moment of logging a data point or the interval the data point belongs to. Must be a Unix timestamp in seconds or milliseconds |
n9valuemandatory | float | The numerical value of the metric being measured |
count(*)mandatory | string | An aggregation function applied to the grouped data. One of: avg | sum | min | max | count | none |
asmandatory | string | A search operator that creates an alias for a field or expression. Used to name the output fields for further processing. Field alias name that follows the Sumo Logic naming convention. Required fields: n9time and n9value |
Querying for logsβ
Sumo Logic queries use pipe operators (|) to chain operations together. Each operator processes the results from the previous operation, progressively filtering and transforming the data to achieve the desired output.
All queries must start with either a keyword or string search.
Special characters are available for pattern matching:
*- Wildcard character that matches zero or more characters?- Matches exactly one character
Here's a detailed example of a Sumo Logic query that calculates successful HTTP responses:
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| if (status_code matches "20*" or status_code matches "30*",1,0) as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc
This query:
- Filters logs from the nginx uploads category.
- Groups data into 1-minute intervals.
- Extracts HTTP status code, size, and remaining content.
- Marks responses with 2xx or 3xx status codes as successful (1) and others as failed (0).
- Sums successful responses for each time interval.
- Sorts results chronologically.
The query produces time-series data in the following format:
"n9_time","n9_value"
"1645371960000","2.0"
"1645372020000","58.0"
"1645372080000","46.0"
"1645372140000","12.0"
"1645372200000","12.0"
"1645372260000","12.0"
"1645372320000","14.0"
"1645372380000","22.0"
For comparison, here's a query that counts total requests in the same time ranges:
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| count(*) as n9_value by n9_time
| sort by n9_time asc
API rate limitsβ
Sumo Logic enforces rate limits on Search Job API requests.
The Nobl9 agent makes several API calls following the documented Process Flow to collect data points. To manage these requests efficiently, the agent distributes them across the 2-minute interval.
To ensure timely SLI data collection, observe these API limits:
- API requestsβ4 requests/second (240/minute) per user
- Concurrent requestsβ10 per access key
Best practices to avoid rate limitsβ
- Use metrics over logs
- Prefer metrics queries as they are ~4x more efficient than logs queries
- Consider converting logs to metrics
- Optimize log queries
- Keep query execution time under 2 minutes
- Utilize Sumo Logic partitions
- Implement scheduled views
- Keep one agent connection for your Sumo Logic data sources
- For agent-based connections, use only one agent for all Sumo Logic data sources
- This restriction doesn't apply to direct connections
- Monitor usage
- Keep the number of log-based objectives within your API limits