Sumo Logic
Sumo Logic is an observability platform that provides visibility into AWS, Azure, and GCP cloud applications and infrastructure.
Sumo Logic parameters and supported features in Nobl9
- General support:
- Release channel: Stable, Beta
- Connection method: Agent, Direct
- Replay and SLI Analyzer: Historical data limit 30 days
- Event logs: Supported
- Query checker: Not supported
- Query parameters retrieval: Supported
- Timestamp cache persistence: Supported
- Query parameters:
- Query interval: 2 min
- Query delay: 4 min
- Jitter: 30 sec
- Timeout: 30 sec
- Agent details and minimum required versions for supported features:
- Plugin name: n9sumologic
- Query delay environment variable: SUMOLOGIC_QUERY_DELAY
- Replay and SLI Analyzer: 0.102.0-beta
- Query parameters retrieval: 0.73.2
- Timestamp cache persistence: 0.65.0
- Additional notes:
- Supported authentication using <accessId>:<accessKey>
Creating SLOs with Sumo Logicβ
You can create Sumo Logic SLOs using the Metrics or Logs types.
Nobl9 Webβ
- Navigate to Service Level Objectives.
- Click
.
- Select a Service.
It will be the location for your SLO in Nobl9. - Select your Sumo Logic data source.
- Modify Period for Historical Data Retrieval, when necessary.
- This value defines how far back in the past your data will be retrieved when replaying your SLO based on Sumo Logic.
- A longer period can extend the data loading time for your SLO.
- Must be a positive whole number up to the maximum period value you've set when adding the Sumo Logic data source.
- Select the Metric type:
- Threshold metric: a single time series is evaluated against a threshold.
- Ratio metric: two-time series for comparison for good events and total events.
For ratio metrics, select the Data count method: incremental or non-incremental.
- Choose the query type: Metrics or Logs.
- Metrics
- Logs
In the Metric query type:
- Quantization refers to aggregating metric data points for time series over an interval of time.
Must be 15s or greater. - Rollup is an aggregation function for metric quantizing.
Default value isnone
- Enter the query.
Query examples:- Threshold query
metric=CPU_usage
- Ratio good query
metric=Mem_Used
- Ratio total query
metric=Mem_Total
- Threshold query
The Logs query type must contain the timeslice
keyword.
Example queries:
- Threshold query
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| if (status_code matches "20*" or status_code matches "30*",1,0) as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc - Ratio good query:
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| if (status_code matches "20*" or status_code matches "30*",1,0) as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc - Ratio total query:
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| count(*) as n9_value by n9_time
| sort by n9_time asc
- Define the Time window for your SLO:
- Rolling time windows constantly move forward as time passes. This type can help track the most recent events.
- Calendar-aligned time windows are usable for SLOs intended to map to business metrics measured on a calendar-aligned basis.
- Configure the Error budget calculation method and Objectives:
- Occurrences method counts good attempts against the count of total attempts.
- Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
- You can define up to 12 objectives for an SLO.
Similar threshold values for objectivesTo use similar threshold values for different objectives in your SLO, we recommend differentiating them by setting varying decimal points for each objective.
For example, if you want to use threshold value1
for two objectives, set it to1.0000001
for the first objective and to1.0000002
for the second one. - Add the Display name, Name, and other settings for your SLO:
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
Use only lowercase letters, numbers, and dashes. - Select No data anomaly alert to receive notifications when your SLO stops reporting data for a specified period:
- Choose up to five supported Alert methods.
- Specify the delay period before Nobl9 sends an alert about the missing data.
From 5 minutes to 31 days. Default: 15 minutes
- Add alert policies, labels, and links, if required.
Limits per SLO: 20 alert policies or links, 30 labels.
- Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
- Click CREATE SLO
countMetrics
), keep in mind that the values ββresulting from that query for both good and total:- Must be positive.
- While we recommend using integers, fractions are also acceptable.
- If using fractions, we recommend them to be larger than
1e-4
=0.0001
. - Shouldn't be larger than
1e+20
.
YAMLβ
Metrics queryβ
- Threshold (rawMetric)
- Ratio (countMetric)
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 200.0
name: ok
target: 0.95
rawMetric:
query:
sumoLogic:
type: metrics
query: metric=CPU_Usage
quantization: 15s
rollup: Avg
op: lte
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: "2022-12-01 00:00:00"
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1.0
name: ok
target: 0.95
countMetrics:
incremental: true
good:
sumoLogic:
type: metrics
query: metric=Mem_Used
quantization: 15s
rollup: Avg
total:
sumoLogic:
type: metrics
query: metric=Mem_Total
quantization: 15s
rollup: Avg
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: "2022-12-01 00:00:00"
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
Your metrics query must contain the following:
Field | Type | Description | Notes |
---|---|---|---|
type | String | One of: metrics | logs | For the metric query type, specify metrics |
quantization | Integer | Metric data point aggregation for time series over an interval of time (e.g, s , h ) | Must be 15s or greater |
rollup | String | An aggregation function used for quantizing metrics | One of: avg | sum | min | max | count | none |
query | String | Your custom query | Example: metric=CPU_usage |
Refer to the YAML guide > SLO for more information on other fields.
Logs queryβ
- Threshold (rawMetric)
- Ratio (countMetric)
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 200
name: ok
target: 0.95
rawMetric:
query:
sumoLogic:
type: logs
query: >-
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1" * * " as (status_code, size, tail)
| if (status_code matches "20" or status_code matches "30*",1,0)
as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc
op: lte
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: '2022-12-01 00:00:00'
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
apiVersion: n9/v1alpha
kind: SLO
metadata:
name: api-server-slo
displayName: API Server SLO
project: default
labels:
area:
- latency
- slow-check
env:
- prod
- dev
region:
- us
- eu
team:
- green
- sales
annotations:
area: latency
env: prod
region: us
team: sales
spec:
description: Example Sumo Logic SLO
indicator:
metricSource:
name: sumo-logic
project: default
kind: Agent
budgetingMethod: Occurrences
objectives:
- displayName: Good response (200)
value: 1
name: ok
target: 0.95
countMetrics:
incremental: true
good:
sumoLogic:
type: logs
query: |-
_collector="app-cluster" _source="logs"
| json "log"
| timeslice 15s as n9_time
| parse "level=* *" as (log_level, tail)
| if (log_level matches "error" ,0,1) as log_level_not_error
| sum(log_level_not_error) as n9_value by n9_time
| sort by n9_time asc
total:
sumoLogic:
type: logs
query: |-
_collector="app-cluster" _source="logs"
| json "log"
| timeslice 15s as n9_time
| parse "level=* *" as (log_level, tail)
| count(*) as n9_value by n9_time
| sort by n9_time asc
primary: true
service: api-server
timeWindows:
- unit: Month
count: 1
isRolling: false
calendar:
startTime: '2022-12-01 00:00:00'
timeZone: UTC
alertPolicies:
- fast-burn-5x-for-last-10m
attachments:
- url: https://docs.nobl9.com
displayName: Nobl9 Documentation
anomalyConfig:
noData:
alertMethods:
- name: slack-notification
project: default
alertAfter: 1h
Your logs query must contain the following:
Keyword | Description | Accepted input |
---|---|---|
timeslice | Groups log messages into fixed time intervals. Used to aggregate data points into consistent time buckets for time-series analysis. The result timestamp marks the beginning of each interval. | Time unit value: [number][unit] where unit is one of: s (seconds), m (minutes), h (hours), d (days).Must be 15s or greater.Examples: 1m , 30s , 2h |
n9time | The timestamp field used for time-based analysis. Represents when the data point was recorded or the interval it belongs to. | Unix timestamp in seconds or milliseconds |
n9value | The numerical value of the metric being measured | Floating-point number representing the metric value |
count(*) | An aggregation function that performs calculations on grouped data | One of:avg - average valuesum - total valuemin - minimum valuemax - maximum valuecount - number of itemsnone - no aggregation |
as | Search operator that creates an alias for a field or expression. Used to name the output fields for further processing. | Field alias name that follows Sumo Logic naming conventions. Required fields: n9time and n9value |
Refer to the YAML guide > SLO for more information on other fields.
Querying for logsβ
Sumo Logic queries use pipe operators (|
) to chain operations together. Each operator processes the results from the previous operation, progressively filtering and transforming the data to achieve the desired output.
All queries must start with either a keyword or string search.
Special characters are available for pattern matching:
*
- Wildcard character that matches zero or more characters?
- Matches exactly one character
Here's a detailed example of a Sumo Logic query that calculates successful HTTP responses:
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| if (status_code matches "20*" or status_code matches "30*",1,0) as resp_ok
| sum(resp_ok) as n9_value by n9_time
| sort by n9_time asc
This query:
- Filters logs from the nginx uploads category.
- Groups data into 1-minute intervals.
- Extracts HTTP status code, size, and remaining content.
- Marks responses with 2xx or 3xx status codes as successful (1) and others as failed (0).
- Sums successful responses for each time interval.
- Sorts results chronologically.
The query produces time-series data in the following format:
"n9_time","n9_value"
"1645371960000","2.0"
"1645372020000","58.0"
"1645372080000","46.0"
"1645372140000","12.0"
"1645372200000","12.0"
"1645372260000","12.0"
"1645372320000","14.0"
"1645372380000","22.0"
For comparison, here's a query that counts total requests in the same time ranges:
_sourceCategory=uploads/nginx
| timeslice 1m as n9_time
| parse "HTTP/1.1\" * * *" as (status_code, size, tail)
| count(*) as n9_value by n9_time
| sort by n9_time asc
API rate limitsβ
Sumo Logic enforces rate limits on Search Job API requests.
The Nobl9 agent makes several API calls following the documented Process Flow to collect data points. To manage these requests efficiently, the agent distributes them across the 2-minute interval.
To ensure timely SLI data collection, observe these API limits:
- API requestsβ4 requests/second (240/minute) per user
- Concurrent requestsβ10 per access key
Best practices to avoid rate limitsβ
- Use metrics over logs
- Prefer metrics queries as they are ~4x more efficient than logs queries
- Consider converting logs to metrics
- Optimize log queries
- Keep query execution time under 2 minutes
- Utilize Sumo Logic partitions
- Implement scheduled views
- Keep one agent connection for your Sumo Logic data sources
- For agent-based connections, use only one agent for all Sumo Logic data sources
- This restriction doesn't apply to direct connections
- Monitor usage
- Keep the number of log-based objectives within your API limits