ThousandEyes

Reading time: 0 minute(s) (0 words)

ThousandEyes monitors the performance of both local and wide-area networks. ThousandEyes combines Internet and WAN visibility, browser synthetics, end-user monitoring, and Internet Insights to deliver a holistic view of your hybrid digital ecosystem – across cloud, SaaS, and the Internet. It's a SaaS-based tool that helps troubleshoot application delivery and maps Internet performance.

Scope of support

ThousandEyes parameters and supported features in Nobl9

General support:

Release channel: Stable Beta

Connection method: Agent Direct

Replay and SLI Analyzer: Historical data limit 30 days

Event logs: Supported

Query checker: Not supported

Query parameters retrieval: Supported

Timestamp cache persistence: Supported

---

Query parameters:

Query interval: 1 min

Query delay: 1 min

Jitter: 15 sec

Timeout: 60 sec

---

Agent details and minimum required versions for supported features:

Plugin name: n9thousandeyes

Query delay environment variable: THOUSANDEYES_QUERY_DELAY

Replay and SLI Analyzer: 0.97.0, 0.97.0-beta

Query parameters retrieval: 0.73.2

Timestamp cache persistence: 0.65.0

Nobl9 supports the following ThousandEyes tests:

Layer	Test name	Response property Data type	Nobl9 YAML name	Aggregation	Min agent version
DNS	DNSSEC	valid boolean	dns-dnssec-valid	Logical conjunction	0.67.2 0.68.0-beta01
	Server metric	resolutionTime number, unit: ms	dns-server-resolution-time	Mean	0.67.2 0.68.0-beta01
Network	End-to-end	avgLatency number, unit: ms	net-latency	Mean	0.55.3
	End-to-end	Loss number, unit: %	net-loss	Mean	0.55.3
Web	API	apiTransactionTime number, unit: ms	api-transaction-time	Mean	0.90.0 0.90.0-beta
	HTTP server	responseCode status code	http-server-availability	Max	0.55.3
	HTTP server	responseTime number, unit: ms	http-response-time	Mean	0.55.3
	HTTP server	throughput number, unit: bps	http-server-throughput	Mean	0.55.3
	HTTP server	totalTime number, unit: ms	http-server-total-time	Mean	0.67.2 0.68.0-beta01
	Page load	domLoadTime number, unit: ms	web-dom-load	Mean	0.55.3
	Page load	pageLoadTime number, unit: ms	web-page-load	Mean	0.55.3
	Transactions	transactionTime number, unit: ms	web-transaction-time	Mean	0.90.0 0.90.0-beta

DNS layer

DNSSEC

Returns 1 if the keychain for a domain record is valid, 0 otherwise.

Server metrics

Measures the time it takes for the ThousandEyes agent to receive a response about your domain from the DNS server.

Network layer

End-to-end metrics—latency

Tracks the interval time from when a network packet is sent to when the test-assigned agent receives the response.

End-to-end metrics—loss

Represents the total loss of the network packet.

Web layer

API

Calculates the total duration of an API request transaction, from the initiation to the point where the server responds. This check includes both network transmission time and server processing time for the API request.

HTTP server—response code

Checks agent availability by examining HTTP status codes, such as 200, 201, 400, or 501.

Handling multiple ThousandEyes agents

When there are multiple ThousandEyes agents, Nobl9 can receive multiple HTTP response status codes in a single request. In this situation, Nobl9 ingests only the highest status code.

HTTP server—response time

Measures the interval between the start of the request and the web server successfully sending the first byte of the response to the client.

HTTP server—throughput

Determines the Wire Size divided by the Receive Time.

HTTP server—total time

Calculates the time it takes for the ThousandEyes agent to receive the last byte of your server's response.

Page load—DOM

Tracks the time interval required by the browser to build the website’s Document Object Model (DOM).

Page load—page

Records the time interval from the load event to the point when the website is fully loaded.

Web transactions

Represents the total time taken to complete a web transaction like page loading, interacting with page elements, or form submission. This metric provides insight into the time taken for an end user to complete a full interaction sequence on your web service.

Points to consider

• Metrics are collected during each ThousandEyes test interval.
• Aggregation is applied when a test returns multiple results within a single test round.

Data source and channel compatibility

Ensure your ThousandEyes SLOs use compatible data source versions and channels to guarantee seamless configuration delivery.

Creating SLOs with ThousandEyes

Nobl9 Web

Follow the instructions below to create your SLOs with ThousandEyes in the UI:

1. Navigate to Service Level Objectives.
2. Click .
3. Select a Service.
   It will be the location for your SLO in Nobl9.
4. Select your ThousandEyes data source.
   
5. Modify Period for Historical Data Retrieval, if necessary.

   • This value defines how far back in the past your data will be retrieved when replaying your SLO based on ThousandEyes.
   • A longer period can extend the data loading time for your SLO.
   • Must be a positive whole number up to the maximum period value you've set when adding the ThousandEyes data source.

   
   

   Non-editable Replay period

   Non-editable Replay period indicates that the maximum period for historical data retrieval set for your ThousandEyes data source is set to zero.
   Adjust the data source settings to create the SLO with Replay.
6. Specify Metric:
   • Test ID is an ID of the test configured in ThousandEyes. It is a mandatory field, and it must be a positive number.
   • Test type is your required ThousandEyes test. You can choose between the supported ThousandEyes tests.

7. Define the Time window for your SLO:

   • Rolling time windows constantly move forward as time passes. This type can help track the most recent events.
   • Calendar-aligned time windows are usable for SLOs intended to map to business metrics measured on a calendar-aligned basis.
8. Configure the Error budget calculation method and Objectives:

   • Occurrences method counts good attempts against the count of total attempts.
   • Time Slices method measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
   • You can define up to 12 objectives for an SLO.
   
   
   Similar threshold values for objectives
   To use similar threshold values for different objectives in your SLO, we recommend differentiating them by setting varying decimal points for each objective.
   For example, if you want to use threshold value 1 for two objectives, set it to 1.0000001 for the first objective and to 1.0000002 for the second one.
9. Add the Display name, Name, and other settings for your SLO:

   • Name identifies your SLO in Nobl9. After you save the SLO, its name becomes read-only.
     Use only lowercase letters, numbers, and dashes.
   • Select No data anomaly alert to receive notifications when your SLO stops reporting data for a specified period:
     • Choose up to five supported Alert methods.
     • Specify the delay period before Nobl9 sends an alert about the missing data.
       From 5 minutes to 31 days. Default: 15 minutes
   • Add alert policies, labels, and links, if required.
     Limits per SLO: 20 alert policies or links, 30 labels.
10. Click CREATE SLO.


SLO configuration use case
Check the SLO configuration use case for a real-life SLO example.

YAML

End-to-end (latency)

Sample ThousandEyes net latency SLO

apiVersion: n9/v1alpha
kind: SLO
metadata:
  name: api-server-slo
  displayName: API Server SLO
  project: default
  labels:
    area:
      - latency
      - slow-check
    env:
      - prod
      - dev
    region:
      - us
      - eu
    team:
      - green
      - sales
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example ThousandEyes SLO
  indicator:
    metricSource:
      name: thousand-eyes
      project: default
      kind: Agent
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good response (200)
      value: 200.0
      name: ok
      target: 0.95
      rawMetric:
        query:
          thousandEyes:
            testID: 2280492
            testType: net-latency
      op: lte
      primary: true
  service: api-server
  timeWindows:
    - unit: Month
      count: 1
      isRolling: false
      calendar:
        startTime: "2022-12-01 00:00:00"
        timeZone: UTC
  alertPolicies:
    - fast-burn-5x-for-last-10m
  attachments:
    - url: https://docs.nobl9.com
      displayName: Nobl9 Documentation
  anomalyConfig:
    noData:
      alertMethods:
        - name: slack-notification
          project: default
      alertAfter: 1h

API transaction time

Sample ThousandEyes API transaction time

apiVersion: n9/v1alpha
kind: SLO
metadata:
  name: api-server-slo
  displayName: API Server SLO
  project: default
  labels:
    area:
      - latency
      - slow-check
    env:
      - prod
      - dev
    region:
      - us
      - eu
    team:
      - green
      - sales
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example ThousandEyes SLO
  indicator:
    metricSource:
      name: thousand-eyes
      project: default
      kind: Agent
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good response (200)
      value: 200
      name: ok
      target: 0.95
      rawMetric:
        query:
          thousandEyes:
            testID: 2280492
            testType: api-transaction-time
      op: lte
      primary: true
  service: api-server
  timeWindows:
    - unit: Month
      count: 1
      isRolling: false
      calendar:
        startTime: '2022-12-01 00:00:00'
        timeZone: UTC
  alertPolicies:
    - fast-burn-5x-for-last-10m
  attachments:
    - url: https://docs.nobl9.com
      displayName: Nobl9 Documentation
  anomalyConfig:
    noData:
      alertMethods:
        - name: slack-notification
          project: default
      alertAfter: 1h

HTTP server (response code)

Sample ThousandEyes HTTP server response code SLO

apiVersion: n9/v1alpha
kind: SLO
metadata:
  name: api-server-slo
  displayName: API Server SLO
  project: default
  labels:
    area:
      - latency
      - slow-check
    env:
      - prod
      - dev
    region:
      - us
      - eu
    team:
      - green
      - sales
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example ThousandEyes SLO
  indicator:
    metricSource:
      name: thousand-eyes
      project: default
      kind: Agent
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good response (200)
      value: 200.0
      name: ok
      target: 0.95
      rawMetric:
        query:
          thousandEyes:
            testID: 2280492
            testType: http-server-availability
      op: lte
      primary: true
  service: api-server
  timeWindows:
    - unit: Month
      count: 1
      isRolling: false
      calendar:
        startTime: "2022-12-01 00:00:00"
        timeZone: UTC
  alertPolicies:
    - fast-burn-5x-for-last-10m
  attachments:
    - url: https://docs.nobl9.com
      displayName: Nobl9 Documentation
  anomalyConfig:
    noData:
      alertMethods:
        - name: slack-notification
          project: default
      alertAfter: 1h

Page load (DOM)

Sample ThousandEyes Page load (DOM) SLO

apiVersion: n9/v1alpha
kind: SLO
metadata:
  name: api-server-slo
  displayName: API Server SLO
  project: default
  labels:
    area:
      - latency
      - slow-check
    env:
      - prod
      - dev
    region:
      - us
      - eu
    team:
      - green
      - sales
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example ThousandEyes SLO
  indicator:
    metricSource:
      name: thousand-eyes
      project: default
      kind: Agent
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good response (200)
      value: 200.0
      name: ok
      target: 0.95
      rawMetric:
        query:
          thousandEyes:
            testID: 2280492
            testType: web-dom-load
      op: lte
      primary: true
  service: api-server
  timeWindows:
    - unit: Month
      count: 1
      isRolling: false
      calendar:
        startTime: "2022-12-01 00:00:00"
        timeZone: UTC
  alertPolicies:
    - fast-burn-5x-for-last-10m
  attachments:
    - url: https://docs.nobl9.com
      displayName: Nobl9 Documentation
  anomalyConfig:
    noData:
      alertMethods:
        - name: slack-notification
          project: default
      alertAfter: 1h

Web transactions time

Sample ThousandEyes Web transactions time SLO

apiVersion: n9/v1alpha
kind: SLO
metadata:
  name: api-server-slo
  displayName: API Server SLO
  project: default
  labels:
    area:
      - latency
      - slow-check
    env:
      - prod
      - dev
    region:
      - us
      - eu
    team:
      - green
      - sales
  annotations:
    area: latency
    env: prod
    region: us
    team: sales
spec:
  description: Example ThousandEyes SLO
  indicator:
    metricSource:
      name: thousand-eyes
      project: default
      kind: Agent
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good response (200)
      value: 200.0
      name: ok
      target: 0.95
      rawMetric:
        query:
          thousandEyes:
            testID: 2280492
            testType: web-transaction-time
      op: lte
      primary: true
  service: api-server
  timeWindows:
    - unit: Month
      count: 1
      isRolling: false
      calendar:
        startTime: "2022-12-01 00:00:00"
        timeZone: UTC
  alertPolicies:
    - fast-burn-5x-for-last-10m
  attachments:
    - url: https://docs.nobl9.com
      displayName: Nobl9 Documentation
  anomalyConfig:
    noData:
      alertMethods:
        - name: slack-notification
          project: default
      alertAfter: 1h

Click to open field reference

Field	Type	Description
apiVersion mandatory	string	API version. Use n9/v1alpha
kind mandatory	string	The resource type. Use SLO
Metadata
metadata.name mandatory	string	Name identifier for the SLO. Use only lowercase alphanumeric characters
metadata.displayName	string	User-friendly SLO name
metadata.project mandatory	string	The name identifier of the project where you need to host your SLO
metadata.labels	object (map: string[])	Grouping labels for filtering or viewing
metadata.annotations	object (map: string)	Flat string annotations
Spec
spec.description	string	SLO description
spec.indicator.metricSource.name mandatory	string	Data source name
spec.indicator.metricSource.project mandatory	string	Project containing the data source
spec.indicator.metricSource.kind mandatory	string	Data source connection method. Can be Agent or Direct
spec.budgetingMethod mandatory	enum	Error budget calculation method. Can be Occurrences or Time slices
spec.objectives mandatory	array	Your SLO objective definition, up to 12 objectives per SLO.
spec.objectives[].displayName	string	User-friendly objective name
spec.objectives[].value mandatory	number	Data point values that is considered "good" (e.g., 200.0). In SLOs with two or more objectives, keep each objective's value unique. In ratio (count) metrics, value is retained for legacy purposes.
spec.objectives[].name mandatory	string	Name identifier for this objective
spec.objectives[].op mandatory	string (enum)	Operator for objective. One of: lte (less than or equal to) lt (less than) gte (greater than or equal to) gt (greater than)
spec.objectives[].target mandatory	float	The percentage of the good minutes or occurrences that must meet the desired performance (e.g., is the target is 0.95, the good performance is expected to be observed in at least 95% of the time window)
spec.objectives[].rawMetric/.countMetric mandatory	object	The metric type indicator. Set: rawMetric for a threshold metric countMetric for a ratio metric. A ratio metric requires the additional fields: countMetric.incremental (boolean) the data count method countMetric.good/.bad and countMetric.total a numerator and denominator queries
spec.objectives[].countMetric.incremental mandatory	boolean	The data count method for a ratio (countMetric) metric type
spec.objectives[].primary	boolean	The indicator of a primary SLO objective
spec.service mandatory	string	The name identifier of a service to host this SLO. The service must exist in the project specified in metadata.project
spec.timeWindows mandatory	array	Defines SLO time window for error budget calculation. Set: isRolling: true for the rolling time window type isRolling: false for the calendar-aligned type
spec.timeWindows.unit mandatory	integer	The time window units. One of: Day | Hour | Minute for the rolling time window Year | Quarter | Month | Week | Day for the calendar-aligned time window
spec.timeWindows.count mandatory	integer	The number of units in a time window
spec.timeWindows.startTime	string	Mandatory for calendar-aligned time windows. Date and time in the format YYYY-MM-DDTHH:mm:ss
spec.timeWindows.timeZone	string	Mandatory for calendar-aligned time-windows. A valid IANA Time Zone Database name
spec.timeWindows.isRolling mandatory	boolean	true for the rolling time window type false for the calendar-aligned type
spec.alertPolicies	array	The name identifiers of alert policies to be linked to this SLO (must be from the same project as the SLO). Up to 20 alert policies per SLO.
spec.attachments	array	Links to any additional attributes of this SLO
spec.anomalyConfig	object	Settings for a manual no data anomaly detection rule
spec.noData.alertMethods	array	List of alert methods for no-data anomaly. Up to five alert methods per SLO. Every alert method must have the name and project fields
spec.noData.alertAfter	string	Waiting time before sending a no-data notification. Must be 5m to 31d. Default: 15m
Source-specific fields
thousandEyes.testID mandatory	string	An identifier of your required test configured in ThousandEyes
thousandEyes.testType mandatory	string	Your ThousandEyes test type. Must be one of the supported tests. If omitted, net-latency is used by default Important: the web-transaction-time and api-transaction-time tests are supported by the Nobl9 agent v0.90.0 / v0.90.0-beta or higher

Find more SLO examples in the Nobl9 SDK

Querying the ThousandEyes server

Nobl9 queries the ThousandEyes instance once every minute. If the agent doesn’t collect any data in a given minute, the next query will be extended to the past two minutes.

ThousandEyes API rate limits

The ThousandEyes API throttles API requests using a 240 request-per-minute (per organization) limit with each page having max 1000 elements included.

Useful links

Check out these related guides and references:

Add ThousandEyes as a data sourceAdding data sourcesThousandEyes knowledge baseThousandEyes documentationThousandEyes testsThousandEyes documentationCreating SLOs with TerraformNobl9 Terraform provider documentation