Nobl9 agent
The Nobl9 agent is a lightweight application for retrieving SLI metrics from data sources you configure and sending data back to the Nobl9 backend for processing and calculating the reliability of your system. You can run the agent in your Kubernetes cluster or as a Docker container.
The Nobl9 agent runs queries for your SLOs;
these queries are tailored to the specific data source and sent using native APIs.
By default,
the Nobl9 agent queries a data source every minute to capture the last-minute data.
However, this frequency can be customized to fit your specific needs.
Starting from release 1.66, any agent versions equal to or earlier than 0.55.3
will be deprecated and rendered non-functional.
If any of your sources currently use deprecated agents, make sure they are updated to the latest agent version.
Nobl9 agent requirements
-
Nobl9 agent deployment: use any Kubernetes cluster or Docker environment.
Agent testingA Docker environment on your local machine with proper firewall access is enough for testing purposes. However, we do not recommend using it past the initial test, as the data flow will stop when your machine sleeps.
-
For access to the most recent functionality, regularly update your agent.
The latest agent version isnobl9/agent:0.69.4
.
Read more about agent releases.
Creating the agent
To ensure data transmission from your data source to Nobl9, it may be necessary to list Nobl9 IP addresses as trusted.
- 18.159.114.21
- 18.158.132.186
- 3.64.154.26
Creating the agent in the UI
Follow these steps to create your agent in the UI:
- Go to Integrations.
-
Click .
- Select the relevant source and choose Agent.
Find integration-specific details in the Agent Configuration in the UI section under the required data source page.
Creating the agent through sloctl
To create your agent through sloctl
, you can apply the YAML file to deploy the agent by using the sloctl apply
command, for example:
sloctl apply -f ./agent.yaml
Find the source-specific YAML definition of agents under the Sources > Agent Using CLI - YAML sections on the required data source page.
You can create only one agent in one YAML file.
If you describe infrastructure as code, you might also consider creating agents via Terraform. You can find more details in our Terraform documentation.
Deploying the Nobl9 agent
Deploying the agent in Kubernetes
When you add a new agent using the Nobl9 UI or sloctl
, Nobl9 automatically generates a Kubernetes configuration in YAML and a Docker command line for you:
-
In the Nobl9 UI, the configurations are generated immediately.
-
For
sloctl
, go to the Integrations > Sources tab. Click the required data source and find the configurations under Details.
If you have a running Kubernetes cluster, you can copy and paste the generated YAML into your Kubernetes configuration to deploy the Nobl9 agent in your cluster.
Swap in your own credentials (e.g., <API_TOKEN>
, <CREDENTIALS_FILE>
);
instructions in the UI specify the credentials to pass.
The credentials depend on the data source.
Deploying the agent in a Docker container
When you add a new data source through the Nobl9 UI, a Docker command line is generated automatically for you to use to deploy the agent.
Swap in your own credentials (e.g., <API_TOKEN>
, <CREDENTIALS_FILE>
); instructions in the UI specify the credentials to pass.
Checking the agent connection in the UI
To verify that the agent has successfully connected to Nobl9, check for a valid timestamp in the Last Connection field in the UI.
Note that the Connected status does not indicate that the agent is connected to the specified data source, only that it has successfully established a connection to the Nobl9 backend.
If you're using Prometheus, you can also check the connection status by scraping in from agent's /health
endpoint. Check Agent metrics for more details.
Updating the Nobl9 agent
If you need to update your Nobl9 agent, you must apply the new config file with the latest image of the agent in your Kubernetes cluster or Docker container.
Go to Integrations, find the relevant source on the list, and click the update button:
The new Docker and Kubernetes configuration will be applied for you immediately, copy-paste the generated file and apply it to your Kubernetes cluster or run it in your Docker container:
You can go to Docker hub to check the latest Docker images of the Nobl9 agent.
Exposing the agent to metrics
To expose the agent's internal metrics, you must make the following changes in your Kubernetes cluster or Docker invocation:
-
To deploy the agent, add the environment variable
N9_METRICS_PORT
, specifying the port on which you would like to expose the metrics (e.g.,N9_METRICS_PORT=9876
). These metrics are then available for scraping at/metrics
. -
If you deploy the agent behind a firewall, make any changes required to allow your metrics system to scrape that port.
-
The agent must be able to open connections to
https://app.nobl9.com/api/input
to send data to Nobl9.
Customizing query delay
Nobl9 agent supports a QUERY_DELAY
environment variable that allows to set a customized delay for queries when pulling the data from the data source.
You can also set the Query delay via Nobl9 UI or sloctl
(the minimum agent's version is 0.65.0-beta09
).
Check the Query delay documentation for more details.
Nobl9, by default, tries to pull data from the previous minute. Sometimes the data in the observability platform may not be available, and, as such, specifying the query delay variable allows you to pull data from a time further than one minute in the past.
The following table specifies the list of the QUERY_DELAY
environment variable names specific to each data source:
Data source | Variable name |
---|---|
Amazon CloudWatch | CW_QUERY_DELAY |
Amazon Redshift | REDSHIFT_QUERY_DELAY |
AppDynamics | APPD_QUERY_DELAY |
Azure Monitor | AZURE_MONITOR_QUERY_DELAY |
BigQuery | BIGQUERY_QUERY_DELAY |
DataDog | DATADOG_QUERY_DELAY |
Dynatrace | DYNATRACE_QUERY_DELAY |
Elasticsearch | ES_QUERY_DELAY |
Google Cloud Monitoring | GCM_QUERY_DELAY |
Grafana loki | GRAFANA_LOKI_QUERY_DELAY |
Graphite | GRAPHITE_QUERY_DELAY |
InfluxDB | INFLUXDB_QUERY_DELAY |
Instana | INSTANA_QUERY_DELAY |
ServiceNow Cloud Observability | LS_QUERY_DELAY |
NewRelic | NR_QUERY_DELAY |
OpenTSDB | OPENTSDB_QUERY_DELAY |
Pingdom | PINGDOM_QUERY_DELAY |
Prometheus | PROM_QUERY_DELAY |
Splunk | SPLUNK_QUERY_DELAY |
Sumo Logic | SUMOLOGIC_QUERY_DELAY |
Thousandeyes | THOUSANDEYES_QUERY_DELAY |
The variable value must be specified in the Golang Duration format. For more information, refer to the go
documentation.
The following is an example of YAML configuration with the DYNATRACE_QUERY_DELAY
variable:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nobl9-agent-nobl9--default-dynatrace-agent
namespace: default
spec:
replicas: 1
selector:
matchLabels:
nobl9-agent-name: dynatrace-agent
nobl9-agent-project: default
nobl9-agent-organization: nobl9-org
template:
metadata:
labels:
nobl9-agent-name: dynatrace-agent
nobl9-agent-project: default
nobl9-agent-organization: nobl9-org
spec:
containers:
- name: agent-container
image: nobl9/agent:0.69.4
resources:
requests:
memory: "350Mi"
cpu: "0.1"
env:
- name: N9_CLIENT_ID
valueFrom:
secretKeyRef:
key: client_id
name: nobl9-agent-nobl9-default-dynatrace-agent
- name: N9_CLIENT_SECRET
valueFrom:
secretKeyRef:
key: client_secret
name: nobl9-agent-nobl9-default-dynatrace-agent
- name: DYNATRACE_TOKEN
valueFrom:
secretKeyRef:
key: dynatrace_token
name: nobl9-agent-nobl9-default-dynatrace-agent
- name: DYNATRACE_QUERY_DELAY
value: 60s