Skip to main content

The Nobl9 Agent

Reading time: 0 minute(s) (0 words)

Nobl9 provides the ability to run the Agent to retrieve SLI metrics from configured data sources and send the data back to the Nobl9 backend for processing and error budget calculation. You can run the Agent in your Kubernetes cluster or as a Docker container.

The Agent is a lightweight application that executes the queries defined for your Nobl9 SLOs. Queries are written in the language supported by the data source in question and executed via native APIs. The interval at which the queries are executed varies by data source, but in most cases is one minute (refer to the Sources section of the documentation for more details about the resolution of the queries).

Why Use the Nobl9 Agent?

Nobl9 users can choose between a Direct or Agent configuration when connecting to a data source.

A Direct configuration requires users to enter their authentication credentials (API key, token, etc.). These values are encrypted and safely stored in Nobl9.

With an Agent configuration, the user passes their credentials when launching the Agent, and those credentials are not stored in the Nobl9 backend. What's more, you have access to the Agent's logs, which makes troubleshooting easier (see the Agent Troubleshooting page). For this reason, we recommend making your initial connection to a data source with the Agent configuration.

note

You can also use the Nobl9 Agent to collect and return data if your company's firewall blocks outbound connections. You still need to open a port for the Nobl9 Agent to send data back to Nobl9, but it removes the need for the Nobl9 application to make direct calls to your environment.

Getting Started with the Nobl9 Agent

Requirements

You can deploy the Nobl9 Agent in any Kubernetes cluster or Docker environment.

note

A Docker environment on your local machine with proper firewall access is enough for testing purposes. However, we do not advise using it past the initial test, as the data flow will stop when your machine sleeps.

Make sure that you are using the latest version of the Agent, so you have access to all the most recently introduced features. For details on the Agent releases, refer to the Agent Release Notes.

Creating the Agent

Creating the Agent in the UI

Follow these steps to create your Agent in the UI:

  1. Go to Integrations.
  2. Click the button.
  3. Select the relevant source and choose Agent.

For more Integration-specific details, refer to the Agent Configuration in the UI section in each Integration's subpage.

Creating the Agent through sloctl

To create your Agent through sloctl, you can apply the YAML file to deploy the Agent by using the sloctl apply command, for example:

sloctl apply -f ./agent.yaml

For more information, refer to the sloctl User Guide. For the Source-specific YAML definition of Agents, check the Integrations section - you can find them in the Agent Using CLI - YAML section in each Integration subpage.

caution

You can create only one Agent in one YAML file.

note

If you describe infrastructure as code, you might also consider creating Agents via Terraform. You can find more details in our Terraform documentation.

Deploying the Nobl9 Agent

Deploying the Agent in Kubernetes

When you add a new Agent via UI or sloctl, Nobl9 automatically generates a Kubernetes configuration in YAML and a Docker command line for you:

  • If you configure your Agent in the UI, these configurations are generated for you immediately.

  • If you create your Agent via YAML, you must go to the Integrations > Sources tab and find the respective Agent configuration tabs under Details.

If you have a running Kubernetes cluster, you can copy and paste the generated YAML into your Kubernetes configuration to deploy the Nobl9 Agent in your cluster.

Kubernetes invocation UI
Image 1: Kubernetes deployment command

Be sure to swap in your own credentials (e.g., <API_TOKEN>, <CREDENTIALS_FILE>); instructions in the UI specify what credentials need to be passed, and these differ depending on the data source.

Deploying the Agent in a Docker Container

When you add a new data source through the Nobl9 UI, a Docker command line is generated automatically for you to use to deploy the Agent. The following is a generic example of what you can see in the UI:

Be sure to swap in your own credentials (e.g., <API_TOKEN>, <CREDENTIALS_FILE>); instructions in the UI specify what credentials need to be passed.

Docker invocation UI
Image 2: Docker invocation for the N9 Agent

Checking the Agent Connection in the UI

To verify that the Agent has successfully connected to Nobl9, check for a valid timestamp in the Last Connection field in the UI.

Agent status UI
Image 3: Agent connection status in the UI

caution

Note that the Connected status does not indicate that the Agent is connected to the specified data source, only that it has successfully established a connection to the Nobl9 backend.

If you're using Prometheus, you can also check the connection status by scraping in from Agent's /health endpoint. Check Agent Metrics for more details.

Updating the Nobl9 Agent

If you need to update your Nobl9 Agent, you must apply the new config file with the latest image of the Agent in your Kubernetes cluster or Docker container.

Go to Integrations, find the relevant source on the list, and click the update button:

Agent update
Image 4: Nobl9 Agent update

The new Docker and Kubernetes configuration will be applied for you immediately, copy-paste the generated file and apply it to your Kubernetes cluster or run it in your Docker container:

Agent update config
Image 5: Nobl9 Agent update configuration

tip

You can go to Docker hub to check the latest Docker images of the Nobl9 Agent.

Exposing the Agent to Metrics

To expose the Agent's internal metrics, you must make the following changes in your Kubernetes cluster or Docker invocation:

  • To deploy the Agent, add the environment variable N9_METRICS_PORT, specifying the port on which you would like to expose the metrics (e.g., N9_METRICS_PORT=9876). These metrics are then available for scraping at /metrics.

  • If you deploy the Agent behind a firewall, make any changes required to allow your metrics system to scrape that port.

  • The Agent must be able to open connections to https://app.nobl9.com/api/input to send data to Nobl9.

Customizing Query Delay

Nobl9 Agent supports a QUERY_DELAY environment variable that allows to set a customized delay for queries when pulling the data from the data source.

note

You can also set the Query delay via Nobl9 UI or sloctl (the minimum Agent's version is 0.65.0-beta09).

Check the Query delay documentation for more details.

Nobl9, by default, tries to pull data from the previous minute. Sometimes the data in the observability platform may not be available, and, as such, specifying the query delay variable allows you to pull data from a time further than 1 minute in the past.

The following table specifies the list of the QUERY_DELAY environment variable names specific to each data source:

Data sourceVariable name
Amazon CloudWatchCW_QUERY_DELAY
Amazon RedshiftREDSHIFT_QUERY_DELAY
AppDynamicsAPPD_QUERY_DELAY
BigQueryBIGQUERY_QUERY_DELAY
DataDogDATADOG_QUERY_DELAY
DynatraceDYNATRACE_QUERY_DELAY
ElasticsearchES_QUERY_DELAY
Google Cloud MonitoringGCM_QUERY_DELAY
Grafana lokiGRAFANA_LOKI_QUERY_DELAY
GraphiteGRAPHITE_QUERY_DELAY
InfluxDBINFLUXDB_QUERY_DELAY
InstanaINSTANA_QUERY_DELAY
ServiceNow Cloud ObservabilityLS_QUERY_DELAY
NewRelicNR_QUERY_DELAY
OpenTSDBOPENTSDB_QUERY_DELAY
PingdomPINGDOM_QUERY_DELAY
PrometheusPROM_QUERY_DELAY
SplunkSPLUNK_QUERY_DELAY
Sumo LogicSUMOLOGIC_QUERY_DELAY
ThousandeyesTHOUSANDEYES_QUERY_DELAY
note

The variable value must be specified in the Golang Duration format. For more information, refer to the go documentation.

The following is an example of YAML configuration with the DYNATRACE_QUERY_DELAY variable:

apiVersion: apps/v1
kind: Deployment
metadata:
name: nobl9-agent-nobl9--default-dynatrace-agent
namespace: default
spec:
replicas: 1
selector:
matchLabels:
nobl9-agent-name: dynatrace-agent
nobl9-agent-project: default
nobl9-agent-organization: nobl9-org
template:
metadata:
labels:
nobl9-agent-name: dynatrace-agent
nobl9-agent-project: default
nobl9-agent-organization: nobl9-org
spec:
containers:
- name: agent-container
image: nobl9/agent:latest
resources:
requests:
memory: "350Mi"
cpu: "0.1"
env:
- name: N9_CLIENT_ID
valueFrom:
secretKeyRef:
key: client_id
name: nobl9-agent-nobl9-default-dynatrace-agent
- name: N9_CLIENT_SECRET
valueFrom:
secretKeyRef:
key: client_secret
name: nobl9-agent-nobl9-default-dynatrace-agent
- name: DYNATRACE_TOKEN
valueFrom:
secretKeyRef:
key: dynatrace_token
name: nobl9-agent-nobl9-default-dynatrace-agent
- name: DYNATRACE_QUERY_DELAY
value: 60s

Agent Timestamps Cache Persistence | Nobl9 Documentation

Agent Troubleshooting | Nobl9 Documentation

Agent Metrics | Nobl9 Documentation

Helm Charts | Nobl9 Documentation