Agent timestamp cache persistence

Reading time: 0 minute(s) (0 words)

The Nobl9 agent keeps information about the last successfully fetched data in the timestamps cache. When it gets data from a data source, it always attempts to do so beginning from the cached timestamp. The cache is by default kept in memory and therefore is lost when the agent is restarted.

Nobl9 has introduced the timestamps cache persistence (or simply, persistence) feature for the Nobl9 agent.

Scope of support

The Nobl9 agent timestamps cache persistence is not yet available for all data sources Nobl9 supports.

The table below presents the list of data sources along with their corresponding plugin names. You can also determine the minimum agent version required to use this feature:

Data source	Plugin name	Agent version
Amazon CloudWatch	n9cloudwatch	≥ 0.56.1
Amazon Prometheus	n9prometheus	≥ 0.65.0
Amazon Redshift	n9redshift	≥ 0.56.1
AppDynamics	n9appd	≥ 0.56.1
Azure Monitor beta	n9azure_monitor	≥ 0.69.0-beta01
BigQuery	n9bigquery	≥ 0.56.1
Datadog	n9datadog	≥ 0.64.0
Elasticsearch	n9elasticsearch	≥ 0.56.1
Dynatrace	n9dynatrace	≥ 0.56.1
Google Cloud Monitoring	n9gcm	≥ 0.56.1
InfluxDB	n9influxdb	≥ 0.56.1
Instana	n9instana	≥ 0.56.1
ServiceNow Cloud Observability	n9lightstep	≥ 0.56.1
New Relic	n9newrelic	≥ 0.56.1
OpenTSDB	n9opentsdb	≥ 0.56.1
Pingdom	n9pingdom	≥ 0.56.1
Prometheus	n9prometheus	≥ 0.65.0
Splunk Observability	n9splunk_observability	≥ 0.56.1
Splunk	n9splunk	≥ 0.56.1
Sumo Logic	n9sumologic	≥ 0.56.1
ThousandEyes	n9thousandeyes	≥ 0.56.1

Rules at a glance

• Persistence is available for agent versions >= 0.56.1.

• Persistence is deactivated by default and can be activated with the following optional parameters that can be set by environment variables:

  N9_PERSISTENCE_ENABLED=true
  N9_PERSISTENCE_ENABLED_PLUGINS= # specify one of the supported data sources by providing its plugin name. For more information, refer to the Scope of Support section of the documentation.
  N9_PERSISTENCE_PATH=/var/timestamps-cache
  note

  You can activate or deactivate persistence at any time. To disable this feature, set the value of the N9_PERSISTENCE_ENABLED variable to false.

• Persistence requires an additional PersistentVolume when the agent is run in a Kubernetes cluster or dedicated disk space when it is run in Docker. Refer to the code snippets below for details.

• The cache is updated only once the agent has successfully pushed data to the Nobl9 data intake.

• When restarted, the Nobl9 agent will fetch data starting from the last successfully pushed data point. As a consequence, if the agent has been disabled for a long time, the data source might reject requests for the resulting large time window or not return all the data requested. To prevent these situations, always consult the external data source’s documentation to check its limits.

Examples

Kubernetes

Here’s an example Kubernetes deployment YAML spec with persistence support and an additional specification for the PersistentVolumeClaim (highlighted in the snippet below):

apiVersion: v1
kind: Secret
metadata:
  name: nobl9-agent
  namespace: default
type: Opaque
stringData:
  client_id: "unique_client_id"
  client_secret: "unique_client_secret"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nobl9-agent
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      nobl9-agent-name: "nobl9-agent"
      nobl9-agent-project: "nobl9-agent"
      nobl9-agent-organization: "nobl9"
  template:
    metadata:
      labels:
        nobl9-agent-name: "nobl9-agent"
        nobl9-agent-project: "nobl9-agent"
        nobl9-agent-organization: "nobl9"
    spec:
      # Use this section if you want to authenticate with IAM role provided through the k8s ServiceAccount.
      # serviceAccount: <SERVICE_ACCOUNT>
      # serviceAccountName: <SERVICE_ACCOUNT_NAME>
      containers:
        - name: agent-container
          image: nobl9/agent:0.73.2
          resources:
            requests:
              memory: "350Mi"
              cpu: "0.1"
          env:
            - name: N9_CLIENT_ID
              valueFrom:
                secretKeyRef:
                  key: client_id
                  name: nobl9-agent
            - name: N9_CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  key: client_secret
                  name: nobl9-agent
            - name: N9_AUTH_SERVER
              value: "auseg9kiegWKEtJZC416"
            - name: N9_PERSISTENCE_ENABLED
              value: "true"
            - name: N9_PERSISTENCE_ENABLED_PLUGINS
              value: "n9appd" # specify one of the supported data sources by providing its plugin name
            - name: N9_PERSISTENCE_PATH
              value: "/var/timestamps-cache"
        volumeMounts:
        - name: timestamps-cache
          mountPath: /var/timestamps-cache
  volumes:
    - name: timestamps-cache
      persistentVolumeClaim:
        claimName: nobl9-agent-cache
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nobl9-agent-cache
  labels:
    app: nobl9
    component: agent-cache
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

Docker

Here’s an example Docker deployment command with persistence support:

docker run -d --restart on-failure \
  --name nobl9-agent \
  -e N9_CLIENT_SECRET="unique_client_secret" \
  -e N9_CLIENT_ID="unique_client_id" \
  -e AWS_ACCESS_KEY_ID="<AWS_ACCESS_KEY_ID>" \
  -e AWS_SECRET_ACCESS_KEY="<AWS_SECRET_ACCESS_KEY>" \
  -e N9_PERSISTENCE_ENABLED="true" \
  -e N9_PERSISTENCE_ENABLED_PLUGINS="plugin_name" \ # specify one of the supported data sources by providing its plugin name
  -e N9_PERSISTENCE_PATH="/var/timestamps-cache" \
  -v /var/timestamps-cache:/var/timestamps-cache \
  nobl9/agent:0.74.0-beta