Timestamp cache persistence

Reading time: 0 minute(s) (0 words)

The Nobl9 agent keeps information about the last successfully fetched data in the timestamp cache. When it gets data from a data source, it always attempts to do so beginning from the cached timestamp. The cache is by default kept in memory and therefore is lost when the agent is restarted.

Scope of support

Since both connection methods—agent and direct—use the Nobl9 agent, timestamp cache persistence is applicable to data sources connected using them both.

The table below presents the list of data source plugin names and minimum required agent versions for timestamp cache persistence.

Data source	Plugin name	Agent version
Amazon CloudWatch	n9cloudwatch	≥ 0.65.0
Amazon Prometheus	n9prometheus	≥ 0.65.0
Amazon Redshift	n9redshift	≥ 0.65.0
AppDynamics	n9appd	≥ 0.65.0
Azure Monitor beta	n9azure_monitor	≥ 0.69.0-beta01
Azure Monitor managed service for Prometheus beta	n9prometheus	≥ 0.78.0-beta
Datadog	n9datadog	≥ 0.65.0
Dynatrace	n9dynatrace	≥ 0.65.0
Elasticsearch	n9elasticsearch	≥ 0.65.0
Google BigQuery	n9bigquery	≥ 0.65.0
Google Cloud Monitoring	n9gcm	≥ 0.65.0
InfluxDB	n9influxdb	≥ 0.65.0
Instana	n9instana	≥ 0.65.0
LogicMonitor beta	n9logic_monitor	≥ 0.76.0-beta
New Relic	n9newrelic	≥ 0.65.0
OpenTSDB	n9opentsdb	≥ 0.65.0
Pingdom	n9pingdom	≥ 0.65.0
Prometheus	n9prometheus	≥ 0.65.0
ServiceNow Cloud Observability	n9lightstep	≥ 0.65.0
Splunk Observability	n9splunk_observability	≥ 0.65.0
Splunk	n9splunk	≥ 0.65.0
Sumo Logic	n9sumologic	≥ 0.65.0
ThousandEyes	n9thousandeyes	≥ 0.65.0

Rules at a glance

• The shorthand for timestamp cache persistence is persistence

• Persistence supports both agent and direct connections.
  Regardless of the connection method, the underlying process involves the agent.

• For data sources connected using the direct method, persistence is ON by default.

• For data sources connected using the agent method, persistence is OFF by default.

• To activate it, add the persistence parameters in your Kubernetes or Docker configuration and deploy the agent with N9_PERSISTENCE_ENABLED=true. To deactivate persistence, set N9_PERSISTENCE_ENABLED to false.

• Persistence requires an additional PersistentVolume when the agent is run in a Kubernetes cluster or dedicated disk space when it is run in Docker. Refer to the code snippets below for details.

• The cache is updated only after the agent successfully pushes data to the Nobl9 data intake.

• When the Nobl9 agent restarts after long inactivity, it requests for the data collected since the last successfully fetched data point. In these cases, the time window can be too large, so the data source returns data only for the part of the requested time window or rejects such a request completely. Consider your data source limitations for prevention.

Examples

Kubernetes

Here’s an example Kubernetes deployment YAML spec with persistence support and an additional specification for PersistentVolumeClaim:

apiVersion: v1
kind: Secret
metadata:
  name: nobl9-agent
  namespace: my-namespace
type: Opaque
stringData:
  client_id: "unique_client_id"
  client_secret: "unique_client_secret"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nobl9-agent
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      nobl9-agent-name: "nobl9-agent"
      nobl9-agent-project: "nobl9-agent"
      nobl9-agent-organization: "nobl9"
  template:
    metadata:
      labels:
        nobl9-agent-name: "nobl9-agent"
        nobl9-agent-project: "nobl9-agent"
        nobl9-agent-organization: "nobl9"
    spec:
      # Use this section if you want to authenticate with IAM role provided through the k8s ServiceAccount.
      # serviceAccount: <SERVICE_ACCOUNT>
      # serviceAccountName: <SERVICE_ACCOUNT_NAME>
      containers:
        - name: agent-container
          image: nobl9/agent:0.82.2
          resources:
            requests:
              memory: "350Mi"
              cpu: "0.1"
          env:
            - name: N9_CLIENT_ID
              valueFrom:
                secretKeyRef:
                  key: client_id
                  name: nobl9-agent
            - name: N9_CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  key: client_secret
                  name: nobl9-agent
            - name: N9_AUTH_SERVER
              value: "YOUR_VALUE"
            # Persistence parameters
            - name: N9_PERSISTENCE_ENABLED
              # Boolean
              value: "true"
            - name: N9_PERSISTENCE_ENABLED_PLUGINS
              # Your required data source's plugin name (see the table above)
              value: "n9plugin_name"
            - name: N9_PERSISTENCE_PATH
              value: "/var/timestamps-cache"
          volumeMounts:
            - name: timestamps-cache
              mountPath: /var/timestamps-cache
      volumes:
        - name: timestamps-cache
          persistentVolumeClaim:
            claimName: nobl9-agent-cache
---
# Persistence volume parameters
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nobl9-agent-cache
  labels:
    app: nobl9
    component: agent-cache
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      # While most cloud providers allow 1Gi as a minimum PVC size,
      # the Nobl9 agent requires 100Mi for reliable operation
      storage: 100Mi

Docker

Here’s an example Docker deployment command with persistence support:

docker run -d --restart on-failure \
  --name nobl9-agent \
  -e N9_CLIENT_SECRET="unique_client_secret" \
  -e N9_CLIENT_ID="unique_client_id" \
  -e AWS_ACCESS_KEY_ID="<AWS_ACCESS_KEY_ID>" \
  -e AWS_SECRET_ACCESS_KEY="<AWS_SECRET_ACCESS_KEY>" \
  -e N9_PERSISTENCE_ENABLED="true" \
  -e N9_PERSISTENCE_ENABLED_PLUGINS="n9plugin_name" \ # Your required data source plugin name from the table above
  -e N9_PERSISTENCE_PATH="/var/timestamps-cache" \
  -v /var/timestamps-cache:/var/timestamps-cache \
 nobl9/agent:0.82.2