Skip to main content

New Relic

Reading time: 0 minute(s) (0 words)

New Relic is a software solution that monitors performance and availability. It sets and rates application performance across the environment using a standardized Apdex (application performance index) score. You can use any New Relic query to gather SLIs and apply them in Nobl9 to define your new SLOs.

New Relic parameters and supported features in Nobl9
General support:
Release channel: Stable, Beta
Connection method: Agent, Direct
Replay and SLI Analyzer: Supported
Event logs: Supported
Query checker: Supported
Query parameters retrieval: Supported
Timestamp cache persistence: Supported
Query parameters:
Query interval: 1 min
Query delay: 1 min
Jitter: 15 sec
Timeout: 30 sec
Agent details and minimum required versions for supported features:
Environment variable: NR_QUERY_DELAY
Plugin name: n9newrelic
Replay and SLI Analyzer: 0.65.0
Maximum historical data retrieval period: 30 days
Query parameters retrieval: 0.73.2
Timestamp cache persistence: 0.65.0
Additional notes:
Support for New Relic queries
Learn more Opens in a new tab

Authentication​

Pass your New Relic Insights API Query Key, NEWRELIC_INSIGHTS_QUERY_KEY, as an environment variable during agent startup.

Query keys are not query-specific—any query key associated with your New Relic account will run any query.

The value of the New Relic account ID is returned by Intake API under the account_id key.

There is no possibility to overwrite the value of account_id with the environmental variables.

Your account_id shows on your New Relic account once you list out your accounts:

  1. Using the New Relic ONE interface, select Manage your plan in User preferences:

    Image 1: New Relic Preferences in the New Relic ONE UI

  2. Copy Account ID from the Plan management page:

    Image 2: Plan management page in the New Relic ONE UI

For more information on account ID, refer to the Account ID | New Relic documentation.

To retrieve your INSIGHTS_QUERY_KEY:

  1. On your New Relic Account, go to API Keys :
Image 3: New Relic Menu in the New Relic UI
  1. In the Looking for other keys? section, click the Insights Query Keys link:
Image 4: Insights Query Keys link in New Relic UI
caution
Don't create or use API Keys from the API Keys section of your New Relic account.

Nobl9 requires an Insights Query Key that begins with the NRIQ- prefix.

You can only obtain this key from the New Relic Insights API (see step 3 below).

  1. Once the New Relic Insights API tab has loaded, click the blue plus button next to the Query Keys section on the right-hand side of the page to create a new INSIGHTS_QUERY_KEY:
Image 5: New Relic Insights query key

Adding New Relic as a data source​

To ensure data transmission between Nobl9 and New Relic, it may be necessary to list Nobl9 IP addresses as trusted.

💻ip allowlist
IP addresses to include in your allowlist for secure access:

If you're using app.nobl9.com instance:
  • 18.159.114.21
  • 18.158.132.186
  • 3.64.154.26
If you're using us1.nobl9.com instance:
  • 34.121.54.120
  • 34.123.193.191
  • 34.134.71.10
  • 35.192.105.150
  • 35.225.248.37
  • 35.226.78.175
  • 104.198.44.161

You can add the New Relic data source using the direct or agent connection methods.

Direct connection method​

Direct configuration for NewRelic requires users to enter their credentials which Nobl9 stores safely.

Nobl9 Web​

To set up this type of connection:

  1. Navigate to Integrations > Sources.
  2. Click .
  3. Click the required Source button.
  4. Choose Direct.
  1. Select one of the following Release Channels:
    • The stable channel is fully tested by the Nobl9 team. It represents the final product; however, this channel does not contain all the new features of a beta release. Use it to avoid crashes and other limitations.
    • The beta channel is under active development. Here, you can check out new features and improvements without the risk of affecting any viable SLOs. Remember that features in this channel can change.

  2. Enter the New Relic Account ID to connect to your data source.
    Your Account ID shows on your New Relic account once you list out your accounts.

  3. Enter the Insights Query Key.
    Refer to the Authentication section above for more details.

  1. Select a Project.
    Specifying a project is helpful when multiple users are spread across multiple teams or projects. When the Project field is left blank, Nobl9 uses the default project.
  2. Enter a Display Name.
    You can enter a user-friendly name with spaces in this field.
  3. Enter a Name.
    The name is mandatory and can only contain lowercase, alphanumeric characters, and dashes (for example, my-project-1). Nobl9 duplicates the display name here, transforming it into the supported format, but you can edit the result.
  4. Enter a Description.
    Here you can add details such as who is responsible for the integration (team/owner) and the purpose of creating it.
  5. Specify the Query delay to set a customized delay for queries when pulling the data from the data source.
    • The default value in New Relic integration for Query delay is 1 minute.
    info
    Changing the Query delay may affect your SLI data. For more details, check the Query delay documentation.
  6. Enter a Maximum Period for Historical Data Retrieval.
    • This value defines how far back in the past your data will be retrieved when replaying your SLO based on this data source.
    • The maximum period value depends on the data source.
      Find the maximum value for your data source.
    • A greater period can extend the loading time when creating an SLO.
      • The value must be a positive integer.
  7. Enter a Default Period for Historical Data Retrieval.
    • It is used by SLOs connected to this data source.
    • The value must be a positive integer or 0.
    • By default, this value is set to 0. When you set it to >0, you will create SLOs with Replay.
  8. Click Add Data Source.

sloctl​

The YAML for setting up a direct connection to New Relic looks like this:

apiVersion: n9/v1alpha
kind: Direct
metadata:
  name: newrelic-direct
  displayName: Newrelic direct
  project: newrelic-direct
spec:
  description: direct integration with Newrelic
  sourceOf:
    - Metrics
    - Services
  queryDelay:
      unit: Minute
      value: 720
  logCollectionEnabled: false # boolean, defaults to 'false'. Set to true if you'd like your source to collect logs. Available for data sources connected using the direct method only. Reach out to support@nobl9.com to activate it.
  newRelic:
    accountId: 12345678
    insightsQueryKey: ""
  historicalDataRetrieval:
      maxDuration:
        value: 30
        unit: Day
      defaultDuration:
        value: 0
        unit: Day
FieldTypeDescription
queryDelay.unit
mandatory
enumSpecifies the unit for the query delay. Possible values: Second | Minute.
• Check query delay documentation for default unit of query delay for each source.
queryDelay.value
mandatory
numericSpecifies the value for the query delay.
• Must be a number less than 1440 minutes (24 hours).
• Check query delay documentation for default unit of query delay for each source.
logCollectionEnabled
optional
booleanOptional. Defaults to false. Set to true if you'd like your direct to collect event logs. Contact us to activate it.
releaseChannel
mandatory
enumSpecifies the release channel. Accepted values: beta | stable.
Source-specific fields
newRelic.accountID
mandatory
string, secretID number assigned to the New Relic user account. See authentication for more details.
newRelic.insightsQueryKey
mandatory
string, secretNew Relic's API Query Key. See authentication for more details.
Replay-related fields
historicalDataRetrieval
optional
n/aOptional structure related to configuration related to Replay.
❗ Use only with supported sources.
• If omitted, Nobl9 uses the default values of value: 0 and unit: Day for maxDuration and defaultDuration.
maxDuration.value
optional
numericSpecifies the maximum duration for historical data retrieval. Must be integer ≥ 0. See Replay documentation for values of max duration per data source.
maxDuration.unit
optional
enumSpecifies the unit for the maximum duration of historical data retrieval. Accepted values: Minute | Hour | Day.
defaultDuration.value
optional
numericSpecifies the default duration for historical data retrieval. Must be integer ≥ 0 and ≤ maxDuration.
defaultDuration.unit
optional
enumSpecifies the unit for the default duration of historical data retrieval. Accepted values: Minute | Hour | Day.

Agent connection method​

Nobl9 Web​

Follow the instructions below to configure your New Relic agent.

  1. Navigate to Integrations > Sources.
  2. Click .
  3. Click the required Source button.
  4. Choose Agent.
  1. Select one of the following Release Channels:
    • The stable channel is fully tested by the Nobl9 team. It represents the final product; however, this channel does not contain all the new features of a beta release. Use it to avoid crashes and other limitations.
    • The beta channel is under active development. Here, you can check out new features and improvements without the risk of affecting any viable SLOs. Remember that features in this channel can change.

  2. Enter the New Relic Account ID to connect to your data source.

  1. Select a Project.
    Specifying a project is helpful when multiple users are spread across multiple teams or projects. When the Project field is left blank, Nobl9 uses the default project.
  2. Enter a Display Name.
    You can enter a user-friendly name with spaces in this field.
  3. Enter a Name.
    The name is mandatory and can only contain lowercase, alphanumeric characters, and dashes (for example, my-project-1). Nobl9 duplicates the display name here, transforming it into the supported format, but you can edit the result.
  4. Enter a Description.
    Here you can add details such as who is responsible for the integration (team/owner) and the purpose of creating it.
  5. Specify the Query delay to set a customized delay for queries when pulling the data from the data source.
    • The default value in New Relic integration for Query delay is 1 minute.
    info
    Changing the Query delay may affect your SLI data. For more details, check the Query delay documentation.
  6. Enter a Maximum Period for Historical Data Retrieval.
    • This value defines how far back in the past your data will be retrieved when replaying your SLO based on this data source.
    • The maximum period value depends on the data source.
      Find the maximum value for your data source.
    • A greater period can extend the loading time when creating an SLO.
      • The value must be a positive integer.
  7. Enter a Default Period for Historical Data Retrieval.
    • It is used by SLOs connected to this data source.
    • The value must be a positive integer or 0.
    • By default, this value is set to 0. When you set it to >0, you will create SLOs with Replay.
  8. Click Add Data Source.

sloctl​

The YAML for setting up an agent connection to New Relic looks like this:

apiVersion: n9/v1alpha
kind: Agent
metadata:
  name: newrelic
  displayName: New Relic Agent
  project: newrelic
spec:
  description: Agent settings for new relic datasource
  sourceOf:
    - Metrics
    - Services
  releaseChannel: stable
  queryDelay:
      unit: Minute
      value: 720
  newrelic:
    accountId: 12345678
  historicalDataRetrieval:
      maxDuration:
        value: 30
        unit: Day
      defaultDuration:
        value: 0
        unit: Day
FieldTypeDescription
queryDelay.unit
mandatory
enumSpecifies the unit for the query delay. Possible values: Second | Minute.
• Check query delay documentation for default unit of query delay for each source.
queryDelay.value
mandatory
numericSpecifies the value for the query delay.
• Must be a number less than 1440 minutes (24 hours).
• Check query delay documentation for default unit of query delay for each source.
releaseChannel
mandatory
enumSpecifies the release channel. Accepted values: beta | stable.
Source-specific fields
newRelic.accountID
mandatory
string, secretID number assigned to the New Relic user account. See authentication section above for more details.
Replay-related fields
historicalDataRetrieval
optional
n/aOptional structure related to configuration related to Replay.
❗ Use only with supported sources.
• If omitted, Nobl9 uses the default values of value: 0 and unit: Day for maxDuration and defaultDuration.
maxDuration.value
optional
numericSpecifies the maximum duration for historical data retrieval. Must be integer ≥ 0. See Replay documentation for values of max duration per data source.
maxDuration.unit
optional
enumSpecifies the unit for the maximum duration of historical data retrieval. Accepted values: Minute | Hour | Day.
defaultDuration.value
optional
numericSpecifies the default duration for historical data retrieval. Must be integer ≥ 0 and ≤ maxDuration.
defaultDuration.unit
optional
enumSpecifies the unit for the default duration of historical data retrieval. Accepted values: Minute | Hour | Day.
note

It is possible that a user is assigned to multiple accounts. In any case, the account ID can be obtained in the same way.

warning

You can deploy only one agent in one YAML file by using the sloctl apply command.

Agent deployment​

When you add the data source, Nobl9 automatically generates a Kubernetes configuration and a Docker command line for you to use to deploy the agent. Both of these are available in the web UI, under the Agent Configuration section. Be sure to swap in your credentials (e.g., replace <NEWRELIC_INSIGHTS_QUERY_KEY> with your organization key).

If you use Kubernetes, you can apply the supplied YAML config file to a Kubernetes cluster to deploy the agent. It will look something like this:

# DISCLAIMER: This deployment description contains only the fields necessary for the purpose of this demo.
# It is not a ready-to-apply k8s deployment description, and the client_id and client_secret are only exemplary values.

apiVersion: v1
kind: Secret
metadata:
  name: nobl9-agent-nobl9-dev-newrelic-week-ble
  namespace: default
type: Opaque
stringData:
  newrelic_insights_query_key: "<NEWRELIC_INSIGHTS_QUERY_KEY>"
  client_id: "<CLIENT_ID>"
  client_secret: "<CLIENT_SECRET>"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nobl9-agent-nobl9-dev-newrelic-week-ble
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      nobl9-agent-name: "newrelicagent"
      nobl9-agent-project: "newrelic-week"
      nobl9-agent-organization: "nobl9-dev"
  template:
    metadata:
      labels:
        nobl9-agent-name: "newrelicagent"
        nobl9-agent-project: "newrelic-week"
        nobl9-agent-organization: "nobl9-dev"
    spec:
      containers:
        - name: agent-container
          image: nobl9/agent:0.82.2
          resources:
            requests:
              memory: "350Mi"
              cpu: "0.1"
          env:
            - name: N9_CLIENT_ID
              valueFrom:
                secretKeyRef:
                  key: client_id
                  name: nobl9-agent-nobl9-dev-newrelic-week-ble
            - name: N9_CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  key: client_secret
                  name: nobl9-agent-nobl9-dev-newrelic-week-ble
            - name: NEWRELIC_INSIGHTS_QUERY_KEY
              valueFrom:
                secretKeyRef:
                  key: newrelic_insights_query_key
                  name: nobl9-agent-nobl9-dev-newrelic-week-ble
            # The N9_METRICS_PORT is a variable specifying the port to which the /metrics and /health endpoints are exposed.
            # The 9090 is the default value and can be changed.
            # If you don’t want the metrics to be exposed, comment out or delete the N9_METRICS_PORT variable.
            - name: N9_METRICS_PORT
              value: "9090"

Creating SLOs with New Relic​

Nobl9 Web​

Follow the instructions below to create your SLOs with New Relic in the UI:

  1. Navigate to Service Level Objectives.

  2. Click .

  3. Select a data source for your SLO.

  4. In step 1 of the SLO wizard, select the service the SLO will be associated with.

  5. Specify the Metric. You can choose either a Threshold Metric, where a single time series is evaluated against a threshold or a Ratio Metric, which allows you to enter two time series to compare (for example, a count of good requests and total requests).

    1. Choose the Data Count Method for your ratio metric:
    • Non-incremental: counts incoming metric values one-by-one. So the resulting SLO graph is pike-shaped.
    • Incremental: counts the incoming metric values incrementally, adding every next value to previous values. It results in a constantly increasing SLO graph.

  6. Use the New Relic's query language (NRQL) to enter a Query or a Good query and Total Query for the metric you selected. For more details on NRQL and query samples, refer to the New Relic query examples section.

    • Nobl9 queries for time series data using the TIMESERIES clause.
      • You can add an empty TIMESERIES clause in your NRQL query or skip it altogether as Nobl9 adds missing TIMESERIES clause or overwrites the existing one.
      • As TIMESERIES clause is overwritten, do not append it with custom parameters.

    warning

    Do not append the time range by using the SINCE and UNTIL clauses in your queries as Nobl9 must have control over queried time range.

    For details on the position of the TIMESERIES clause, go to New Relic query examples below.

  7. In step 3, define a Time Window for the SLO.

  • Rolling time windows are better for tracking the recent user experience of a service.

  • Calendar-aligned windows are best suited for SLOs that are intended to map to business metrics measured on a calendar-aligned basis, such as every calendar month or every quarter.

  1. In step 4, specify the Error Budget Calculation Method and your Objective(s).

    • Occurrences method counts good attempts against the count of total attempts.
    • Time Slicesmethod measures how many good minutes were achieved (when a system operates within defined boundaries) during a time window.
    • You can define up to 12 objectives for an SLO.

    See the use case example and the SLO calculations guide for more information on the error budget calculation methods.

  2. In step 5, add the Display name, Name, and other settings for your SLO:

    • Create a composite SLO
    • Set notification on data, if this option is available for your data source.
      When activated, Nobl9 notifies you if your SLO hasn't received data or received incomplete data for more than 15 minutes.
    • Add alert policies, labels, and links, if required.
      You can add up to 20 links per SLO.

  3. Click Create SLO.

SLI values for good and total
When choosing the query for the ratio SLI (countMetrics), keep in mind that the values ​​resulting from that query for both good and total:
  • Must be positive.
  • While we recommend using integers, fractions are also acceptable.
    • If using fractions, we recommend them to be larger than 1e-4 = 0.0001.
  • Shouldn't be larger than 1e+20.

sloctl​

Here’s an example of New Relic using a rawMetric (threshold metric)

apiVersion: n9/v1alpha
kind: SLO
metadata:
  displayName: My SLO
  name: my-slo
  project: my-project
spec:
    budgetingMethod: Timeslices
    description: ""
    indicator:
      metricSource:
        name: newrelic
    service: newrelic-polakpotrafipl
    objectives:
    - target: 0.6
      op: lte
      rawMetric:
        query:
          newRelic:
            nrql: SELECT average(duration) FROM Transaction TIMESERIES
      displayName: rather-bad
      timeSliceTarget: 0.6
      value: 0.6
    - target: 0.99
      op: lte
      rawMetric:
        query:
          newRelic:
            nrql: SELECT average(duration) FROM Transaction TIMESERIES
      displayName: stretched
      timeSliceTarget: 0.99
      value: 1.2
    timeWindows:
    - count: 1
      isRolling: true
      unit: Hour

Specification for metric from New Relic always has one mandatory field:

  • nrql – It can be used to retrieve detailed New Relic data and get insight into your applications, hosts, and business-important activity.

Query examples​

caution: position of the TIMESERIES clause

Nobl9 queries to New Relic are sensitive to the position of the TIMESERIES clause. For example, to make the following correct NewRelic query work in Nobl9:

SELECT
    count(*)
FROM
// highlight-next-line
    Transaction TIMESERIES
WHERE
    httpResponseCode IN ('301', '200', '302')

Move the TIMESERIES clause to the end of your Nobl9 query:

SELECT
    count(*)
FROM
    Transaction
WHERE
// highlight-next-line
    httpResponseCode IN ('301', '200', '302') TIMESERIES

The following are valid query examples:

  • TIMESERIES clause:
    SELECT count(*) FROM Transaction TIMESERIES 1 hour

    1 hour will be overwritten by the agent and set to a different value.

  • Threshold metric for New Relic:
    Query: SELECT average(duration) FROM Transaction TIMESERIES

  • Ratio metric for New Relic:
    Good Query: SELECT count(*) FROM Transaction WHERE httpResponseCode IN ('200','301','302') TIMESERIES

    Total Query: SELECT count(*) FROM Transaction TIMESERIES

For details on New Relic Query Language, refer to the Introduction to NRQL | New Relic documentation.

One point in results​

Queries must be written in such a way that they return only one time series. In other words, you must always query for a single attribute or use a single function (or expression).

Examples:

❌ Incorrect queries:

  • SELECT * FROM Transaction - selects ALL attributes

  • SELECT average(duration), min(duration) FROM Transaction - selects two functions

✔ Correct queries:

  • SELECT count(*) FROM Transaction TIMESERIES - selects a single function.

  • SELECT average(duration) FROM Transaction TIMESERIES - selects a single attribute.

Other clauses mutually exclusive with TIMESERIES

❌ Other clauses that are mutually exclusive with TIMESERIES clause must not be used. Example:

  • SELECT count(*) FROM PageView FACET city

Querying the New Relic server​

The Nobl9 agent leverages NewRelic Insights API parameters. It pulls data at a two-minute interval from the New Relic server.

New Relic API rate limits​

For the information on New Relic rate limits, refer to the following documentation:

For a more in-depth look, consult additional resources:
Introduction to NRQLNew Relic docsQuery insights event data via APINew Relic docsRate limits for NRQL queriesNew Relic docsAgent metricsAgentCreating SLOs via TerraformTerraformCreating agents via TerraformTerraform
Last updated on
On this page