# SLO Annotations

Reading time: 0 minute(s) (0 words)

When you experience any outstanding event related to your services, it can be helpful to add a note using the user annotations about what happened and have that live alongside your metrics. You can also leverage Annotations to record and track recent or upcoming deployments and hotfixes.

Thanks to system annotations that are added each time to the SLO Objective charts when an alert is triggered or resolved, you can find answers to the following questions more easily:

• Why an alert was triggered?
• How does the triggered alert correlates with the information on the SLO charts?
• What are the next steps that should be taken?

## Limitations​

The Nobl9 SLO Annotations service adheres to the following criteria:

• Annotations are stored for one year.
• The maximum number of characters per annotation is 1000.

## Overview​

Annotations can be added using the Nobl9 UI, sloctl, or through the Nobl9 API.

As you can create annotations for a specific SLO Objective via sloctl or Nobl9 API, annotations (both, created by the user or system annotations added by default) attached to a specific objective are displayed only on the charts related to that objective:

• If you select all of the underlying SLO Objectives to be displayed on the charts in the SLO Details tab, the SLI chart will display every annotation created for all the Objectives and the SLO itself.
• If you select only one Objective to be displayed on the SLI chart, only the annotations created for that particular Objective will be displayed.

## System Annotations​

System annotations are added to the SLO Objective charts each time an alert is triggered or resolved - they are created by default and are not related to any specific user.

System annotations regarding alerts are created based on metric timestamp. Those annotations represent the time when an SLI indicates that alert was triggered or resolved. For the delayed metrics, Nobl9 will draw annotation in the past - note that it is not related to time when you receive a notification.

note

System annotations are displayed regardless of whether an Alert Policy is silenced. That is, even if you silence an Alert Policy for a given SLO, Nobl9 will create the annotation for the silenced alert and display it on the SLO charts.

### RBAC​

The following are the RBAC limitations for system annotations:

• You cannot edit system annotations.
• Only Organization Admins can delete system annotations, for example, when a false alert was triggered.
• Users with other permissions can only view system annotations.

### Managing System Annotations in the UI​

You can choose to show or hide system or user annotations in the SLO Details. Refer to the Managing System Annotations section for detailed instructions.

## User Annotations​

When Nobl9 users experience any outstanding event related to their services, it can be helpful for them to add a note about what happened and have that live alongside their metrics. This information is crucial when performing historical analyses of product reliability. It can help answer the following questions:

• What caused my error budget to burn so quickly?
• Why did my SLI spike?
• When and how long was my Service down?

Nobl9 users can also leverage Annotations to record and track recent or upcoming deployments and hotfixes. Such annotations, alongside your SLI metrics, can help investigate the following questions:

• We just deployed a new release; how can I track when the deployment occurred?
• Was the spike in my SLI caused by a hotfix (scheduled or unscheduled)?”

### Adding Annotations in the UI​

Refer to the Managing Annotations in the UI section for detailed instructions.

## Managing Annotations in sloctl​

The following sloctl commands are available to use in the SLO Annotations service:

caution

sloctl does not return system annotations. When applying the sloctl get annotation[s] command, sloctl will return only annotations created by the user.

### Applying Annotations​

You can add or update annotations using the sloctl apply -f {yamlFile} command.

The following YAML example shows how to add an annotation to your SLO:

apiVersion: n9/v1alphakind: Annotationmetadata:  name: annotation-name  project: defaultspec:  slo: test-slo  description: test annotation description  objectiveName: objective-1 # optional, must refer to existing SLO Objective  startTime: 2021-11-02T17:10:05Z  endTime: 2021-11-22T17:15:05Z

Important Notes:

• The metadata.name field contains a unique annotation name, required for distinguishing project annotations.

• The available YAML file spec fields (all mandatory) are:

• slo - the name of the SLO the annotation applies to.

• description - a string (plain text) describing the annotation. The maximum number of characters is 1000.

• startTime, endTime- these fields define the date-time point where the annotation will be placed in the graph. The values must be in the YYYY-MM-DDTh:mm:ssZ format that complies with ISO8601. If startTime == endTime , the annotation will be placed at a single time point.

• The objectiveName field is optional, it allows you to add an annotation to a particular, already existing SLO Objective. If this field is not specified, annotation is attached to the SLO.

note

By default, sloctl always returns time in UTC. You can adjust this for your time zone by replacing the Z (for Zulu; a shorthand for UTC) with the offset from UTC, prefaced with either + or -. For instance, to have sloctl return times adjusted for Eastern Standard Time you would use startTime: 2021-11-02T17:10:05-05:00.

### Deleting Annotations​

To delete an annotation, run the following command: sloctl delete -f {yamlFile}.

### Validation Errors​

The following are possible errors you may run into while applying annotations through sloctl:

• If the specified SLO does not exist, as in this example:

apiVersion: n9/v1alphakind: Annotationmetadata:  name: annotation-name  project: my-projectspec:  slo: bad-slo-name  description: test annotation description  startTime: 2006-01-02T17:10:05Z  endTime: 2006-01-02T17:15:05Z

sloctl returns the following error:

applying Annotation annotation-test-1 in 'default' project failed, because object SLO 'bad-slo-name' referenced in its spec does not exist in 'my-project' project

• If the startTime or endTime format is invalid, as shown here:

apiVersion: n9/v1alphakind: Annotationmetadata:  name: annotation-name  project: my-projectspec:  slo: bad-slo-name  description: test annotation description  startTime: 2006-01-02T17:10:05Z  endTime: 2006-01-02T15:04:05Z07:00

sloctl returns the following error:

'Annotation.spec.startTime' Error:Field validation for 'startTime' failed on the 'iso8601dateTimeFormatRequired' tag

• If the annotation name is blank:

apiVersion: n9/v1alphakind: Annotationmetadata:  name:  project: my-projectspec:  slo: bad-slo-name  description: test annotation description  startTime: 2006-01-02T17:10:05Z  endTime: 2006-01-02T15:04:05Z07:00

sloctl returns the following error:

Annotation.ObjectHeader.MetadataHolder.metadata.name' Error:Field validation for 'name' failed on the 'required' tag

• If the endTime is before the startTime:

apiVersion: n9/v1alphakind: Annotationmetadata:  name:  project: my-projectspec:  slo: bad-slo-name  description: test annotation description  startTime: 2020-01-02T17:10:05Z  endTime: 2019-01-02T15:04:05Z07:00

sloctl returns the following error:

'Annotation.spec.endTime' Error:Field validation for 'endTime' failed on the 'endTimeBeforeStartTime' tag

• If the required description is missing:

apiVersion: n9/v1alphakind: Annotationmetadata:  name:  project: my-projectspec:  slo: bad-slo-name  description: test annotation description  startTime: 2021-01-02T17:10:05Z  endTime: 2021-01-02T15:04:05Z07:00

sloctl returns the following error:

'Annotation.spec.description' Error:Field validation for 'description' failed on the 'required' tag

• If the value in the description field is too long (>1000 characters), sloctl returns the following error:

'Annotation.spec.description' Error:Field validation for 'description' failed on the 'max' tag