Good over total anomaly

Reading time: 0 minute(s) (0 words)

The Good over total anomaly applies only to ratio SLO objectives configured with good and total count metrics. It does not apply to objectives configured with bad and total count metrics.

For every calculated interval, the number of good events must be less than or equal to the number of total events:

good <= total

Equality is valid and represents an interval where all events were good. A larger good count means that the two queries or data streams do not describe the same event population for that interval.

Availability

Good over total is being rolled out gradually and may not yet be enabled for your organization. Contact Nobl9 Support for availability details.

How Nobl9 handles invalid counts

Nobl9 checks the good and total counts reported for each interval. If good is greater than total, Nobl9 reports the anomaly and adjusts the interval before calculating reliability. This prevents reliability from exceeding 100%.

Reported counts and adjusted reliability inputAt interval 3, the reported good count exceeds total. Nobl9 reports the anomaly and uses the adjusted good count to calculate reliability.

Reported goodTotalAdjusted good

The SLO chart can therefore look correct, including 100% reliability and a full error budget, while a Good over total annotation identifies invalid source counts that Nobl9 adjusted before calculating the chart. Very small differences caused by number rounding are ignored.

Good catch-up for incremental metrics

Incremental counters can be reported at slightly different times. If total leads good, delayed good events can catch up within the good catch-up window, which is one minute by default. A good excess fully explained by that earlier total lead does not trigger an anomaly. Any unmatched good excess triggers immediately.

Good catches up within one minuteThe cumulative total counter increases by five before the good counter. Good catches up in the following interval, so no Good over total anomaly is created.

Good counterTotal counter

This catch-up handling applies only to incremental count SLOs. For non-incremental count SLOs, an interval where good exceeds total triggers the anomaly.

Detection and resolution

An unmatched good excess triggers the anomaly immediately. While violations continue, Nobl9 keeps one anomaly open. With the default settings, the anomaly resolves after 30 minutes without another unmatched excess.

A good catch-up that is fully explained by a recent total lead does not extend the open anomaly. If 30 minutes have already passed since the last violation, the anomaly can resolve at that point.

What to investigate

Use the annotation's affected period to compare the good and total queries in the data source. Inspect the reported values rather than relying only on the adjusted SLO chart.

Possible cause	How to check it
Different filters or scopes	Confirm that every event counted as good is also included in total
Different grouping dimensions	Apply the same grouping to both queries
Different aggregation windows	Compare both query windows and alignment boundaries
Different units or transformations	Return both values as counts in the same unit
Incorrect count method	Verify whether both source metrics are incremental or non-incremental
Delayed incremental series	Check whether total led first and good caught up within the configured window
Counter reset or transient decrease	Inspect raw values around the decrease and confirm whether both counters reset consistently

Counter decreases and transient glitches

For incremental metrics, Nobl9 treats a decreasing value as a counter reset so later values remain cumulative. A sequence such as value, 0, value can represent a real reset or a temporary source glitch; Nobl9 cannot distinguish those cases from the values alone. When the good counter returns, it can appear to have increased faster than total. Nobl9 reports the anomaly unless a recent total lead fully explains that increase.

After correcting the query or source data, run Replay for the affected period. Replay applies the same checks to historical data. See data anomaly detection after Replay for how repeated Replay results are handled. If historical source data cannot be corrected, separately decide whether a budget adjustment is appropriate; Replay findings do not create budget adjustments automatically.

API access

Query continuously detected and Replay findings with the Anomalies API and anomalyTypes: ["GoodOverTotal"]. Retrieve continuously detected annotations with the Annotations API and category=GoodOverTotalAnomaly. Replay findings are not returned by the Annotations API.