alertescalation.com Add your tool
Guides

Alert deduplication and grouping: dedup keys, grouping and flapping

Learn how dedup keys, grouping, and flap handling reduce noise in PagerDuty, Opsgenie, and Alertmanager with concrete field examples.

Published 14 November 2026.

One outage rarely sends one signal. The same check fires every evaluation, several hosts fail together, and a metric that hovers near its threshold opens and resolves over and over. Three separate mechanisms deal with this. Deduplication merges repeats of the same alert. Grouping bundles different but related alerts into one notification or incident. Flap handling stops one unstable alert from notifying on every change. This guide covers all three without assuming a particular tool, with the exact fields PagerDuty, Opsgenie and Alertmanager use.

What a dedup key is

A dedup key is a string that names one problem. The rules are much the same across tools:

  • The first event with a new key opens an alert.
  • Later events with the same key, while that alert is open, update it instead of opening a second one.
  • Acknowledge and resolve events name the key, so they reach the right alert.
  • Once the alert is resolved, the next event with that key opens a new alert. PagerDuty documents this behaviour (Send an alert event).

The key is chosen by the sender. The on-call tool only compares strings. A bad key cannot be fixed downstream.

How common formats express it

FormatFieldSet byLength limitNotes
PagerDuty Events API v2dedup_keySender; if omitted on a trigger, PagerDuty generates one and returns it255 charactersA trigger without dedup_key always creates a new alert
Opsgenie Alert APIaliasSender512 charactersIdentifies open alerts; a duplicate raises the open alert's Count
Alertmanager (incoming alerts)The alert's label set, shown as fingerprintDerived from labels-Annotations are not part of identity
Alertmanager to PagerDuty or Opsgeniededup_key or aliasAlertmanager: the SHA-256 hash of the group key64 hex charactersOne downstream alert per Alertmanager group

Sources: PagerDuty send alert event, Opsgenie Alert API, Opsgenie alert de-duplication, Alertmanager clients, and the Alertmanager v0.28.1 source (notify/util.go, pagerduty.go, opsgenie.go).

PagerDuty dedup_key

You send the trigger and the matching resolve to https://events.pagerduty.com/v2/enqueue.

{
  "routing_key": "YOUR_INTEGRATION_KEY",
  "event_action": "trigger",
  "dedup_key": "checkout.eu-1.HighErrorRate",
  "payload": {
    "summary": "Checkout 5xx ratio above 5% in eu-1",
    "source": "checkout.eu-1",
    "severity": "critical"
  }
}
{
  "routing_key": "YOUR_INTEGRATION_KEY",
  "event_action": "resolve",
  "dedup_key": "checkout.eu-1.HighErrorRate"
}
  • event_action accepts trigger, acknowledge or resolve. Only trigger creates alerts.
  • Events with the same key apply to the open alert only when they use the same routing_key as the original trigger. An acknowledge or resolve sent with a different routing_key is dropped.
  • An acknowledge or resolve with no open alert does not create one.

The PagerDuty Events API v2 guide covers the rest of the payload.

Opsgenie alias

You create an alert, then close it later using the alias field.

curl -X POST https://api.opsgenie.com/v2/alerts \
  -H "Content-Type: application/json" \
  -H "Authorization: GenieKey $OPSGENIE_API_KEY" \
  -d '{"message": "Checkout 5xx ratio above 5% in eu-1", "alias": "checkout.eu-1.HighErrorRate", "priority": "P1"}'

curl -X POST "https://api.opsgenie.com/v2/alerts/checkout.eu-1.HighErrorRate/close?identifierType=alias" \
  -H "Content-Type: application/json" \
  -H "Authorization: GenieKey $OPSGENIE_API_KEY" \
  -d '{"note": "Error ratio back under 5%"}'
  • Opsgenie allows only one open alert per alias. A subsequent alert with that alias increments the open alert's Count and appends an entry to its activity log.
  • The system stops updating Count and log entries after 100 occurrences. Deduplication remains active while the alert stays open.
  • The EU region instance requires https://api.eu.opsgenie.com.
  • Atlassian stopped selling Opsgenie on June 4, 2025 and ends support on April 5, 2027 (Atlassian). The alias pattern remains relevant for tools that accept Opsgenie-compatible requests.

Alertmanager fingerprint and group key

Alertmanager deduplicates incoming alerts by label set. Each label set has a fingerprint. It then groups alerts per route by the group_by labels. Each group has a group key: the route's matcher path, a colon, and the group's label values.

In a test on v0.28.1 with group_by: ['alertname', 'cluster'], three InstanceDown alerts were posted, two of them with identical labels for web-1:9100. The webhook received one notification with two alerts (trimmed):

{
  "status": "firing",
  "groupKey": "{}:{alertname=\"InstanceDown\", cluster=\"eu-1\"}",
  "groupLabels": {"alertname": "InstanceDown", "cluster": "eu-1"},
  "alerts": [
    {"labels": {"alertname": "InstanceDown", "cluster": "eu-1", "instance": "web-1:9100", "severity": "critical"}, "fingerprint": "89e8128d89a8b13c"},
    {"labels": {"alertname": "InstanceDown", "cluster": "eu-1", "instance": "web-2:9100", "severity": "critical"}, "fingerprint": "bbd7af7ed9ce6969"}
  ]
}

For PagerDuty and Opsgenie, Alertmanager sends the SHA-256 hash of the group key as dedup_key or alias. The source comment says it hashes because integrations may limit key length. Two consequences:

  • PagerDuty sees one alert per Alertmanager group, not per instance. group_by decides the granularity downstream too.
  • Changing group_by or a route's matchers changes the group key. Alerts already open downstream under the old key will not be resolved by the new key, so close them by hand after the change.

See Alertmanager routing, grouping and inhibition for group_by and the timers.

Grouping is a different job

Deduplication merges repeats of one alert. Grouping bundles different alerts that share something, such as a cluster or a service, so a responder gets one notification or one incident.

  • In Alertmanager, grouping happens per route with group_by, before anything is sent.
  • In PagerDuty, Alert Grouping bundles alerts into one incident. Its methods are Intelligent, Alert Content, Intelligent + Alert Content, and Time only. It comes with the AIOps add-on, or as Signal Intelligence in PD Reliability Platform plans (PagerDuty alert grouping).

The sensible default is to do grouping once, as close to the source as you can. Group in Alertmanager and let the on-call tool deduplicate on the key it receives.

Choosing a key

  • Include what identifies the issue: the alert or check name, the affected service or resource, and the environment or region. Example: checkout.eu-1.HighErrorRate.
  • Leave out anything that changes between repeats: timestamps, metric values, counts, request IDs, or message text with numbers in it. HighErrorRate 5.3% at 14:02 opens a new alert on every event.
  • Do not make it too broad. HighErrorRate alone merges eu-1 and us-1 into one alert, and the first resolve closes both.
  • Pick the granularity on purpose. A per-host key gives 40 alerts for 40 failed hosts. A per-service key gives one.
  • Keep it stable for the whole life of the alert, because the resolve must send the same string.
  • Keep it under the limit: 255 characters for PagerDuty, 512 for Opsgenie. Hash long keys, as Alertmanager does.
  • Always send one. A PagerDuty trigger without dedup_key always creates a new alert.

Flapping and hysteresis

A flapping alert crosses its threshold back and forth. It fires, resolves and fires again. Each change notifies someone. Four fixes handle this, from simplest to most complex.

  • Wait before firing. Prometheus for keeps an alert pending until the condition has held for the whole duration. Grafana calls this the pending period.
  • Wait before resolving. Prometheus keep_firing_for keeps the alert firing for a duration after the condition was last met. It was added in Prometheus 2.42.0 (alerting rules). Grafana has Keep firing for, with a Recovering state in between.
  • Hysteresis. Use a lower threshold to resolve than to fire. Grafana's recovery threshold example: fire at 1000ms, resolve only below 900ms. A value that moves 980, 1010, 990, 1020 fires once and stays firing (Grafana queries and conditions).
  • Flap detection. Nagios keeps the last 21 check results. It computes a weighted percent state change. It marks a check as flapping above a high threshold and as stable below a low one. It suppresses notifications while it flaps (Nagios flapping).

This Prometheus rule uses both waits. It passes promtool check rules on Prometheus v3.5.0:

groups:
  - name: checkout
    rules:
      - alert: HighErrorRate
        expr: |
          sum by (service, cluster) (rate(http_requests_total{service="checkout", code=~"5.."}[5m]))
            /
          sum by (service, cluster) (rate(http_requests_total{service="checkout"}[5m]))
            > 0.05
        for: 5m
        keep_firing_for: 10m
        labels:
          severity: critical
          team: payments
        annotations:
          summary: "Checkout 5xx ratio above 5% in {{ $labels.cluster }}"

The sum by (service, cluster) also sets the alert's identity, so there is one alert per service and cluster, not one per pod. The cost of both waits is time. The alert fires up to 5 minutes later and resolves up to 10 minutes later. See alert fatigue for other noise fixes and the alert lifecycle for the states an alert moves through. Tools that deduplicate and group are in the alert routing directory.