alertescalation.com Add your tool
Guides

How to test an escalation policy before a real outage

How to test an escalation policy before a real outage. Check contacts, send a test alert, and walk every level to verify your on-call setup.

Published 14 November 2026.

An escalation policy encodes assumptions. It assumes the integration key works, the schedule has the right person, their phone rings through do-not-disturb, and level 2 fires after the configured wait. A real outage is the worst time to discover one of these is wrong. A test checks every assumption on purpose while everyone involved expects it. This guide covers contact methods, an end-to-end test alert, walking each level, time zone and handover checks, and a quarterly drill. For policy design, see how to design an escalation policy.

When to test

  • After any change to the escalation policy, the schedule, the alert routing or an integration key.
  • Before a new person's first shift, with that person at level 1.
  • After someone changes phone, or reinstalls the on-call app.
  • Once a quarter as a full drill, even if nothing changed.

Tell everyone on the policy before you start. The PagerDuty incident response guide calls this "taking the pager": tell the person on call that you are taking the pager while you test (PagerDuty, Being On-Call).

Step 1: check contact methods

Test each person's contact methods before testing the policy.

  • PagerDuty. Each contact method in a user's profile has a Test button. Test notifications for phone and SMS are not available in all countries. SMS needs a verified number, and users created through the REST API, for example with Terraform, must verify their phone number in the web app (PagerDuty, Contact Information). On some plans, On-Call Readiness Reports check every responder's notification rules against a chosen profile (PagerDuty, On-Call Readiness Reports).
  • Opsgenie. Since July 2024, adding an SMS or voice contact method no longer creates a default alert notification rule, so check the rules after adding a number. To test, create an alert from the Alerts page with yourself as the only responder (Opsgenie, How to get your first alert notification).
  • Do-not-disturb. The PagerDuty guide asks responders to make sure texts and calls from the tool can bypass their phone's do-not-disturb settings. Test with the phone in its normal night-time state.

Step 2: send a test alert end to end

Trigger the test from the alert source. Use the same integration that production traffic uses. Creating an alert manually in the on-call tool skips the integration key and the routing logic. A manually created incident in PagerDuty assigned to yourself does not notify you and starts as acknowledged (PagerDuty, Incidents).

For a service using the PagerDuty Events API v2, send a trigger event to the enqueue endpoint (PagerDuty, Send an event). You must provide routing_key, event_action, payload.summary, payload.source and payload.severity. Severity is one of critical, error, warning or info; use the severity your real urgent alerts use. Set your own dedup_key so you can resolve the test later.

curl -sS -X POST https://events.pagerduty.com/v2/enqueue \
  -H 'Content-Type: application/json' \
  -d '{
    "routing_key": "YOUR_INTEGRATION_KEY",
    "event_action": "trigger",
    "dedup_key": "escalation-drill-2026-11-14",
    "payload": {
      "summary": "TEST escalation drill, no action needed",
      "source": "escalation-drill",
      "severity": "critical"
    }
  }'

A 202 response with "status": "success" and "message": "Event processed" means PagerDuty accepted the event. It does not mean anyone was notified; that is what the next step checks. To end the test from the source side, send a resolve with the same dedup_key:

curl -sS -X POST https://events.pagerduty.com/v2/enqueue \
  -H 'Content-Type: application/json' \
  -d '{"routing_key": "YOUR_INTEGRATION_KEY", "event_action": "resolve", "dedup_key": "escalation-drill-2026-11-14"}'

If Prometheus Alertmanager routes your alerts, first check the routing offline with amtool config routes test, then inject a test alert with amtool alert add (Alertmanager README). See Alertmanager routing for the route tree.

amtool config routes test --config.file=alertmanager.yml --verify.receivers=team-X-pager service=checkout severity=critical

amtool alert add alertname=EscalationDrill service=checkout severity=critical \
  --annotation=summary='TEST escalation drill, no action needed' \
  --end=2026-11-14T15:00:00Z \
  --alertmanager.url=http://localhost:9093

Set --end past the time the whole walk will take. An alert without an end time is declared resolved after resolve_timeout, 5 minutes by default (Alertmanager configuration). The pagerduty_configs and opsgenie_configs receivers send resolved notifications by default, so the drill would end itself before level 2. The first notification also waits for group_wait, 30 seconds by default.

Check at level 1: the right person was notified, on the right channel, and the message says what broke and links to the runbook. See alert message design.

Step 3: walk every level

Do not acknowledge the test alert. Let it run through every escalation level and time each step.

  • PagerDuty. The default escalation timeout is 30 minutes. The minimum is 1 minute with one target on a rule and 3 minutes with more than one. The first rule is always notified the moment the incident triggers. A policy can repeat up to nine times (PagerDuty, Escalation Policy Basics).
  • Opsgenie. Opening the alert details in the web or mobile app stops further notifications to you for that alert. With a rule that sends an SMS after 2 minutes, opening the alert within 2 minutes means the SMS is never sent (Opsgenie, How to get your first alert notification). The person at level 1 should not open the alert during the drill. An escalation can repeat at most 20 times (Opsgenie, How do escalations work).
  • Grafana IRM. "Skip to next escalation" runs the next notifying step at once (Grafana, Respond to alerts). It is useful for a quick check after a change, but it skips the waits, so the quarterly drill should run in real time.

At the last level, acknowledge the alert, then send the resolve from the source. Check that the incident closes and that everyone notified sees it resolved. If the service has a PagerDuty acknowledgement timeout, which is off by default, also check that an acknowledged test incident re-triggers when it expires (PagerDuty, Configurable Service Settings).

Step 4: check time zones and handover boundaries

  • Which time zone the schedule runs in. PagerDuty sets the time zone for each rotation in a shift-based schedule. The time zone menu at the top of the schedule page only changes the display. Users always see schedules in their own time zone. Daylight saving changes can move a handover in one person's view and not another's (PagerDuty, Time Zone Settings).
  • Handover time. Look up who is on call one minute before and one minute after the handover. Send a test alert a few minutes after the handover and confirm the incoming person gets it.
  • Overrides. During an active override, confirm the person covering is notified, and the person they replaced is not. See on-call overrides and shift swaps.
  • Follow the sun. Test at each regional handover, and in the weeks when any team's clocks change.

Quarterly drill checklist

  • Everyone on the policy knows the drill time.
  • Each person's contact methods pass a test notification.
  • The test alert goes through the real integration and routing.
  • Level 1 is notified within the first minute, on the expected channel.
  • Each later level is notified at its configured wait, within a minute.
  • The last level acknowledges, and escalation stops.
  • A resolve from the source closes the incident.
  • A test at the handover time reaches the incoming person.
  • Every failure becomes a ticket with an owner before the drill ends.

What to record

Store a single record for each drill alongside the policy document. Review it against previous entries to track changes.

Escalation drill: <policy name>   Date: <date> <time zone>
Run by: <name>   Alert source and integration: <name>

Level  Target             Expected at  Notified at  Channel     Pass
1      <schedule>         T+0          <time>       <push/SMS>  <yes/no>
2      <schedule>         T+<wait>     <time>       <channel>   <yes/no>
3      <person or role>   T+<wait>     <time>       <channel>   <yes/no>

Acknowledged by: <name> at <time>
Resolved from source at: <time>   Incident closed: <yes/no>
Handover test: <who was notified> at <time>   Expected: <name>
Failures and tickets: <links>

The on-call directory lists tools that run escalation policies.