Silences and maintenance windows: muting alerts without missing outages

Planned work and known problems produce alerts that require no action. A deploy restarts pods. A database failover test drops connections. A disk swap takes a host offline. Every alerting tool has ways to mute them. The risk is a mute that covers more than the work, or lasts longer than the work, and hides a real outage. This guide compares the four muting mechanisms, shows the Alertmanager commands, and ends with a checklist.

Four ways to mute

Mechanism	Started by	Scope	Ends	Example
Silence	A person or a script, when needed	Label matchers	At its end time, or when expired by hand	Alertmanager silence
Maintenance window	A schedule, one-off or recurring	A service, or a route	When the window closes	PagerDuty maintenance window, Alertmanager mute_time_intervals
Inhibition	Another alert firing	A rule in the config	When the source alert resolves	Alertmanager inhibit_rules
Snooze	A responder, on one incident	One acknowledged incident	When the timer runs out, the incident triggers again	PagerDuty snooze

All four stop notifications. In Alertmanager, muted alerts remain received, evaluated, and visible as suppressed. The silencedBy field holds the silence ID, while inhibitedBy holds the source alert. Inhibition is covered in Alertmanager routing, grouping and inhibition.

Alertmanager silences

A silence mutes alerts for a set time. It uses matchers like the routing tree. An alert is muted when it matches all matchers of an active silence (concepts).

• Silences are created in the web UI, with amtool or through the API, not in alertmanager.yml.
• In a cluster, silences are replicated between peers.
• A silence is pending before its start time, active during it, and expired after it. Expired silences are kept for the --data.retention period, 120h by default.
• A silence applies to any matching alert while it is active, including alerts that start after it was created.

amtool silence commands

These commands were run against Alertmanager v0.28.1. Flags match amtool silence add --help (amtool README).

# See what a silence would cover before you create it.
amtool alert query service="checkout" cluster="eu-1"

# Silence for 2 hours. Prints the new silence ID.
amtool silence add --author="[email protected]" --duration=2h \
  --comment="Disk replacement on db-3, CHG-1234" \
  alertname="DiskFilling" cluster="us-1"

# Silence a fixed window in the future.
amtool silence add --author="[email protected]" \
  --start=2026-11-15T01:00:00Z --end=2026-11-15T03:00:00Z \
  --comment="DB failover test, CHG-1240" \
  'service=~"orders|billing"' cluster="eu-1"

# List active silences, and those ending in the next 3 hours.
amtool silence query
amtool silence query --within=3h

# Shorten or extend a silence.
amtool silence update --duration=30m aa3e3b7b-a726-4c30-b94b-be2bab9aa1b1

# Expire one silence, or every silence that matches.
amtool silence expire aa3e3b7b-a726-4c30-b94b-be2bab9aa1b1
amtool silence expire $(amtool silence query -q cluster="us-1")

• amtool silence add requires --comment. The error is comment required by config (silence_add.go).
• --duration defaults to 1h. --end overrides --duration. Times are RFC3339.
• --author defaults to the local username.
• amtool silence query --expired lists expired silences.
• Point amtool at the server with --alertmanager.url, or set alertmanager.url, author and comment_required in $HOME/.config/amtool/config.yml or /etc/amtool/config.yml.

Scope a silence by labels

• Match the narrowest set of labels that covers the work: the service, the instance or host, and the environment or cluster.
• A silence on alertname alone mutes that alert everywhere, in every cluster.
• A broad regex such as cluster=~".+" mutes everything. Use it only if the whole estate really is in maintenance.
• During a deploy, a sensible default is to mute severity="warning" for the service being deployed and leave critical alerts live.
• Run amtool alert query with the same matchers first. It lists the firing alerts the silence would cover.
• A silenced alert can still inhibit others. In a test on v0.28.1, silencing a critical HighErrorRate alert left the matching warning inhibited by it, so the silence hid both. Check inhibit rules before silencing a source alert.

Expiry and forgotten silences

Every silence has an end time. Set it to the expected length of the work plus a margin, not to a day or a week. Expire the silence when the work ends instead of waiting for the timer. When a silence expires, matching alerts that are still firing notify again.

• amtool silence query --within=3h shows what ends soon, so a long job can be extended before it is noisy.
• Review active silences at each on-call handover. A silence nobody can explain should be expired.
• A silence that keeps being renewed is a disabled alert. Fix or delete the rule instead; see alert fatigue.

Deploy-time muting

A deploy script can create a silence, keep its ID, and expire it on exit, even when the deploy fails. Tested with sh and Alertmanager v0.28.1:

#!/bin/sh
set -eu
AM_URL=http://alertmanager.internal:9093
SILENCE_ID=$(amtool --alertmanager.url="$AM_URL" silence add \
  --author="deploy-bot" \
  --duration=20m \
  --comment="Deploy checkout v2.14.0, pipeline 5821" \
  service="checkout" cluster="eu-1" severity="warning")
trap 'amtool --alertmanager.url="$AM_URL" silence expire "$SILENCE_ID"' EXIT

./deploy.sh checkout v2.14.0

• The --duration is a backstop. If the script is killed hard and the trap never runs, the silence still ends after 20 minutes.
• The comment names the version and the pipeline run, so anyone looking at the silence knows why it exists.
• The same works over HTTP: POST /api/v2/silences returns a silenceID, and DELETE /api/v2/silence/{id} expires it.

Recurring maintenance windows

For work on a fixed schedule, Alertmanager has named time intervals that a route can reference. This passes amtool check-config on v0.28.1:

time_intervals:
  - name: sunday-db-maintenance
    time_intervals:
      - weekdays: ['sunday']
        times:
          - start_time: '02:00'
            end_time: '04:00'
        location: 'Europe/London'

route:
  receiver: platform-slack
  group_by: ['alertname', 'cluster', 'service']
  routes:
    - matchers:
        - service="postgres"
        - severity="warning"
      receiver: platform-slack
      mute_time_intervals:
        - sunday-db-maintenance

receivers:
  - name: platform-slack
    slack_configs:
      - api_url_file: /etc/alertmanager/secrets/slack-webhook-url
        channel: '#platform-alerts'

• Times are UTC unless location names an IANA time zone. The start time is included and the end time is not.
• The root route cannot have mute times. mute_time_intervals and active_time_intervals are not inherited by child routes.
• A muted route sends nothing but otherwise acts normally. It still ends route matching unless continue is set.

Source: Alertmanager configuration.

PagerDuty maintenance windows work per service (PagerDuty maintenance windows):

• A window disables a service and all its integrations for a set time, so no new incidents trigger.
• Existing open incidents do not resolve when the window begins. They stay open and keep notifying until someone resolves them.
• Incidents cannot be resolved through the Events API while the service is in a window.
• A window can start now or at a later date and time. Disabling the service is the indefinite version.

Snooze

Use PagerDuty snooze to pause escalation for an incident you have already acknowledged. You cannot snooze incidents that remain in a triggered state. The "Other" option supports a maximum duration of 168 hours. When the timer expires, the incident returns to triggered status and notifies responders again if it is still unresolved (PagerDuty edit incidents). Set the snooze duration to match the expected fix time. Do not use snooze to push work to another shift.

Checklist

Before:

• The silence or window names the service and the environment, not only an alert name.
• amtool alert query with the same matchers shows only the alerts you expect.
• Critical alerts outside the work stay live.
• Inhibit rules that use the muted alerts as a source are checked.
• The end time is the length of the work plus a margin.
• The comment has an author and a ticket or change ID.
• If customers will notice the work, post it on the status page; see status page updates.

After:

• Expire the silence as soon as the work ends.
• Check that alerts which fired during the work have resolved.
• Run amtool silence query at handover and expire anything nobody can explain.

Tools that support silences and maintenance windows are listed in the alert routing directory.