How to Design an On-Call Escalation Policy That Works
Design an on-call escalation policy that reaches a human fast: tier design, timeouts by severity, channels, and the failures that swallow pages.

An on-call escalation policy is the part of your incident response that runs when nobody is looking. It fires at 3:14am, decides who gets woken up, decides how long to wait before waking up somebody else, and decides when to stop trying. Most teams write one in an afternoon, wire it to a rotation, and never touch it again until an incident goes badly and the retro asks the uncomfortable question: why did it take forty minutes for a human to acknowledge?
This guide is about designing an escalation policy that actually reaches a responder quickly, without burning your team down in the process. It covers tier design, timeout selection, notification channels and their quiet failure modes, how the policy should change with team shape, and how to test the thing before an incident tests it for you. Everything here is vendor neutral and applies whether you run a homegrown paging script or a full incident platform.
What an on-call escalation policy actually does
Strip away the tooling and an escalation policy is a small state machine with one job: convert an alert into an acknowledged alert, held by a named human, in bounded time. That is it. Everything else, the severity taxonomy, the runbook links, the incident channel automation, happens after acknowledgement. If the escalation policy fails, none of the rest of your incident response exists.
That framing is useful because it clarifies what to optimise for. You are not optimising for fairness, or for tidy org charts, or for making sure the right specialist sees the alert first. You are optimising for time to acknowledgement, subject to the constraint that you do not wake people up unnecessarily. Those two goals pull against each other, and every design decision below is a negotiation between them.
Acknowledgement is not resolution
A common design error is treating the escalation policy as though it should keep escalating until the problem is fixed. It should not. The policy's contract ends when a competent human has the alert and has said so. What happens next belongs to the incident process: severity assessment, comms, subject matter expert pull-in, status page updates. Conflating the two produces policies that page the VP of Engineering because a database migration took longer than the escalation timer allowed.
Keep the two separate. The escalation policy answers "who is awake and looking at this". The incident process answers "who else do we need and what do we tell people". Mixing them is how you end up with executives receiving phone calls for alerts that a responder already had well in hand.
The four questions every escalation policy answers
Before you configure anything, write down the answers to four questions for each service or alert class you own. If you cannot answer them in plain language, the tool configuration will just encode your confusion.
- Who is first? Which rotation owns the first notification, and is that rotation staffed right now, including holidays and the gap between shifts?
- How long do we wait? What is the acknowledgement timeout before we escalate, and does that number differ by severity?
- Who is next, and why them? Is the second tier a peer on the same rotation, a different team, or a manager? What can they actually do that the first tier could not?
- Where does it stop? What is the terminal state when nobody acknowledges? Silence is not an acceptable answer, and neither is an infinite loop.
The fourth question is the one teams skip most often, and it is the one that produces the worst outcomes. An escalation policy with no terminal tier does not fail loudly. It fails quietly, cycling through the same two unreachable people while your error rate climbs.
Designing the tiers: primary, secondary, and beyond
Three tiers is the right default for most teams. Fewer than three and you have no depth when someone's phone is in a tunnel. More than three and the later tiers are decorative, because by the time you have burned through four timeouts the incident has either resolved itself or somebody noticed it another way.
Tier one: the primary responder
The primary is the person on rotation for the owning team. The design goal here is unambiguity. Exactly one human should be the primary at any instant. Shared primaries, where an alert goes to a channel or a group and everyone assumes someone else has it, are the single most reliable way to lose ten minutes at the start of an incident. Diffusion of responsibility is a well documented effect, and paging a group of five is a good way to page nobody.
If your tooling lets you page a Slack user group, resist the temptation to make that the first tier. Use the group for visibility and the individual for responsibility. A useful pattern is to notify the individual directly and simultaneously post to the team channel, so the team can see an alert is live and who owns it, without anyone assuming the mention was for them.
Tier two: the secondary responder
The secondary exists for one reason: the primary did not respond. The most useful secondary is a peer with the same or similar context, drawn from the same rotation but offset, so the two are never the same person. Some teams run a formal secondary rotation. Others use "the person who was primary last week", which has the nice property of guaranteeing recent context and requiring no extra scheduling.
What the secondary is not is a specialist. Do not design tier two as "the database expert" or "whoever wrote this service". Specialist pull-in is an incident activity, done deliberately by an engaged responder who has assessed the problem. Encoding it into the escalation chain means you page specialists for alerts that turn out to be a full disk.
Tier three: the terminal tier
Tier three should be a person with the authority to make noise, usually an engineering manager or a duty lead. Their job is not to debug. Their job is to notice that the automated process failed and to start finding humans by other means. That means a phone call, not a push notification, and it means the tier three responder needs to understand that being paged is itself the signal: two people already did not answer.
Give tier three an explicit playbook that is about finding people, not about fixing systems. Who has a personal phone number for the primary? Is there an adjacent team that can hold the fort? Is there a documented "break glass" contact for the service? Writing this down is a fifteen minute exercise that pays for itself the first time it runs.
Choosing timeouts that match your severity levels
Timeout selection is where most escalation policies go wrong, usually in the direction of being far too generous. A fifteen minute acknowledgement window sounds reasonable in a planning meeting. In practice it means that if the primary's phone is on silent, your severity one incident gets its first human at minute fifteen, its second at minute thirty, and a manager at minute forty five. That is an unacceptable outcome for anything customer facing.
A better approach is to derive timeouts from what you have already promised. If your service level objective allows twenty minutes of error budget burn before you breach, your entire escalation chain has to complete inside a fraction of that. Work backwards.
Reasonable starting points
- Severity one, customer facing outage: five minutes to first escalation, five minutes to second, then terminal. Full chain exhausted in ten minutes.
- Severity two, degradation with workaround: ten minutes per tier. Full chain in twenty minutes.
- Severity three, internal or deferred: do not page at all outside business hours. Route to a queue or a channel and let the next working day handle it.
The third line is the most important one and the one teams find hardest to commit to. If an alert does not warrant waking someone up, it should not be able to wake someone up, ever, under any escalation path. The moment a severity three alert can reach a phone at 4am, your team learns to distrust the whole system, and distrust is what produces slow acknowledgements in the first place.
Timeouts and human reality
Five minutes assumes a responder who is asleep, wakes to a loud phone, and taps acknowledge before they have any idea what is happening. That is fine. Acknowledgement should be cheap and fast, a signal that a human is now conscious and holding the alert, not a claim that they understand it. If your tooling makes acknowledgement expensive, for example by requiring a laptop and a VPN, your timeouts will always be wrong because you are measuring the wrong thing.
Make acknowledgement possible from a phone in one tap. In a Slack-native setup like Pagerly, acknowledging from the mobile Slack notification is the whole interaction, which means the five minute timeout is realistic rather than aspirational.
Notification channels and the quiet failure modes
A tier is only as good as the channel that reaches it. Every channel has a failure mode, and the failure modes are not independent, which is the part that catches teams out.
- Push notifications fail on do not disturb, on low power mode, on flaky mobile data, and when the operating system decides your app has been too chatty. They are the most convenient channel and the least reliable one.
- SMS fails on carrier delays, which can be minutes and are invisible to you. It also fails silently in the sense that delivery receipts frequently lie.
- Voice calls are the most reliable waking mechanism and the most disruptive. Reserve them for later tiers or for severity one.
- Email is not a paging channel. It is a record. Do not put it in an escalation tier.
- Chat mentions are excellent for visibility and coordination and unreliable for waking people, because chat apps are exactly the apps people mute at night.
The correlated failure that matters most: if your paging tool routes everything through a single vendor's push infrastructure, and that vendor has an incident, every tier of your policy fails at once. This is not hypothetical. Build at least one tier that uses a structurally different path, typically a voice call through a telephony provider, so that a single vendor problem does not silence the entire chain.
Escalate across channels, not just across people
A refinement that costs nothing and helps a lot: make each tier escalate the channel as well as the person. Tier one gets a push notification. If that times out, tier one gets a phone call at the same time tier two gets a push. You have now covered the most common single failure, which is not an absent responder but a phone that did not ring.
Escalation policies for different team shapes
The three tier default assumes a team with enough people to staff a rotation. Plenty of teams are not in that position, and copying a large company's policy into a five person startup produces something that looks rigorous and works badly.
The very small team
With four or fewer engineers, a formal secondary rotation is theatre. Everyone is always nearly on call. What actually helps is ruthless alert reduction, so the small number of pages that do fire are unambiguously worth waking for, plus a terminal tier that reaches the founder or technical lead by phone. Accept that your depth is shallow and compensate by making the surface area small.
The follow the sun team
If you have coverage in multiple regions, your escalation policy should prefer the region that is awake before it prefers the team that owns the service. A primary in a business hours timezone acknowledging in ninety seconds is worth more than the service owner acknowledging in eleven minutes from a dead sleep. Design tier one around the clock and tier two around ownership.
The platform team with many consumers
Platform teams get paged for problems that belong to their consumers. The escalation policy should reflect that: tier one is the platform on-call, and the policy should make it trivially easy to hand the alert to the owning service team rather than escalating within the platform. If your tooling supports reassignment, that is often more valuable than another tier.
Testing your escalation policy before an incident tests it
An untested escalation policy is a guess. The good news is that testing one is cheap, and the exercise reliably finds problems.
- Run a monthly synthetic page. Trigger a test alert at a random time during the on-call week and measure how long each tier took. Do not warn people it is coming, but do make the test alert obviously labelled as a test.
- Audit the roster quarterly. People leave, change phone numbers, and change chat accounts. Every escalation policy accumulates dead contacts. A quarterly pass that verifies every contact method in every tier takes an hour.
- Test the handoff boundary. The riskiest moment in any rotation is the shift change. Fire a test alert deliberately in the handover window and confirm it reaches the incoming person, not the outgoing one.
- Test the terminal tier. Once or twice a year, let a test alert run the full chain to exhaustion. This is the only way to discover that tier three's phone number has been wrong for eight months.
- Verify holiday coverage. Check the rotation against the actual calendar before every public holiday and every company shutdown week.
Record the results somewhere durable. Time to acknowledgement per tier, over time, is one of the few incident metrics that is both easy to collect and genuinely actionable.
Metrics that tell you the policy is working
Most incident dashboards over-index on mean time to resolution, which is dominated by the difficulty of the underlying problem and therefore says very little about your process. The escalation policy has its own metrics, and they are much cleaner.
- Time to acknowledgement, at the ninety fifth percentile. The mean hides exactly the cases you care about. If p95 acknowledgement is twenty minutes, you have a paging problem regardless of what the average says.
- Escalation rate. What fraction of pages get past tier one? Anything above roughly ten percent means your first tier is not reliably reachable, or your timeouts are too tight, or your rotation has gaps.
- Terminal tier activations. This should be close to zero. Every activation deserves an individual look.
- Out of hours page volume per person. The health metric. A policy that acknowledges fast because everyone is permanently anxious is not a good policy.
- Actionability rate. What fraction of pages resulted in a human doing something? If it is below half, fix the alerts before you touch the escalation policy.
That last point deserves emphasis. Escalation policy design has a hard ceiling set by alert quality. If your team receives thirty pages a week and twenty five are noise, no timeout configuration will produce fast acknowledgement, because people learn, correctly, that the page probably does not matter. Google's SRE material makes this argument at length and it remains the highest leverage change available to most teams. Fix the alerts first, then tune the policy.
Common escalation policy mistakes
- Paging a group as tier one. Produces diffusion of responsibility and slow acknowledgement. Page an individual, notify the group.
- Timeouts inherited from a template. Fifteen minutes is a default, not a decision. Derive yours from your objectives.
- No terminal tier. The chain must end somewhere loud.
- Specialists in the chain. Pull specialists in deliberately during the incident, not automatically before anyone has looked.
- Managers as tier two. A manager two minutes after the primary adds pressure without adding capability. Managers belong at the terminal tier.
- Single vendor for every channel. One vendor incident silences every tier. Diversify at least one path.
- Severity blind escalation. One policy for all alerts means either your low severity alerts wake people or your high severity alerts move too slowly.
- Policies that are never tested. An escalation policy that has not fired in anger and has not been tested is a hypothesis, not a control.
A worked example
Consider a twelve person product engineering group split into two service teams, with a shared weekly rotation per team and business hours coverage in one timezone. Here is a policy that works for that shape.
Alerts carry a severity label set at the alert definition, not at page time. Severity one covers customer facing unavailability and payment failures. Severity two covers elevated error rates with a working fallback. Severity three covers everything else and is explicitly non-paging.
For severity one, tier one notifies the team's current primary by push and, simultaneously, posts to the team incident channel with the primary's name attached so the rest of the team can see who owns it. After four minutes without acknowledgement, the primary receives a voice call and the secondary receives a push. After a further four minutes, the duty engineering manager receives a voice call and the incident channel receives an unacknowledged alert notice.
For severity two, the same chain runs with ten minute intervals and no voice call until the terminal tier. For severity three, the alert posts to a triage channel and appears in the next standup. It cannot page anyone, and no configuration change short of relabelling the alert can make it page anyone.
Three operational rules complete the design. First, handover happens at a fixed time with an explicit acknowledgement from the incoming primary, and alerts fired within thirty minutes either side go to both. Second, any severity one page that reaches the terminal tier generates a mandatory review item, separate from the incident's own retro. Third, alert definitions are reviewed monthly, and any alert with an actionability rate below fifty percent over the quarter is either fixed, downgraded, or deleted.
None of this is exotic. What makes it work is that every element is a decision somebody made on purpose, rather than a default that survived because nobody looked at it.
Where the tooling helps, and where it does not
Tooling cannot fix an escalation policy that has not been thought through, but it can remove a lot of friction from the thinking. The properties worth caring about are: can a responder acknowledge in one tap from a phone, can you see the current on-call for any team without asking, does the rotation stay in sync with wherever your team actually coordinates, and can you diverge channels across tiers.
The Slack-native approach that Pagerly takes is useful here for a specific reason. The page, the acknowledgement, the incident channel, and the handover all live in the place the team is already looking, which collapses the gap between "alert fired" and "team is coordinating". It also makes the visibility half of the tier one design free: the individual gets paged, the channel sees it, and nobody has to context switch into a separate tool to find out who is holding the alert. Overrides, swaps, and holiday coverage happen in the same place, which matters because rotation gaps are one of the main causes of escalation in practice.
That said, the diversification point stands regardless of tooling. Whatever your primary path is, make sure at least one tier reaches people by a structurally different route.
Practical takeaways
- Treat the escalation policy as a machine for producing acknowledgement in bounded time, and keep it separate from your incident process.
- Default to three tiers: an individual primary, a peer secondary, and a terminal tier with authority and a find-the-humans playbook.
- Derive timeouts from your service objectives rather than copying a template, and differentiate them by severity.
- Make sure severity three alerts are structurally incapable of paging anyone out of hours.
- Escalate the channel as well as the person, and keep at least one tier on a structurally different delivery path.
- Test with synthetic pages monthly, audit contacts quarterly, and exhaust the full chain at least once a year.
- Watch p95 time to acknowledgement, escalation rate, terminal activations, and out of hours volume per person.
- Fix alert quality before tuning timeouts, because actionability sets the ceiling on everything else.
Closing thought
The best on-call escalation policy is one your team has forgotten about, because it has never surprised them. It reaches the right person quickly, it never wakes anyone for something that could have waited, and when it does escalate, everyone involved understands exactly why. Getting there is not a tooling problem so much as a decision-making problem: four questions per service, timeouts derived from real objectives, and a test schedule that keeps the whole thing honest.
Spend an afternoon on it this quarter. Write down the answers to the four questions for your highest severity alerts, check the contact details in your terminal tier, and fire one synthetic page. If any of those three exercises surprises you, you have just found the thing that would otherwise have surprised you during an outage instead. Useful background reading on the underlying practice is available in Google's SRE material on being on call and on incident response.
