What the July 2026 AWS Outage Teaches On-Call Teams

Category
Falit Jain
July 27, 2026
5 min read
What the July 2026 AWS Outage Teaches On-Call Teams
Table of Content

On July 24, 2026, another AWS outage reminded the internet how fragile the shared cloud can be. For roughly 80 minutes, AWS lost connectivity between its us-west-2 region in Oregon and the Seattle metro, and the blast radius was enormous. Apple Pay, Reddit, Hulu, DoorDash, and the PlayStation Network all stopped responding at once, and incident trackers lit up with reports from teams scrambling to figure out whether the problem was theirs or Amazon's. It was the third notable AWS incident in three months, and for on-call engineers it was a live fire drill in how to respond when the platform underneath you goes dark.

This post breaks down what actually happened, why a region-boundary failure behaves so differently from the outages most runbooks are written for, and what on-call and incident-management teams should change before the next one. The payoff is practical: a clearer mental model of dependency failures, a set of concrete on-call habits, and a realistic view of what tooling can and cannot do when a hyperscaler stumbles.

What happened in the July 2026 AWS outage

According to the timeline reconstructed by IncidentHub, the disruption began when AWS lost connectivity between the us-west-2 (Oregon) region and the Seattle metro area. Amazon's own status messaging during the event was telling: it stated that "connectivity within the region was not affected by this event." In other words, compute and storage inside Oregon kept humming. The failure lived at the boundary between AWS's Oregon data centers and the public internet.

That distinction matters more than it sounds. Any request that both started and ended inside the region continued to work. Anything that had to cross the region boundary, which includes most user-facing traffic and, for some customers, even the AWS Management Console, saw timeouts and errors. Most customers felt about 20 minutes of pain. A smaller group connecting through AWS Direct Connect at the Westin Building Exchange in Seattle experienced a longer window of roughly 1 hour and 17 minutes, with a recovery tail that dragged on well after Amazon posted its resolution notice.

Amazon has not fully specified whether the fault originated inside its own network, in a peering relationship with an upstream transit provider, or somewhere further out on the public internet. What is documented is the shape of the failure: networking hardware that carried routing between the region and the Seattle metro went sideways, and the internal systems stayed healthy while the doors to the outside world jammed shut.

The cascade was the real story

IncidentHub recorded nine confirmed cascade incidents across seven providers, plus three more possible cascade events at three additional providers. This is the part that should keep platform teams up at night. One networking fault at one cloud boundary rippled into payment processing, food delivery, streaming, gaming, and social platforms. The PlayStation Network's recovery notably lagged behind AWS's own resolution, with intermittent problems persisting for hours after the underlying networking issue was fixed. That gap between "upstream is healthy" and "we are healthy" is where a lot of avoidable customer pain lives.

Why region-boundary failures break most runbooks

Most incident runbooks are written for failures you can see and touch inside your own stack. A pod is crash-looping, a database is at connection saturation, a bad deploy needs a rollback, a certificate expired. These are the bread and butter of on-call work, and good teams get fast at them. The July 24 AWS outage was a different animal, and it exposes three assumptions that quietly live inside a lot of runbooks.

Assumption one: the problem is inside your walls

When alerts fire, the instinct is to look at your own recent changes. Did we ship something? Did a config drift? That instinct is usually correct, which is exactly why it becomes dangerous during a dependency outage. Engineers can burn the critical first 20 minutes rolling back healthy deploys and restarting healthy services because the symptoms (timeouts, elevated error rates, failed health checks) look identical to a self-inflicted incident. A region-boundary failure produces the same red dashboards as a bad release, but the fix is entirely outside your control.

Assumption two: your monitoring can see the truth

During the July outage, some customers could not even reach the AWS Management Console. If your observability stack, your alerting pipeline, or your status page depends on the same region that is failing, you can go blind at the exact moment you most need to see. Teams that route all telemetry through a single region learned this the hard way. Your ability to diagnose an outage should never share fate with the thing that is down.

Assumption three: recovery is instant once upstream recovers

The PSN recovery tail is the lesson here. When a major provider comes back, your systems do not automatically snap back to healthy. Connection pools hold stale entries, queues have backed up, caches are cold, retries have created a thundering herd, and circuit breakers may still be open. Declaring "AWS is back, we are fine" too early is how teams turn an 80-minute vendor incident into a multi-hour customer incident. Recovery needs its own checklist, not just an all-clear from the provider status page.

What on-call teams should do differently

You cannot prevent a hyperscaler from having a bad day. You can control how quickly your team orients, communicates, and recovers. Here is what separates the teams that handled July 24 gracefully from the ones that flailed.

Confirm scope before you touch anything

The single highest-leverage habit during a suspected dependency outage is to answer one question first: is this us, or is this them? Before rolling back a deploy or paging the database owner, check the provider status page, check third-party outage trackers, and check whether the failure pattern matches a boundary problem (external requests failing while internal traffic is healthy). Those signals were all available within minutes on July 24. A team that confirms "this is an AWS outage, not our deploy" in the first five minutes saves itself from a cascade of pointless remediation.

Have a dependency map you actually trust

When AWS us-west-2 wobbles, the responder needs to know instantly which of your services live there, which third parties you rely on live there, and what the blast radius is for each. Too many teams reconstruct this knowledge live, during the incident, by asking around. A current dependency map, including the region each critical service runs in and which external vendors sit on your critical path, turns a panicked archaeology session into a 30-second lookup.

Keep incident coordination out of the failing region

Your incident response process should not depend on the infrastructure that is on fire. That means your paging and escalation, your incident channel, and your status communications need to run somewhere independent. This is a strong argument for coordinating incidents in a tool that lives where your team already works and does not share fate with your production cloud region. When your observability is blind, the human coordination layer becomes your primary instrument, and it has to stay up.

Write a recovery checklist, not just a detection alert

Detection is well-covered by most teams. Recovery is not. Build an explicit post-upstream-recovery checklist: drain and refill connection pools, verify queue depth is returning to baseline, warm critical caches, confirm circuit breakers have closed, watch for retry storms, and only then declare the incident resolved. The goal is to close the gap between "the provider is healthy" and "our customers are healthy," which is precisely where PSN lost hours on July 24.

The architecture questions this outage forces

Beyond the immediate on-call response, an AWS outage of this kind reopens architecture debates that teams tend to defer. None of these have universally correct answers, but every platform team should have a deliberate position rather than a default one.

Single region versus multi-region

Multi-region active-active is expensive, complex, and introduces its own failure modes around data consistency and cross-region latency. Many teams rationally choose single-region and accept the risk. That is fine, as long as it is a conscious decision with a documented recovery-time expectation, and as long as your incident tooling and observability do not also live entirely in that one region. The mistake is not being single-region; it is being single-region by accident and discovering it during the outage.

Graceful degradation over hard failure

The services that fared best on July 24 were the ones that degraded instead of collapsing. If Apple Pay is unreachable, can checkout fall back to another method rather than blocking the cart entirely? If a recommendation service times out, can the page render without it? Designing for partial availability, where the core path survives even when peripheral dependencies fail, is often a better investment than chasing five nines through raw redundancy.

Timeouts, retries, and circuit breakers

Cascade failures are frequently made worse by well-intentioned retry logic. When a dependency is down, aggressive retries amplify load and slow recovery for everyone, including the provider trying to come back. Sensible timeouts, capped retries with exponential backoff and jitter, and circuit breakers that fail fast are the difference between a contained incident and a self-inflicted denial of service. Review these settings before the next outage, not during it.

Turning the outage into an organizational habit

Individual heroics during an incident are a sign that the system around the responder is weak. The teams that consistently handle dependency outages well have built habits that make good responses the default.

  • Run game days for provider outages. Simulate a region-boundary failure and practice the "is it us or them" decision, the dependency lookup, and the recovery checklist. The first time you run the play should not be during a real AWS outage.
  • Keep escalation paths current. An out-of-date on-call schedule or a stale escalation policy turns a 2-minute page into a 20-minute scramble. Someone with real authority to make tradeoff decisions needs to be reachable fast.
  • Centralize the incident conversation. Fragmented communication across DMs, email, and three tools wastes the responder's attention. A single, durable incident channel with a clear timeline is worth more than any dashboard during a chaotic dependency outage.
  • Write blameless postmortems that produce action items. A postmortem that ends in a list of insights but no owned, dated follow-ups is theater. Track the fixes to completion.
  • Measure time-to-orient, not just time-to-resolve. For dependency outages, the metric that matters most is how quickly the team correctly identified the outage as external. That number is coachable.

Where a Slack-native on-call tool fits

When a shared platform fails, the human coordination layer becomes the most important part of your response, and it has to be fast, reliable, and where your engineers already are. This is the practical case for running on-call and incident management directly inside Slack. During the July 24 AWS outage, the teams that recovered cleanly were not the ones with the fanciest dashboards. They were the ones who paged the right person in seconds, pulled the incident conversation into one place, and kept a clean timeline while their observability was degraded.

Pagerly is built for exactly this moment. Because it lives inside Slack, on-call scheduling, escalations, and incident coordination happen where your team is already looking, without asking a stressed responder to context-switch into a separate console that might itself be affected by the outage. Alert routing gets the page to the right engineer, escalation policies make sure a missed page rolls over instead of going silent, and the incident thread becomes the single source of truth for who is doing what. When your production region is the thing that broke, keeping incident response in a tool that does not share fate with that region is not a nice-to-have; it is the difference between an organized 80-minute event and a chaotic multi-hour one.

The point is not that any tool prevents an AWS outage. Nothing you own can do that. The point is that when the platform underneath you fails, your team's ability to orient quickly, communicate clearly, and recover deliberately is entirely within your control, and that capability is worth investing in before the next incident, not after.

Key takeaways from the July 2026 AWS outage

  • Region-boundary failures look like your fault but are not. Confirm scope first. External traffic failing while internal traffic is healthy is a strong signal the problem is upstream.
  • Do not let observability share fate with production. If your monitoring and status page live in the failing region, you go blind when it counts.
  • Recovery needs its own checklist. The gap between "the provider is back" and "we are back" is where hours get lost, as the PSN recovery tail showed.
  • Cascade failures are amplified by bad retry logic. Sensible timeouts, capped retries with backoff, and circuit breakers contain the blast radius.
  • Coordination beats dashboards during a dependency outage. Fast paging, current escalation paths, and a single durable incident channel matter more than any graph.

The next outage is a matter of when

The July 24, 2026 incident was the third AWS event in three months, and the pattern is clear: the concentration of the internet on a handful of cloud providers means that a single networking fault can take down payments, delivery, streaming, and gaming in the same 80-minute window. You cannot opt out of that shared risk, but you can decide how your team meets it. Build the dependency map, run the game days, write the recovery checklist, and keep your incident coordination in a place that stays up when your cloud region does not. The teams that treat the next outage as a rehearsed play rather than a surprise are the ones whose customers barely notice. That readiness is the whole job of a modern on-call practice, and there is no better time to invest in it than the quiet stretch between incidents.

If your team took a beating on July 24, treat it as free reconnaissance for the next event. Pull the timeline apart honestly, find the minutes you lost to the wrong assumption, and turn each of those minutes into a documented habit. The concentration of critical services on a small number of cloud regions is not going to reverse, so the winning move is to make your response to shared failure boring, repeatable, and fast. An outage you cannot prevent is still an outage you can rehearse for, and the difference shows up directly in how long your customers feel the pain.

View all
Design
Product
Software Engineering
Customer Success

Latest blogs

On-Call Lessons From Meta's Repeat Outage
July 26, 2026

On-Call Lessons From Meta's Repeat Outage

Meta's second outage in 72 hours shows why on-call incident response is judged by the recurrence, not the first page.
Opsgenie Migration: Beat the 2027 On-Call Deadline
July 25, 2026

Opsgenie Migration: Beat the 2027 On-Call Deadline

A practical guide to planning your Opsgenie migration before the April 2027 shutdown, without dropping a single page.
AWS Outage Incident Response: What July 24 Taught Us
July 25, 2026

AWS Outage Incident Response: What July 24 Taught Us

How the July 24, 2026 AWS outage reveals what strong on-call and incident response really require.