Google Cloud London Outage: When One Zone Hits a Region
The Google Cloud London outage began as a power event in one zone. Learn how on-call teams stop a zone failure becoming a regional incident.

The Google Cloud London outage on Tuesday, October 20, 2026 began as a power event inside a single zone, europe-west2-b, and within hours it was the reason people across Britain could not open their banking apps, process orders, or join remote court hearings. Google's own status dashboard lists the start at 23:08 Pacific time on October 19 (06:08 UTC), reports mitigation of the power event just before noon UTC, and says most workloads had recovered by about 12:10 UTC, with a very small number of customers still affected into the afternoon. That is roughly six hours of disruption from one zone in one region.
The detail every on-call engineer should sit with is the shape of it. Google described the zone as unavailable, and it also described "elevated errors and intermittent network issues" for some other services in the wider europe-west2 region. A zonal failure is exactly the thing multi-zone architectures are supposed to absorb. For a meaningful set of customers, it was not absorbed. This post walks through what happened, why a zone failure can still become your regional incident, and the concrete on-call, detection, and communication changes that keep the next one contained.
What happened in the Google Cloud London outage
Here is the timeline as reported by Google's status dashboard and summarised by IBTimes UK and ABC News. Times are converted to UTC.
- 06:08 UTC: Google's dashboard records the start of the incident. A power event makes services in europe-west2-b unavailable.
- From about 07:00 UTC: Customer reports climb. Outage trackers log well over a thousand reports for Revolut alone shortly after 07:00 UTC (08:00 in London). Customers of Lloyds Bank, Halifax, Bank of Scotland and HSBC also report problems with online banking.
- Morning: The video platform used for remote hearings in courts in England and Wales fails, delaying hearings. Media also report problems at Royal Mail, DPD, and small businesses that could not reach their own order systems.
- 11:58 UTC: Google says the power event is mitigated and that it is prioritising recovery of existing workloads. Creating new workloads may still fail.
- 12:10 UTC: Google reports recovery for most workloads.
- 14:03 UTC: Google says engineers are still working on issues for "a very small number of customers" and advises anyone still affected to open a support case.
Google has said it will publish an incident analysis once its internal investigation concludes. Until that report exists, nobody outside Google knows exactly why a zonal power problem leaked into regional errors, and this post will not guess at the internals. What we can do is look at the failure from the customer side, because that is the side your on-call team lives on.
This is not the first London power incident
It is worth knowing the history. In August 2024, Google published a full incident report for europe-west2 after a London point of presence lost both its primary and backup power feeds following a substation switchgear failure. That event took roughly a third of the region's first layer Google Front Ends offline, slowed BGP convergence, and overloaded the replacement front ends, which stretched the impact well beyond the time power was actually lost. Different facility, different mechanism, same lesson: power is a shared dependency, and the recovery path after power returns is often where the minutes pile up.
Why a single zone failure became a regional incident for customers
Most teams think about cloud failure domains as a neat hierarchy. Instances fail, so you run several. Zones fail, so you spread across three. Regions fail rarely, so you either accept the risk or pay for a second region. The Google Cloud London outage is a reminder that the hierarchy is cleaner on the architecture diagram than in production. There are several common ways a zone failure becomes your regional incident even when you think you are multi zone.
Stateful components that quietly live in one zone
Stateless web tiers are usually spread properly. The trouble is the stateful pieces: a primary database that is zonal with a standby that has never been promoted under load, a self managed Kafka or Redis cluster whose leader election depends on a quorum that two zones cannot form quickly, or a single zonal persistent disk behind an "internal tool" that turns out to sit in the payment path. When the zone with the primary goes dark, every request that needs a write fails, no matter how many healthy web pods you have in the other two zones.
Capacity that exists only on paper
Google's note that creating new workloads might still fail after mitigation is important. Many autoscaling and failover designs assume you can launch replacement capacity in the surviving zones the moment you need it. During a zonal event, every customer in the region is trying to do the same thing at the same time. If your surviving zones are running at 60 percent utilisation and you lose a third of your fleet, you need instant headroom, not a scale up request that may queue or fail. N+1 per zone is the difference between a blip and an outage.
Regional services and shared control planes
Some managed services are regional by design, and their health in a zonal event depends on how the provider has built them. Google reported elevated errors and intermittent network issues for other services in europe-west2, not only for the dark zone. From the customer side that looks like your load balancer, your managed queue, or your secret store being flaky even though "your" zones are fine. You cannot fix the provider's control plane, but you can know which of your dependencies are regional managed services and design timeouts, retries and degraded modes around them.
Retry storms and connection pinning
Clients that hold long lived connections to backends in the failed zone can keep trying them for far longer than you expect, especially with aggressive keepalives or DNS caching. Meanwhile, healthy clients retry failing writes with little or no jitter, multiplying load on whatever is still up. The result is that the surviving zones get hammered at the exact moment they are absorbing failover traffic. The August 2024 report described a version of this, where replacement front ends became overloaded and needed manual intervention.
Detection: how fast did your team know?
For the businesses caught up in this incident, the first signal was often customers, not monitoring. Outage trackers were filling with reports about banking apps around an hour after the incident began. If your users are on social media before your pager goes off, your detection is backwards.
Alert on customer journeys, not just infrastructure
Infrastructure alerts in a zonal event are noisy and ambiguous. You get node not ready, disk detach failures, and pod evictions in one zone, which can look like routine churn. What tells you that customers are hurting is a synthetic check or an SLO burn rate on a real journey: log in, view balance, submit payment, place order. Run those checks from outside your cloud provider, so they are not sharing the failure with the thing they are watching. An external uptime monitor that pings your public endpoints and alerts in Slack is the cheapest version of this, and it catches the case where everything inside the region looks green to itself.
Tag telemetry with zone
If your metrics and logs carry a zone label, a zonal failure becomes obvious on a dashboard: one line falls off a cliff, two stay flat. Without that label, you see a 30 percent error rate and spend twenty minutes arguing about whether it is a bad deploy. Add zone as a dimension to your error rate, latency, and saturation panels, and build one "error rate by zone" view that the on-call engineer opens first.
Do not wait for the provider status page
Provider status pages are written by humans who are also busy fixing the problem, and they often lag what customers are seeing. We have written about this before in why vendor status pages are not an on-call signal. Use the provider page to confirm and to read their mitigation guidance, not as the trigger that wakes your team up. Your own telemetry should get there first.
The first thirty minutes: a zonal failure runbook
When a zone goes dark, the decisions in the first half hour matter more than anything after. Here is a pragmatic sequence for the on-call engineer and the incident commander.
Minute 0 to 5: declare early and name the scope
If error rates are elevated on customer journeys and the signal correlates with one zone, declare an incident. Do not wait to be sure it is the provider. Name the suspected scope plainly in the incident channel: "Elevated errors on payments, correlated with europe-west2-b, investigating provider zonal issue." Being explicit about the hypothesis keeps the room focused and makes it easy to pull in the right people.
Minute 5 to 15: drain the zone, do not debug it
The instinct of good engineers is to find out what is wrong. During a provider zonal failure, that instinct wastes time. You cannot fix the provider's power. The useful action is to stop sending traffic to the bad zone: remove it from load balancer backends, cordon and drain the nodes, and fail over any zonal primaries that have healthy standbys. Do this deliberately and in a known order, written down in advance. If you are unsure whether a component can survive a zone loss, that is the component to check first.
Minute 15 to 30: protect the survivors
Once traffic is shifted, the risk moves to the remaining zones. Watch saturation closely, shed non essential work (batch jobs, reports, background syncs), and consider temporarily disabling features that create heavy write load. If your retry logic is aggressive, this is the moment a feature flag that reduces retry counts or enables a degraded read only mode pays for itself many times over.
Throughout: one owner for customer communication
Assign a communications owner who is not the person doing the failover. Their job is to post an honest status update within the first fifteen minutes and then on a fixed cadence. Customers forgive outages much more readily than silence. A public status page that your incident channel can update quickly beats a stream of reactive replies on social media.
What regulated teams should take from this
The most visible impact of the Google Cloud London outage fell on banking and public services. Revolut said customers' money and data were safe, which is the right first message, but customers still could not make payments, see transfers, or reach support for part of the morning. For teams in fintech, insurance, health, and the public sector, there are extra layers to plan for.
Impact tolerances are measured in customer time
UK financial firms are expected to set impact tolerances for their important business services: the maximum tolerable disruption before customers suffer intolerable harm. Those tolerances do not care whose data centre lost power. If your tolerance for card payments is two hours and your provider has a six hour zonal problem, your architecture has to make the zone irrelevant or you breach. That is a business decision with an engineering price tag, and on-call leads should make sure the people who set tolerances know what the current architecture actually delivers.
Third party dependency maps must include zones
Most vendor risk registers say "Google Cloud" or "AWS" as a single line. A useful register says which region, which zones, which managed services, and which of your important business services depend on each. When the alert fires, the on-call engineer should be able to answer "which customer journeys does europe-west2-b touch?" in under a minute.
Evidence matters after the fact
Regulators and auditors will ask what you knew and when. A timestamped incident channel, with the declaration, the hypothesis, each failover step, and each customer update, is your evidence. Reconstructing that from memory a week later is painful and unreliable. Run the incident somewhere that keeps the record for you.
Test zone evacuation before the provider tests it for you
The single most effective resilience practice for this failure mode is a scheduled zone evacuation drill. It is less dramatic than full chaos engineering and much more useful than a document nobody reads.
How to run a zone evacuation game day
- Pick a quiet window and a single zone. Announce it to support and leadership in advance, with a clear abort criterion.
- Drain, do not destroy. Remove the zone from load balancers, cordon nodes, and fail over zonal primaries using the same runbook you would use in a real incident.
- Measure customer journeys, not component health. The test passes if login, payment, and order flows stay within SLO, not if every pod reschedules.
- Time every step. Record how long detection, decision, drain, and failover take. Those numbers become your real recovery time, which is usually longer than the one in the architecture review.
- Write down every surprise. The zonal disk nobody remembered, the cron job pinned to one node pool, the license server with no standby. Each one becomes a ticket with an owner.
- Repeat quarterly. Systems drift. A design that survived a drill in spring may not survive one in autumn after three new services shipped.
Capacity planning for N+1 per zone
Zone evacuation only works if the remaining zones can absorb the load without new capacity. For a three zone deployment, that means each zone needs to handle at least half of peak traffic, so normal utilisation should sit around two thirds of what each zone can handle. It costs more. It also means a provider's capacity crunch during a regional event does not become your problem. If that cost is unacceptable for everything, apply it selectively to the services that back your most important customer journeys.
Decide in advance when you would leave the region
A zonal failure that spills into regional errors raises an uncomfortable question: at what point do you fail over to another region? Make that decision before the incident, with explicit criteria such as "regional error rate above 5 percent for 30 minutes on payment APIs and no provider ETA." Cross region failover for stateful systems is slow and risky, and making that call at 07:30 with no prior agreement is how teams end up doing half a failover and owning two broken regions instead of one degraded one.
On-call practices that make the difference
Architecture sets the ceiling for how well you survive a zonal outage. On-call practice decides how close you get to it. A few habits separate teams that ride these events out from teams that get dragged through them.
Page the right people quickly
An incident that starts at 06:08 UTC catches a UK team before breakfast. The primary on-call engineer needs to be reachable, and the escalation path to the database owner, the platform lead, and the communications owner needs to be automatic, not a hunt through a wiki for phone numbers. Paging and escalation that live where your team already works, such as Slack with on-call paging that escalates if nobody acknowledges, removes minutes from the most expensive part of the incident.
Spin up a dedicated incident channel
A zonal outage pulls in platform, database, application, support, and leadership at the same time. Running that in a general engineering channel buries decisions in noise. A dedicated channel per incident, created automatically with the right responders invited and a clear incident commander, keeps the timeline readable and doubles as your audit trail. Pagerly's incident response bot does this inside Slack.
Use pre written customer messages
The first public update in a provider outage is almost always the same: we are aware, here is what is affected, your data is safe, next update at a stated time. Write that template now, get legal and support to approve it once, and let the communications owner publish it within minutes. The fifteen minutes you save are the fifteen minutes customers spend deciding whether you are hiding something.
Hold the review even if it was "not our fault"
It is tempting to close a provider outage with a one line note blaming the cloud. Resist that. The review question is not "why did Google lose power" but "why did a zone failure hurt our customers, and how long did it take us to notice and act?" Those answers are fully within your control, and they are where the next improvement comes from.
Key takeaways for on-call and platform teams
- Zonal redundancy is not automatic resilience. Stateful components, regional managed services, and capacity assumptions can turn a single zone failure into a regional incident for you.
- Detect from the outside. External synthetic checks on customer journeys, plus zone labelled telemetry, will beat both your infrastructure alerts and the provider status page.
- Drain, do not debug. In a provider zonal event, the useful move is to stop sending traffic to the bad zone fast, using a runbook written in advance.
- Protect the survivors. Shed load, tame retries, and make sure the remaining zones have real headroom.
- Communicate early and on a cadence. One owner, a pre approved template, and a status page beat ad hoc replies every time.
- Drill it. A quarterly zone evacuation game day turns a scary unknown into a measured, rehearsed procedure.
- Decide the region failover criteria before you need them. Ambiguity at 07:30 is expensive.
Conclusion: make a zone failure stay a zone failure
The Google Cloud London outage will get a formal analysis from Google, and it is worth reading carefully when it lands. But the most useful work for your team does not depend on that report. A power event in one zone of one region disrupted banks, courts, and businesses for hours because, for many of them, the zone was more important than their architecture diagrams admitted. The fix is unglamorous: know which of your components are truly zonal, keep real headroom in the other zones, detect customer pain from outside the provider, drain fast, communicate early, and practise the whole sequence before the next power event decides to practise it for you.
If your on-call process still depends on someone noticing a spike and then hunting for the right people, start there. Put paging, escalation, incident channels, and status updates in the place your team already works, and the first thirty minutes of the next zonal outage get a lot shorter.
