Meta's DNS Outage: An On-Call Response Playbook

Category
Falit Jain
July 24, 2026
5 min read
Meta's DNS Outage: An On-Call Response Playbook
Table of Content

On Wednesday, July 22, 2026, millions of people around the world reached for Facebook, Instagram, and WhatsApp and got nothing back. Feeds refused to refresh, messages hung, and a chorus of "account temporarily unavailable" errors spread across every timezone starting around 1:00 p.m. GMT. Early analysis from security researchers pointed to a familiar villain: a DNS outage, the internet equivalent of the phonebook forgetting every number at once. Meta has not published a full postmortem yet, but the pattern is one every on-call engineer has seen before, and it is worth learning from while it is fresh. This post breaks down what a DNS outage does to your systems, why it is uniquely hard for on-call teams to handle, and a practical, minute-by-minute response playbook you can adopt before your own phonebook goes blank.

When Meta Went Dark: What a DNS Outage Actually Does

DNS, the Domain Name System, is the layer that turns a human readable name like instagram.com into the IP address a machine can actually route to. It sits so far beneath the application that most engineers never think about it until it fails. When it does fail, the symptoms are strange and scattered. Users do not see a clean "service is down" message. Instead they get timeouts, half loaded pages, login loops, and errors that look like they could be coming from a dozen different services at once.

That is exactly what the Meta event looked like from the outside. Reports centered on being logged out, feeds that would not load, and messages that would not send. Because DNS resolution can be cached at many layers, the outage did not hit everyone at the same second. Some users were fine for a few minutes while their local resolver still held a valid record, then dropped offline as those cached records expired. Others recovered in waves as fixes propagated. This uneven, rolling behavior is a hallmark of DNS trouble, and it is one of the reasons these incidents are so disorienting to triage.

The Meta outage is not an isolated case. Just a week earlier, on July 16, an AWS CloudFront control plane failure that originated in a single Frankfurt availability zone cascaded into global 5xx errors that took down Canvas, Blackboard, Hugging Face, and the UK National Lottery for more than three hours. Different root cause, same lesson: a small, deeply embedded piece of shared infrastructure can take down enormous, seemingly unrelated services in one motion. DNS is the most shared piece of infrastructure there is.

Why DNS Outages Are Uniquely Brutal for On-Call Teams

Not all incidents are created equal. A crashed application pod is annoying but legible: you can see it, restart it, and move on. A DNS outage is different in kind, and it stresses an on-call rotation in ways that ordinary incidents do not.

The detection problem

Most monitoring assumes name resolution works. Your synthetic checks, your alerting pipeline, your paging provider, and your status page all likely depend on DNS themselves. When resolution breaks, the very tools you would use to detect and communicate the problem can break at the same time. Teams have been caught in the cruel situation where their monitoring goes quiet not because everything is healthy, but because the monitors cannot resolve the endpoints they are supposed to check. Silence is not safety. An on-call engineer who trusts a quiet dashboard during a DNS event is being lied to by their own tooling.

The blast radius problem

Because nearly every outbound call in a modern system starts with a name lookup, a DNS failure does not stay contained. Service to service calls fail. Third party API calls fail. Database connections that use hostnames fail. Even internal tools, CI pipelines, and deployment systems can seize up. The blast radius is effectively the entire estate at once, which makes it very hard to answer the first question every responder asks: what is actually broken? The honest answer during a DNS outage is often "a lot of things, all at the same time, for the same reason," and getting to that realization quickly is half the battle. A team that spends the first thirty minutes triaging each broken service as a separate incident will burn precious time before anyone connects the dots and realizes there is a single shared cause underneath all of it.

The "is it us?" problem

When Facebook goes down, half the internet assumes their own wifi is broken. On-call teams face a more expensive version of the same confusion. Is the problem in our zone, our provider, our registrar, or the public resolvers our customers use? During the Meta event, plenty of engineers at unrelated companies spent time chasing ghosts because their own dependency on Meta login or embedded content made their apps look broken too. Distinguishing "our fault," "our vendor's fault," and "the internet's fault" is one of the hardest and most important judgments an on-call responder makes, and DNS incidents blur all three lines.

A DNS Outage Incident Response Playbook, Minute by Minute

You cannot prevent every DNS outage, especially the ones that live inside a vendor you depend on. What you can do is respond with a calm, rehearsed sequence instead of improvising under pressure. Here is a practical playbook that on-call teams can adapt.

Minutes 0 to 5: confirm the signal and declare fast

The first job is to establish that something real is happening and to say so out loud. Resist the urge to quietly investigate alone for twenty minutes. Pull up an external vantage point that does not share your infrastructure: a public status aggregator, a downdetector style trend, or a simple resolution check from a network outside your own. If names are not resolving, declare an incident immediately and page the right people. Declaring early is almost never the wrong call. The cost of a false alarm is a few minutes of attention. The cost of a late declaration during a DNS outage is a widening blast radius and a customer base that found out before you did.

Minutes 5 to 20: establish a single source of truth

The most damaging thing that happens during a broad outage is fragmentation. Three engineers open three separate investigations, a manager starts a side thread, and support is answering customers with stale information. You need one channel where the incident lives, one clearly named incident commander, and one running timeline that everyone can see. This is where the practical value of a well run incident process shows up. When resolution is spotty and adrenaline is high, the team that has a single, obvious place to gather moves twice as fast as the team that is still asking "where are we tracking this?"

Assign roles explicitly and early. You want a commander who runs the response, a communications lead who owns customer and internal updates, and a scribe who keeps the timeline honest. During a DNS event especially, the communications role is not optional. Customers, executives, and support all need a steady drumbeat of updates, and without a dedicated owner those updates either flood the technical channel or never happen at all.

Minutes 20 to 60: diagnose the layer, not the symptom

DNS is a stack of layers, and effective diagnosis means walking down it methodically rather than guessing. Check resolution from multiple locations and multiple resolvers. Compare what your internal resolvers return against public ones. Look at whether the authoritative nameservers are responding, whether a recent record change or automation run correlates with the start time, and whether the failure is scoped to one zone, one provider, or the whole domain. The goal is to place the failure at a specific layer: local caching, internal resolver, authoritative server, registrar, or an upstream vendor. Each of those has a different owner and a different fix, and misplacing the layer sends the whole team down the wrong path.

Write down what you rule out as you go. In a fast moving incident, the timeline of eliminated hypotheses is as valuable as the confirmed root cause. It prevents the team from re-investigating the same dead end and gives the eventual postmortem a clean narrative of how the diagnosis actually unfolded.

After recovery: run the postmortem that actually changes something

When resolution returns, the incident is not over. The rolling nature of DNS recovery means you should confirm health from many vantage points before you call all clear, because your own cache may have recovered while customers on other resolvers are still stuck. Once you are genuinely stable, hold a blameless postmortem while the details are fresh. Focus less on the specific misconfiguration and more on the systemic questions: did our monitoring depend on the thing that failed, how long did it take us to declare, was there a single source of truth, and did our customers hear from us in time? Those answers improve every future incident, not just the next DNS one.

Building DNS Resilience Before the Next Outage

The best incident is the one that never pages anyone. While you cannot control a vendor's control plane, you can meaningfully reduce both the likelihood and the impact of DNS trouble in the parts of the stack you own. A few practices consistently pay off:

  • Use redundant DNS providers. Running authoritative DNS across two independent providers means a single provider outage degrades rather than destroys resolution for your domains.
  • Mind your TTLs deliberately. Very long time to live values make recovery slow because stale records linger. Very short values increase query load and blast radius when the authoritative layer struggles. Choose TTLs on purpose for each record, and know which ones you would want to shorten ahead of a planned change.
  • Keep monitoring off the critical path. Make sure your alerting, paging, and status page do not depend on the same resolution path they are meant to watch. An out of band check that resolves from an external network is worth its weight in gold during exactly these events.
  • Treat DNS changes like code. Version control zone files, require review, and roll out record changes gradually. A surprising share of self inflicted DNS outages trace back to an unreviewed change pushed straight to production.
  • Rehearse the failure. Run a game day where DNS resolution is intentionally broken in a test environment. The muscle memory of finding the right vantage points and declaring quickly is only built by practice, and a scheduled game day is far cheaper than learning these reflexes during a real customer facing outage.
  • Map your vendor dependencies. Know which third parties your product relies on for login, content, or resolution, so that when one of them has a DNS outage you can tell customers "this is upstream" within minutes instead of hours.

None of these are exotic. They are the unglamorous hygiene that separates teams who ride out a DNS outage with a shrug from teams who spend an afternoon in chaos. The Meta event is a reminder that even the largest engineering organizations on earth are not immune, which means the smaller shops downstream of them need their own plan rather than a hope that the giants never stumble.

Where a Slack-Native On-Call Tool Fits

Everything in the playbook above depends on one thing working: your team's ability to gather, coordinate, and communicate the moment an incident starts. That is precisely the coordination layer, and it is where a Slack-native on-call tool earns its keep. Most engineering teams already live in Slack, so the fastest possible path from "something is wrong" to "the right person is engaged" is one that never leaves the place work already happens.

With Pagerly, on-call schedules, escalations, and alerts live directly inside Slack. When an incident fires, the right responder is paged where they already are, a dedicated incident channel becomes the single source of truth automatically, and the escalation policy makes sure that if the first person does not acknowledge, the next one does. During a DNS outage, when your fancier external tooling might itself be struggling to resolve endpoints, the ability to run the human side of the response inside the chat platform your team already trusts is a real advantage. You are not asking people to context switch into a separate console at the worst possible moment. You are declaring, assigning roles, and updating stakeholders in the same window where the conversation is already happening.

The point is not that a coordination tool prevents DNS from breaking. Nothing prevents that entirely. The point is that when the phonebook goes blank, the teams who recover fastest are the ones whose people snap into a rehearsed structure without friction. Getting the right human paged, giving them one clear channel, and keeping a truthful timeline are the difference between a tidy thirty minute event and a sprawling, multi hour scramble. That is coordination, and coordination is a solved problem when your on-call lives where your team lives.

The Takeaway From Meta's Outage

The Meta outage on July 22 will eventually get a proper postmortem, and the specific misstep, whether a bad record, an expired configuration, or an automation gone wrong, will be interesting reading. But the durable lesson does not depend on those details. DNS is the most shared and most invisible dependency in your stack, its failures are rolling and disorienting, and the tools you would normally reach for may be compromised by the very outage you are fighting. That combination is what makes a DNS outage a true test of an on-call program.

Teams that come through these events well are not the ones with the cleverest debugging. They are the ones who declare early, gather in one place, assign roles without hesitation, communicate relentlessly, and keep their monitoring and coordination off the critical path. Build that muscle now, while the Meta outage is a headline and not a scar on your own quarter. When your phonebook eventually goes blank, and at internet scale it eventually will, you will be glad the response was rehearsed rather than improvised.

View all
Design
Product
Software Engineering
Customer Success

Latest blogs

Slack-Native Incident Management: PagerDuty's Big Bet
July 22, 2026

Slack-Native Incident Management: PagerDuty's Big Bet

PagerDuty made dedicated Slack incident channels generally available in July 2026, and that milestone confirms Slack-native incident management is now the category default.
Cloud Outage Response: Lessons From July's Spike
July 22, 2026

Cloud Outage Response: Lessons From July's Spike

US public cloud outages surged 84 percent in early July 2026. Here is how to build a cloud outage response that holds up when the provider fails.
Open-Source On-Call: Build vs Buy in 2026
July 21, 2026

Open-Source On-Call: Build vs Buy in 2026

Open-source on-call management is back in play in 2026, so here is how to make the build versus buy call and where the real costs hide.