On the morning of August 5, 2026, a major AI provider went dark for roughly seven and a half hours, and thousands of engineering teams learned in real time what an AI provider outage actually costs them. Anthropic's Claude models returned elevated error rates and failed API requests starting around 3:00 AM Eastern, and applications that quietly route user traffic through a large language model suddenly had no model to route to. Chatbots stopped answering. Summarization pipelines stalled. Code assistants timed out. For a lot of on call engineers, the first sign was not a clean alert that said "vendor is down." It was a flood of confusing symptoms that looked like their own bug.
If that morning felt familiar, that is the point. The stack most teams shipped in 2024 and 2025 now has a new tier of dependency sitting on the critical path, and it behaves nothing like the databases and queues we spent a decade learning to operate. This post is a practical guide to running on call when an AI provider outage, or any critical third party outage, becomes your incident. We will look at what happened, why these failures are uniquely nasty, and the concrete monitoring, runbook, failover, and postmortem practices that keep your service reliable when someone else's service is not.
The August 5 event was not exotic. According to reporting from outlets tracking the disruption, Claude models including Opus 5 and Sonnet 5 were degraded or unavailable for about seven and a half hours, with developers seeing failed requests, elevated latency, and intermittent availability across the API (see Android Authority's coverage). It was widely reported as one of many disruptions the provider had logged during the year. Whether the exact count matters less than the pattern: large AI platforms are now high traffic, rapidly evolving systems, and rapidly evolving systems fail.
This is consistent with the broader reliability picture. Analysts have started describing cloud outages as the new normal rather than the rare exception, in part because more of the internet now depends on a small number of very large providers (TechTarget covered this shift). AI inference is the newest and least mature member of that club. The models are enormous, the demand is spiky, capacity is constrained, and the operational playbooks are still being written. When one of these platforms has a bad morning, the blast radius now includes every product that wired an LLM into a user facing feature.
The uncomfortable takeaway for engineering leaders is simple. Your effective availability is no longer just your own number. It is your number multiplied by the availability of every critical upstream you depend on, and you just added a big new one.
For years we treated third party APIs as background risk. A payment gateway or a maps API might hiccup, and we would degrade gracefully or wait it out. AI dependencies break that mental model because they often sit directly on the primary user journey, and because they fail in ways your existing alerts were never designed to catch.
Calling one provider directly makes that provider a single point of failure. Engineers who work on LLM infrastructure have been blunt about this: if you send every request to one vendor, that vendor's incident automatically becomes your incident, and your uptime can never be better than theirs (a point argued well in this piece on treating your LLM provider as an unreliable upstream). The math is unforgiving. If your app is architecturally perfect but your only model provider delivers 99.5 percent availability, your ceiling is 99.5 percent. You inherited their bad morning.
What makes this worse than a classic dependency is coupling. A model call is frequently the whole point of the feature. When the maps API is down you can still show an address. When the model is down, the "answer the customer's question" feature has nothing to fall back on unless you designed one deliberately.
Traditional dependencies fail loudly. A database refuses connections, a queue backs up, and your dashboards light up. AI dependencies fail in quieter and stranger ways that slip past naive health checks. During a degradation you may see a mix of hard 500 errors, 429 rate limit storms, sharply higher p99 latency, truncated responses, and requests that technically succeed but return low quality or empty output. A health check that only asks "did I get a 200" will happily report green while your users get garbage.
These modes matter for on call because they change what you must instrument. It is not enough to know the vendor status page says "investigating." You need to know, from inside your own system, that your real request path to the model is degraded, by how much, and since when. That is the difference between paging on truth and paging on rumor.
The good news is that AI dependencies are still dependencies, and distributed systems gave us a durable playbook for operating unreliable upstreams. You do not need to invent a new discipline. You need to apply an old one deliberately to a new tier of your stack.
Most teams monitor their own services well and their upstreams poorly. Flip that habit for anything on the critical path. For every critical AI or third party dependency, track request level metrics the same way you would track your own service: latency percentiles at p50, p75, p95, and p99, error rates broken out by status code, throughput, and a quality signal where you can define one. Guidance on monitoring AI applications repeatedly lands on the same short list, because those numbers are table stakes for tracking any upstream (a solid overview lives in this LLM monitoring best practices guide).
Then add a synthetic probe. Run a small, cheap, representative request against the provider on a schedule from inside your own environment, and alert on that. A synthetic canary that actually exercises the real code path will often detect a degradation before the vendor updates their public status page, and it removes the ambiguity of "is it them or is it us." When the canary and your production error rate move together, you have your answer in seconds instead of arguing about it in a war room.
At 3:00 AM, nobody should be improvising the response to a known failure class. A runbook per incident type is standard practice for exactly this reason. For an AI provider outage the runbook should be short and mechanical, and it should answer the questions a half awake on call engineer will actually have.
A runbook that lives three clicks deep in a wiki nobody opens during an incident is not a runbook. The most useful version is the one that surfaces in the same place the alert does, so the responder reads it without breaking focus or hunting through tabs.
The strongest mitigation for a provider outage is not being trapped on one provider. When an upstream is rate limiting you or is down, route the request to a different one that is not. Do that and your effective availability stops being any single vendor's number and becomes the combined number, which can be dramatically higher. Teams that build a routing or gateway layer in front of their models get exactly this benefit, and the pattern is well documented (see this walkthrough of LLM failover and load balancing for provider outages).
Failover is not free, and honesty about the tradeoffs is part of the job. A second provider means a second integration to test, prompt behavior that differs subtly between models, and cost and quality that vary by route. A cheaper approach for less critical features is graceful degradation: cache recent results, fall back to a smaller or local model, queue the request for later, or simply show an honest "this feature is temporarily unavailable" state instead of a spinner that never resolves. The worst outcome is the one many teams shipped by accident, where a model timeout silently becomes a broken page with no message and no fallback.
Reliability is not only an architecture problem. It is a people problem, and the people are tired. Industry research keeps finding that a large share of engineers tie their exhaustion directly to operational load, and that burnout and context switching climb when work is chaotic or poorly triaged (the DORA program has documented the link between operational health and wellbeing for years, summarized in the State of DevOps research). AI dependencies add a fresh source of pages, and if you route all of that noise straight to a human at 3:00 AM without filtering, you are manufacturing burnout.
A vendor degradation can generate hundreds of correlated alerts in minutes: every retry, every downstream timeout, every queue backup firing separately. If each one pages, the on call engineer drowns and the single alert that actually matters gets buried. Invest in grouping and deduplication so a storm of symptoms collapses into one incident that says "AI provider degraded, N services affected." Route by severity, so a full outage wakes someone and a minor latency blip files a ticket for the morning. The goal is not fewer signals in your system. It is fewer interruptions to the human, so the interruptions that remain are trustworthy.
It is tempting to write off a provider outage with "not our fault, nothing to learn." That wastes the incident. A blameless postmortem, a core practice of site reliability engineering, asks not who broke it but how our system let someone else's failure hurt our users, and how we shorten the next one (Google's SRE workbook is the canonical reference). For an AI dependency, the honest questions are pointed. Did we detect it ourselves or did a customer tell us. How long from first symptom to confirmed cause. Did the runbook exist and did it work. Should this feature have a fallback it currently lacks. Every one of those produces a concrete action item that makes the next outage smaller. The point of the exercise is a shorter, calmer response next time, not a list of names.
If you take one thing from this post, make it this list. Walk it top to bottom for every AI or critical third party dependency on your user path, and you will be in far better shape the next time a provider has a bad morning.
Most of this work has to happen where your responders already live, and for a large share of engineering teams that place is Slack. When an AI provider outage hits at 3:00 AM, the difference between a five minute and a fifty minute response is usually not raw talent. It is friction: hunting for the runbook, working out who is on call, chasing people to acknowledge, and manually copying context into a channel while the clock runs.
A Slack native on call and incident management tool like Pagerly is built to remove that friction. On call schedules and escalations live in Slack, so the right person is paged and the page is acknowledged without leaving the tool the team already has open. Alerts can be grouped so a vendor degradation shows up as one incident instead of a hundred, which is exactly the noise reduction that protects your responders from burnout. Runbooks and the incident channel sit side by side, so the confirm, mitigate, and communicate steps happen in one place instead of five browser tabs. And because the whole timeline is captured in the channel, the blameless postmortem afterward writes half of itself.
The tool is not the strategy. The strategy is treating your AI providers as unreliable upstreams and operating them with intent. But when the strategy has to execute at 3:00 AM, running it inside Slack, where the team already coordinates, is what turns a good plan into a fast recovery.
The August 5 outage will not be the last time a major AI platform has a bad day, and treating each one as a surprise is a choice, not a fate. The teams that came through that morning calmly were not lucky. They had instrumented the dependency, written the runbook, designed a fallback, tuned their alerting so the real page landed, and set up their response to run where they already worked. Everyone else spent seven and a half hours discovering, live, that an AI provider outage is now their incident.
You get to decide which group you are in before the next one. Pick your most critical AI dependency this week. Map what breaks when it degrades, add a synthetic probe, write the one page runbook, and put a fallback behind the feature. None of it is glamorous, and all of it pays for itself the first time an upstream goes dark. Reliability in the AI era is less about hoping your vendors never fail and more about making sure that when they do, your users barely notice and your on call engineer still gets to sleep.
