Da Nang, VN,--:--:--
All articles
  • #resilience
  • #devops

What the November 2026 Microsoft 365 outage teaches web developers

A months-old bug, a routine traffic shift and automation that made things worse. Five resilience lessons from one of the year's biggest outages — and what they look like in a frontend codebase.

NDNgoc Duc3 min read

On November 12, 2026, Outlook and Exchange Online stopped working for a huge slice of the world. Microsoft's preliminary post-incident report puts the start at 16:20 UTC and the end at 16:10 UTC the next day — almost 24 hours — though public acknowledgement came later and final notifications ran into November 14.

Microsoft was clear that it wasn't a cyberattack. It was a chain of ordinary things lining up badly. That's what makes it worth studying.

What happened, in order

  1. July 2026: a Windows update reached the servers behind Microsoft 365's networking gateway. Under certain workloads, an internal process hit lock contention and burned CPU and memory until the server stopped responding. The defect sat dormant for more than three months; according to the report it slipped through testing because lab machines had fewer CPU cores than production.
  2. The day before: roughly 14% of gateway capacity was already out of service. (Microsoft's EVP Charles Lamanna attributed it to maintenance; the preliminary report, as summarised by Wccftech, to an unrelated issue.)
  3. About 15 hours before: Teams traffic failed over — routinely — onto the gateway group that Exchange Online shared.
  4. The trigger: with more load on less capacity, the latent bug started firing. As servers went unresponsive, automated traffic management sent their requests to the survivors, which then failed too. Within about two hours, more than 80% of gateway servers were down, and regional containment didn't stop the spread.
  5. Recovery was its own fight: rollback couldn't help because the bad change was months old. Restarted servers were immediately flattened by queued traffic, so Microsoft had to switch off automated routing and bring hosts back gradually, then ship a code fix.

The impact went far beyond email: Teams, SharePoint, OneDrive and Copilot features that depend on Exchange degraded, and Wikipedia's write-up lists disruption to hospitals, schools, courts and governments across several countries.

Five lessons that apply to your app

1. Your test environment is lying to you a little

The bug needed production-class hardware to show up. For web apps the equivalent is testing on a fast laptop with a warm cache. Throttle the CPU in DevTools, run Lighthouse on a mid-range phone profile, and load-test with realistic data volumes before trusting "it works on my machine".

2. Failover moves load — it doesn't remove it

A routine failover was part of the trigger. Whenever you add a fallback (a secondary API, a backup region, a retry), ask what happens to the receiving side when it suddenly gets everyone's traffic.

3. Automation can turn a fire into a wildfire

Health checks pulled sick servers and pushed their traffic onto healthy ones — exactly as designed, and exactly how the failure spread. Load-shedding beats load-moving: it's better to return a fast error to some users than a timeout to all of them.

4. "Just roll back" has an expiry date

A change from July couldn't be cleanly reverted in November. Keep risky behaviour behind feature flags that you can switch off independently of deploys, long after the code shipped.

5. Recovery needs backoff and jitter

When servers came back, queued work knocked them over again — a classic thundering herd. Every client you write can avoid contributing to one:

lib/retry.ts
ts
export async function withRetry<T>(fn: () => Promise<T>, attempts = 5) {
  for (let i = 0; ; i++) {
    try {
      return await fn();
    } catch (err) {
      if (i >= attempts - 1) throw err;
      // Exponential backoff with "full jitter": wait a random time in [0, base * 2^i]
      const cap = Math.min(30_000, 500 * 2 ** i);
      await new Promise((r) => setTimeout(r, Math.random() * cap));
    }
  }
}

The randomness matters as much as the exponent: without jitter, every client retries at the same moment and you recreate the spike you were trying to avoid.

The bigger picture

Microsoft has said a final report will follow with more corrective actions; the preliminary one already lists traffic-distribution safeguards, extra monitoring and more capacity. Meanwhile, politicians in the UK and EU used the outage to question how much critical work depends on a single provider — a question worth asking at the scale of your own stack, too.

Sources

Got a project in mind?

Let's talk