← All incidents
Incident dossier · Rank #12

Delta Air Lines Atlanta Data Center Power Failure Grounds the Global Fleet

Delta Air Lines 2016-08-08 6h 0m core impact PowerFire

In the early hours of 8 August 2016, a routine scheduled transfer of load onto a backup generator at Delta Air Lines' Atlanta headquarters data center precipitated a fire in the power control room. The fire failed a transformer and cut a primary power path feeding the facility. Roughly 300 of Delta's 7,000 servers lacked a working alternate feed and went dark, and because surviving servers could no longer communicate with the failed nodes, the entire airline operations platform collapsed. Delta grounded its global fleet, cancelling about 2,100 flights over three days and delaying another 2,400, at an estimated cost of ~$150 million. CEO Ed Bastian took full responsibility. The event is a textbook case of a common-mode power fault upstream of server-level redundancy defeating an otherwise diverse design, compounded by a tightly-coupled system with no graceful degradation and an unmodernized legacy reservation tier.

Failure cascade

Failure cascade: trigger → fault → downstream impactTriggerPrimary faultDownstream impactTrigger — Power (2016-08-08)Trigger · Power2016-08-082016-08-08Primary fault at Delta Air Lines — Delta Technology Data Center, Atlanta HeadquartersDelta Air LinesDelta Technology Data Center, Atlanta HeadquartersDelta Technology Data Center,Downstream service degraded by the fault: Flight schedulingFlight schedulingDownstream service degraded by the fault: Passenger check-in and servicesPassenger check-in and servicesDownstream service degraded by the fault: Baggage handlingBaggage handlingDownstream service degraded by the fault: Aircraft fuelingAircraft fueling+3 more downstream services

Trigger → primary fault → downstream blast radius, derived from the sourced root cause and affected-services record.

Facility & location

Operator
Delta Air Lines
Data center
Delta Technology Data Center, Atlanta Headquarters
Location
Atlanta, United States
Date
2016-08-08

Impact & scale

Users affected
Tens of thousands of passengers worldwide; thousands stranded overnight and sleeping on airport floors (exact passenger count not stated by any source)
Financial
~$150 million (Delta's own estimate)
Scope
Critical (SEV-1 equivalent; full worldwide operational outage)
Services / systems down
  • Flight scheduling
  • Passenger check-in and services
  • Baggage handling
  • Aircraft fueling
  • Airport video displays
  • Public websites
  • Self-service kiosks

Impact data & metrics

Fire origin time & location02:30 AM EST, 8 Aug 2016, power-control room
Servers lacking alternate power source~300 of 7,000 (~4.3%)
Servers requiring manual reboot~500
Utility power feeds lost1 of 2 (transformer destroyed by fire)
Time to systems operational~6 hours (02:30 → 08:30 EST)
FAA ground stop lifted08:40 AM EST, 8 Aug 2016
Flights operating by 1:30 PM day-of1,700 of 6,000 scheduled
Flights cancelled over 3 days2,100 (1,000 Mon / 800 Tue / 300 Wed)
Flights delayed~2,400
Core reservation platform age52-year-old 'Deltamatic' legacy system
Estimated cost to Delta~$150 million USD
Fatalities / injuriesNone reported

Magnitude profile

Magnitude sub-scores (0–10)Magnitude sub-scores (0–10)Users 8Users affected (0–10) — breadth of the user/customer population impacted. — scored 8/10.Financial 9Financial impact (0–10) — direct + consequential cost. — scored 9/10.Duration 7Outage duration (0–10) — how long service was degraded/down. — scored 7/10.Blast 9Blast radius (0–10) — how wide the fault propagated across systems/regions. — scored 9/10.
Magnitude 8.3 = blast 9×0.35 + users 8×0.25 + financial 9×0.20 + duration 7×0.20 (sub-scores 0–10 · weighted composite)

Financial impact (~$150M) is roughly 200x the Ponemon industry-average data-center outage cost (~$730K per the cited source), hence a 9. Blast radius is near-total: a ~4% node loss (300 of 7,000 servers) cascaded into a 100% platform outage that grounded a global fleet across 334 destinations in 64 countries. Core power/IT restoration took ~6 hours (durationScore 7 reflects this rather than the 3-day operational tail). Users score reflects tens of thousands stranded worldwide, though no exact passenger figure was published.

Sequence of events (SOE)

Phased sequence of events2016-08-08 ~02:29 EST · TRIGGER — Operators perform a scheduled load transfer toward the backup generator for testing — the proximate trigger. Availability Digest: the fire 'was the result of a routine scheduled switch for testing purposes to the backup generator.'TRIGGER2016-08-08 ~02:29 ES2016-08-08 02:30 EST · TRIGGER — A fire erupts in the power-control room: 'On Monday morning, August 8, 2016, at 2:30 AM EST, a fire erupted in the power control room of Delta's data center at its Atlanta headquarters.'TRIGGER2016-08-08 02:30 ES2016-08-08 02:30 EST · DETECTION — The fire is detected/sensed at origin, but NO verifiable source names the detection system (aspirating/VESDA, spot smoke, or heat) or the exact detection moment; the only fixed data point is the 2:30 AM origin time.DETECTION2016-08-08 02:30 ES2016-08-08 ~02:31 EST · CASCADE — REPORTED, NOT RE-VERIFIED: Delta stated the failing 'power control module... malfunctioned, causing a surge to the transformer and a loss of power.' This review could not re-fetch the ABC News page (403/404).CASCADE2016-08-08 ~02:31 ES2016-08-08 ~02:31 EST · CASCADE — VERIFIED mechanism: the transformer is destroyed by the fire — 'The fire caused a transformer to fail, killing one of the two power feeds to the data center.'CASCADE2016-08-08 ~02:31 ES2016-08-08 ~02:31 EST · CASCADE — REPORTED, NOT RE-VERIFIED: a single transfer point is said to have isolated the load from both sources — 'the switchgear failure locked Delta out of its reserve generators as well as from Georgia Power.' Source page not re-fetchable this pass.CASCADE2016-08-08 ~02:31 ES2016-08-08 ~02:32 EST · IMPACT — ~300 of 7,000 servers wired only to the failed feed lose power, and their backups are also dead: '300 of Delta's 7,000 servers... were not linked to an alternate power source'; 'many of them could not fail over to their backups, which were also down due to being unpowered.'IMPACT2016-08-08 ~02:32 ES2016-08-08 ~02:33 EST · IMPACT — Whole-platform collapse: surviving servers 'could not communicate with the servers that failed. This took down Delta's entire system.' Passenger check-in, baggage, websites, kiosks, and airport displays go dark.IMPACT2016-08-08 ~02:33 ES2016-08-08 ~02:35 EST · MITIGATION — Fire SUPPRESSION: no fixed suppression system is named and no discharge is confirmed in any verifiable source; secondary trade material characterized it as 'a small fire that was quickly extinguished' but could not be re-fetched and is not certified here.MITIGATION2016-08-08 ~02:35 ES2016-08-08 ~pre-dawn EST · MITIGATION — EMERGENCY SERVICES (fire): a retrospective states firefighters were called to extinguish the blaze, but that page returned HTTP 404 this pass and the call/arrival/on-scene timeline was never officially published.MITIGATION2016-08-08 ~pre-dawn ES2016-08-08 morning EST · MITIGATION — EMERGENCY SERVICES (utility): Georgia Power crews are reported to have responded on site and worked with Delta; the utility maintained there was no area outage. Not independently re-verified this pass.MITIGATION2016-08-08 morning ES2016-08-08 morning EST · IMPACT — EVACUATION / LIFE SAFETY: none reported — an unmanned/low-occupancy overnight electrical-room fire; no injuries reported in any source (life-safety egress not triggered).IMPACT2016-08-08 morning ES2016-08-08 morning EST · RECOVERY — Manual recovery begins: '500 servers had to be rebooted' to bring the platform back online.RECOVERY2016-08-08 morning ES2016-08-08 morning EST · RECOVERY — Core reservation platform returns degraded: the 52-year-old 'Deltamatic' system comes back with 'only the old Deltamatic green-screen interface... operational', forcing manual check-in and handwritten boarding passes.RECOVERY2016-08-08 morning ES2016-08-08 08:30 EST · RECOVERY — Systems operational again: 'It wasn't until 8:30 AM, six hours later, that the Delta systems were once again operational.'RECOVERY2016-08-08 08:30 ES2016-08-08 08:40 EST · RESTORED — FAA ground stop lifted: 'the ground stop was lifted at 8:40 AM on Monday' — though 'cancellations and delays continued.' By 1:30 PM only 1,700 of 6,000 scheduled flights were operating.RESTORED2016-08-08 08:40 ES2016-08-08 → 08-10 · RESTORED — Operational recovery stretches three days: 'Delta cancelled 2,100 flights over a span of three days... 1,000 flights on Monday... 800 more on Tuesday and 300 on Wednesday. An additional 2,400 flights were delayed.' Estimated cost ~$150M.RESTORED2016-08-08 → 08-

Root cause

PROXIMATE IGNITION AND MECHANISM (verified core). A fire began at 2:30 AM EST on Monday, 8 August 2016 in the power-control room of Delta's data center at its Atlanta headquarters, and it was an electrical-equipment fire — "Delta, whose outage was caused by electrical-equipment failure" (Data Center Knowledge). Crucially, the ignition was triggered by a maintenance/test activity, not a random fault: "The fire was the result of a routine scheduled switch for testing purposes to the backup generator" (Availability Digest, verbatim). The verified physical mechanism is that the fire then destroyed a transformer: "The fire caused a transformer to fail, killing one of the two power feeds to the data center" (Availability Digest). Delta's and Georgia Power's public statements added component-level detail that this review could NOT independently re-verify this pass (source pages 403/404): Delta (COO Gil West / CEO Ed Bastian, as reported) called the failed item a "power control module" that "malfunctioned and caught fire, causing a surge to a Georgia Power transformer"; Georgia Power (spokesman John Kraft, as reported) called it "a switchgear... [that] malfunctioned for reasons that were not immediately clear," stressing "It was a failure of Delta equipment... there wasn't an area power outage." These are consistent with the verified spine but should be read as reported, not confirmed.\n\nCRITICAL DISCLOSURE GAP (well-supported). No manufacturer, model number, installation year, insulation type, or arc-fault-versus-mechanical-failure determination was ever publicly released. No official root-cause analysis, regulatory filing, or vendor status document exists; "for reasons that were not immediately clear" remained the effective ceiling on component-level causation. The physical failure mode is therefore officially unverified.\n\nLATENT ROOT — AS-BUILT REDUNDANCY DEFECT (verified). The catastrophe was not the fire, which was small and confined to the power-control room, but the redundancy defect it exposed. Delta's design intent was A+B power — "Each server in the data center has redundant power supplies, and each power supply is supposed to be plugged into different power strips" — but the as-built reality diverged: "300 of Delta's 7,000 servers in the data center were not linked to an alternate power source" (Availability Digest, verbatim). Those single-fed servers went down and "could not fail over to their backups, which were also down due to being unpowered." CEO Ed Bastian "took full responsibility for the failure" (Availability Digest); his widely-reported admission that the vulnerability "went undetected" is consistent but was not re-verified here.\n\nLATENT ROOT — TEST/TRANSFER AS TRIGGER + CONCENTRATED TRANSFER NODE. The true root cause is a stack: (1) a maintenance transfer sequence intended to prove resilience that instead ignited the gear ("result of a routine scheduled switch for testing purposes to the backup generator", verified); (2) an undetected A+B wiring defect on ~300 servers that no commissioning audit or failover test had caught (verified); and (3) a concentrated transfer/power node whose failure severed a utility feed and defeated failover simultaneously. The claim (analyst Robert Mann, EE Power, as reported — NOT re-verified) that "the switchgear failure locked Delta out of its reserve generators as well as from Georgia Power" describes this single-point-of-failure but rests on an unverified source; the verified facts alone (one feed killed + backups unpowered + whole system down) already establish that a single downstream event defeated a dual-utility-feed design.

Contributing factors

Correction of errors (COE)

Lessons learnt

Improvements & remediation

Comprehensive analysis

What is verified versus what remains reported-only

The verified spine (Availability Digest, read verbatim; corroborated on cause/cost by Data Center Knowledge) establishes: a fire at 2:30 AM EST on 8 Aug 2016 in the power-control room, triggered by a scheduled test-switch to the backup generator, destroyed a transformer and killed one of two feeds; ~300 of 7,000 servers were single-fed and their backups were unpowered; the whole system fell over; recovery took ~6 hours to 'operational' (8:30 AM), the ground stop lifted at 8:40 AM, and the airline cancelled 2,100 flights over three days at ~$150M. The component nomenclature — Delta's 'power control module' with a 'surge to the transformer', and Georgia Power's 'switchgear... reasons not immediately clear' — comes from ABC News, the AJC, EE Power and NBC News that this review could not re-fetch this pass (HTTP 403/404, archive.org blocked, WebSearch budget exhausted). Those claims are retained as reported, not certified.

The single-point-of-failure that defeated dual utility feeds

Delta built for backhoe protection — two utilities entering opposite sides of the building — yet a single overnight fire simultaneously killed a transformer/feed and left the ~300 single-fed servers' backups dead. Whether the precise 'switchgear locked Delta out of its reserve generators as well as from Georgia Power' framing (analyst Robert Mann, as reported) is exact or not, the verified facts already prove a concentrated downstream failure defeated an envelope-redundant design. The lesson is topological: redundancy must be verified as isolated and independent from utility entry all the way to each server's two power cords.

Why a small, contained fire caused a global grounding

The fire never spread beyond the power-control room, yet it grounded the world's largest airline. Two amplifiers turned a ~4% power loss into a total outage: tight interdependency (survivors 'could not communicate with the servers that failed. This took down Delta's entire system') and legacy fragility (the 52-year-old Deltamatic core returned green-screen-only, forcing handwritten boarding passes). Extinguishing the fire did not restore service — the destroyed transformer, lost feed, and failed failover had already done the damage, and recovery was gated by manual reboots of ~500 servers and a brittle monolith.

The official-RCA vacuum and source reliability

There is no official post-mortem: no Delta RCA, no regulatory or court filing, no vendor status page. The failed gear's make, model, age, insulation type, and failure mechanism were never disclosed; 'for reasons that were not immediately clear' is the effective ceiling. Fire detection and fixed suppression are entirely undocumented. Accordingly officialPostmortem is FALSE, approvedSources is limited to the two pages verified this pass (Availability Digest, Data Center Knowledge), and every component-level or emergency-response detail sourced to pages that returned 403/404 is flagged as reported-not-verified or struck.

Technical deep-dive

Delta's Atlanta data center was engineered for utility resilience at the building envelope: "Power is brought into Delta's data center from two separate utilities through opposite sides of the building to prevent power from being accidently cut with a back hoe" (Availability Digest, verbatim) — a classic backhoe-protection topology, with each server carrying redundant PSUs meant to draw from independent power strips. The 8 August event demonstrated that resilience at the envelope is worthless if a single downstream event can sever a source and defeat failover at the same time.\n\nVERIFIED FAILURE CHAIN (Availability Digest): (1) At 2:30 AM EST operators performed a scheduled transfer toward the backup generator for testing; the fire "was the result of" that switch. (2) A fire erupted in the power-control room. (3) "The fire caused a transformer to fail, killing one of the two power feeds to the data center." (4) Of ~7,000 servers, ~300 were wired only to the failed feed and lost power; "many of them could not fail over to their backups, which were also down due to being unpowered." (5) Even surviving servers "could not communicate with the servers that failed. This took down Delta's entire system" — a small (~4%) power loss collapsed the whole platform through tight interdependency.\n\nREPORTED-BUT-UNVERIFIED COMPONENT DETAIL: Delta publicly attributed the event to a "power control module" that "malfunctioned, causing a surge to the transformer and a loss of power" (Gil West, ABC News, as reported); Georgia Power attributed it to "a switchgear... malfunctioned for reasons that were not immediately clear" and insisted it was "a failure of Delta equipment... there wasn't an area power outage" (John Kraft, as reported). This review could not re-fetch those pages (HTTP 403/404) and does not treat the "surge" mechanism or the "switchgear/module" nomenclature as confirmed — the verified account is only that a fire, triggered by a test transfer, destroyed a transformer and killed one feed.\n\nRECOVERY: Manual recovery was severe — "500 servers had to be rebooted. It wasn't until 8:30 AM, six hours later, that the Delta systems were once again operational." When Delta's core platform returned it did so degraded: the reservation/passenger-service systems run on "a 52-year old legacy system called Deltamatic," and "only the old Deltamatic green-screen interface was operational," forcing agents to "manually check[] in passengers and handwrit[e] boarding passes" (Availability Digest, verbatim). The FAA ground stop lifted at 8:40 AM, but by 1:30 PM "only 1,700 of Delta's scheduled 6,000 flights were in operation," and operational recovery stretched three days (2,100 cancellations; 2,400 delays; ~$150M cost).\n\nFORENSIC UNKNOWNS (disclosed). No public source names the fire-DETECTION system (aspirating/VESDA, spot smoke, or heat) or the detection moment — the only fixed data point is the 2:30 AM origin. No source documents any fixed SUPPRESSION system (clean-agent gaseous, pre-action sprinkler, or none) or confirms a discharge; extinguishment characterizations ("a small fire that was quickly extinguished"; "firefighters were called") appear only in secondary trade/retrospective material this review could not re-fetch and does not certify. No evacuation and no injuries were reported, consistent with an unmanned overnight electrical-room fire. Critically, extinguishing the fire did not restore service — the destroyed transformer, the lost feed, and the failed failover had already done the damage.

References & provenance

Sourced from public post-incident reports. Quotes are short attributed excerpts for provenance only; the analysis above is original and substantially shorter than its sources. Last verified 2026-08-01.

Root access required

The DC Incidents dossier is a root-only module. Sign in with an authorized account to continue.

Back to Home