The Event

On Friday morning, cybersecurity company CrowdStrike launched a software update from their Falcon Sensor security software. It was quickly made apparent that the update contained a bug within the code and was faulty, however, it was too late to rectify. The update had automatically been applied to Windows OS devices, immediately making over 8.5 million appliances unusable.

Users were met with this screen when they tried to start their devices.¹

While 8.5 million devices is an unprecedented number to go down at one time, this was still under 1% of all Microsoft devices, and this is because CrowdStrike are an enterprise security solutions provider, so only organisations using their Falcon Sensor security solution for security were harmed, meaning that no home PCs were affected as they don't use the software.

CrowdStrike were quick to confirm that this was an internal problem and not the result of a cyber attack, as had been speculated, but the damage was done and a day of chaos was about to ensue.

Despite CrowdStrike's best and continuous efforts to reduce the damage, devices were inoperable for hours and the 'biggest IT outage in history' became global news.

The Impacts

The impacts were widespread as users were met with a crash screen and could not access their computers. Many banks, airports, and retailers couldn't use vital software to operate their businesses.

It did not affect Microsoft software, however, so Office 365 wasn't impacted. These applications, most notably Outlook, Teams, Excel and Word continued as normal and so organisations using only Office 365 were unharmed.

Plenty of major airports were grounded for hours, leaving thousands of passengers stranded with little idea when or how the issue would be resolved. 

Airports tried to get their flights running by using last-resort methods such as hand-written tickets and whiteboards displaying flight times. These techniques are nowhere near as secure as airlines use normally, and are very unreliable for an industry that needs such high security and confidence in its systems.

Some airports used whiteboards to display flight times²

Even with the efforts to get flights running, over 4,000 flights were cancelled and 35,500 were delayed globally.³

The impact was felt in many different industries, such as in UK hospitals that used CrowdStrike software, as some declared it a critical incident and suspended some treatment as a result. In the USA, 911 operators that relied on Windows software were unable to take calls and had to release lists of alternative numbers that could be called instead.⁴ TV was also impacted when two major UK channels, Sky News and CBBC, were unable to broadcast.

The Outcomes

Most systems are now back to normal after the crash, with Microsoft releasing a tool for IT admins to get machines working again. This is a bootable USB drive and is the most reliable and efficient way of getting systems back online.

Airports are still struggling with delays due to the backlog of passengers while flights were grounded, and this is slowly being worked out, and passengers are advised to check for any disruption before they travel.⁵

CrowdStrike has suffered a gargantuan loss as a result of the mishap, losing around $12.5 billion from their market value⁶, and their stock price has plummeted.

CrowdStrike's Stock Prices before and after the event⁷

Their CEO responded to the outage on the day that it happened, acknowledging the severity of the event, and mentioning that a fix was quickly implemented and that they have been working with impacted customers to restore their services. The blog also mentioned that CrowdStrike services are back to normal and that measures are being put in place to ensure that nothing like this happens again.⁸

It is also a wake-up call for all IT firms who need to thoroughly test and check anything before launch, and also for the affected industries that having alternative methods of operating are necessary. When strange things happen without warning, it is best to be prepared and not risk your services becoming unusable.

We hope the world never sees disruption caused by an IT incident like this in the future, but you can never say never. If any of our customers need any advice on product updates or patch management, contact us today by calling 01246 454 222 or email hello@coolspirit.co.uk.

Sources: 1: Blue Screen of Death | 2: Airport Whiteboard | 3: Airline Disruption Stats | 4: 911 Call Disruption | 5: Airline Travel Advice | 6: CrowdStrike Value Loss | 7: CrowdStrike Stock Price | 8: CrowdStrike CEO Response | 9: Featured Image