Last year, an international hotel chain revealed news that a breach of its guest reservation database exposed the personal information of up to 500 million people. According to the company, the hackers accessed sensitive information including people’s names, addresses, phone numbers, email addresses, passport numbers, dates of birth, gender, loyalty program account information, and reservation information. For some, they also stole payment card numbers and expiration dates; while that data was encrypted, it is unknown if the hackers also stole the information needed to decrypt it.
Before this news was to be shared with the public, a long-standing client of ours, a corporate investigations and risk consulting firm tasked with managing the website to which the hotel chain directed people for information on the breach and next steps, approached us to help ensure the process went smoothly. It was expected that up to hundreds of millions of visitors would log onto the website shortly after the news was released, creating too heavy a load for a normal website to handle. Our client needed to ensure the website would stay up and running smoothly while also maintaining safety and security of sensitive information and data.
When our client was apprised of the magnitude of the situation, they contacted us a few days prior to the press conference and announcement of the data breach to begin drafting an action plan. Our client sought our expertise in Microsoft Azure, including ways to drastically scale up website load accommodation and ensuring resilience. We immediately began twice-daily check-ins with our client as we jointly assessed the situation.
We made recommendations for preparing the website to handle the increased load, including:
- Ensuring applications could withstand scaling up to effectively distribute the load
- Creating low-fi versions of the website in case the regular website could not handle the load
- Changing configuration settings
- Choosing large, temporary servers
- Rigorous testing to guarantee resilience
- Configuring a Content Delivery Network – ensuring a low latency delivery of content worldwide
Complicating matters, the site that would be responsible for hosting pertinent information was a legacy website—meaning limitations in its code wasn’t easily addressable, and therefore making infrastructure enhancements all the more vital.
From Wednesday at noon to when the announcement was made public on Friday at 5 a.m., the Concurrency team was on standby to assist with any complications. We monitored the situation actively with the client. Additionally, our team was ready to be on call the entire weekend to assist with the situation; however, assessments made throughout the day determined that the website and applications were properly prepared and doing their jobs.
After the news and interest quieted down, we worked with the client to ‘turn down the dials’ and scale back the measures that were put in place to handle the increased load of visits associated with the announcement.