An outage that lasted more than 10 hours on markets operated by CME Group Inc. was triggered by human error at a data center owned by CyrusOne, as reported by Bloomberg. The incident, which occurred at the facility in Aurora, Illinois, was a result of on-site staff and contractors neglecting standard procedures for draining cooling towers in preparation for freezing temperatures. This oversight led to an overload of the cooling system and subsequently, a spike in temperatures.
The data center in question acts as the central hub for CME, a prominent derivatives exchange handling trillions of dollars daily across various global markets. The disruption in operations, affecting markets from Tokyo to London on November 28, halted trading activities in commodities like gold and oil, as well as interest rates.
Although CyrusOne took swift measures to rectify the cooling systems, CME stated that the initial remedial efforts exacerbated the situation, leading to the failure of several chillers at the data center.
This incident highlights the vulnerabilities associated with CME’s reliance on the data center, which was sold to CyrusOne in 2016 and leased back on a long-term agreement. Despite having a disaster recovery plan that includes relocating to a data center in New York during emergencies, CME decided against switching to the backup facility, anticipating only a brief interruption based on available information.
Acknowledging the global impact of the outage, CME expressed regret for the inconvenience caused to its clients.
In response to the outage, CyrusOne mobilized additional personnel, including top engineering teams and executives such as CEO Eric Schwartz, to manage the situation. The deployment of extra equipment helped restore cooling capacity, ensuring that the facility remained powered throughout the crisis.
To prevent future occurrences, CyrusOne has revised its cold weather protocols, bolstered on-site engineering teams, and initiated infrastructure enhancements in stages, accompanied by enhanced staff training.
The equipment malfunction serves as a wakeup call for investors heavily invested in the AI data center industry, emphasizing that lease agreements for such facilities are subject to termination clauses in cases of recurring outages.
Following the incident, Goldman Sachs Group Inc. halted a planned $1.3 billion mortgage-bond sale for CyrusOne, with concerns raised about cooling-related issues that could potentially trigger contract termination clauses.
Citigroup Inc. emphasized in a report that data center lease agreements often include provisions allowing tenants to terminate contracts due to specific deficiencies, including cooling-related challenges.
Assuring stakeholders that the incident was an isolated occurrence, Citi expressed confidence in the owner-operator and tenant’s ability to address redundancy issues and prevent similar disruptions in the future.
