- “It would be against our own interests to fabricate downtime,” – BitMEX CEO Arthur Hayes
The exchange stated that they “operate a fair and efficient platform.” Trading downtime downgrades the experience for its customers and reduces their “stature in the market.”
“It would be against our own interests to fabricate downtime,” wrote BitMEX CEO Arthur Hayes.
In his clarification for the events that occurred on March 13, Hayes said, the platform was subject to two distributed denial-of-service (DDoS) attacks at 02:16 UTC and 12:56 UTC which delayed requests going to and from the platform, causing direct disruption to users.
Unlike a hack, a DDoS attack is a distributed effort to slow down a system by overwhelming it with requests. The company is now reviewing the most vulnerable parts of the system and developing the public-facing protocols around downtime, market suspension, resumption, and communication to enhance security.
The post-mortem shared that at 02:16 UTC a botnet began an attack against the platform, the same one which unsuccessfully attacked a month ago. According to BitMEX, “the attackers identified their target in February, then waited for the moment their attack would make the most market impact.”
Then last Friday, during the peak of market volatility, the botnet overwhelmed the platform via a “specially-crafted query to the Trollbox feature, prompting the database’s query optimiser to run an extremely inefficient query plan.”
On that day, their database CPU usage reached 100%, with 99.6% of that CPU IO wait which was misdiagnosed as a failed disk as the team believed it to be a hardware failure with their cloud provider. Then 10 hours later, the same attack happened again.
During the first downtime, the BitMX team identified 156 accounts to be “ clearly erroneously triggered” on ETH/USD caused by the unintended late processing of market order. For each stop triggered erroneously, BitMEX will refund the user, putting it at a total of 40.297 XBT.