Ethereum Undergoes ‘Unannounced Hard Fork’ After Infura Goes Down
Ethereum infrastructure provider Infura is currently down.
As per the latest update on Nov 11, 12:37 UTC, Infura has been deploying the fix, and service is getting back to normal. It has been at Nov 11, 08:12 UTC that Infura first reported a service outage with the root cause traced to
“several components within our infrastructure which were locked to an older stable version of the go-ethereum client which encountered a critical consensus bug at block 11234873.”
“This affected several Geth versions, including 1.9.9 and 1.9.13. Components running 1.9.19 and later were unaffected,” added the service provider.
The outage also caused a delay in the price feeds of Ether and ERC-20 tokens. Moreover, cryptocurrency exchanges using Infura suspended withdrawing ETH and ERC20 tokens.
Binance has since then resumed the withdrawals.
There was a possible ETH chain split at block 11234873. Etherscan and Blockchair are showing two different chains and data after this block. We’re resolving now but have temporarily closed withdrawals. Funds are #SAFU.
— CZ Binance (@cz_binance) November 11, 2020
In response, Tether CTO Paolo Ardoino said, “Exchanges should run their own Ethereum nodes.”
Exchanges actually run their own ETH node; at least Binance does, but as has been in this case, they “suspend some services pending an investigation if the local chain does not reconcile to a third-party provider such as Infura.”
"minor hard fork without anyone noticing" -things. infura down. another day in eth-land.
— Adam Back (@adam3us) November 11, 2020
The popular Ethereum wallet MetaMask is also having issues since Infura is its default service provider. This means its users can actually change the setting while using the wallet, such as 1inch.exchange.
A Serious Issue
Nikita Zhavoronkov, the lead developer at Blockchair, shared how at one point, Ethereum developers introduced a change in the code that led to a chain split, and those like Blockchair, Infura, some miners, and others that haven’t upgraded, got stuck on a minority chain which mined about ~30 blocks in 2 hours.
The fix is to upgrade geth and run debug.setHead. One Ethereum coder said, “Older geth-versions contained consensus-flaw(s) which were hit today on mainnet. The “change introduced” was actually fixing those issue(s). This is a reminder to keep your node(s) up to date!”
This has technically been an “unannounced hard fork,” which Zhavoronkov likened to what happened with Bitcoin 7 years ago when there was a database upgrade.
“In my opinion, today’s consensus failure in Ethereum shouldn’t be underestimated and should be considered as the most serious issue Ethereum has faced since the DAO debacle 4 years ago. An investigation is in order,” he said.