MEV: Blockchain Arbitrage Bot Warfare

A comprehensive and complete guide to the MEV landscape

Maximal Extractable Value (MEV) is a radically new financial phenomenon that takes advantage of transaction ordering economics on public blockchain networks. It was an unexpected and new occurrence for most and came to the limelight during 2020’s Decentralized Finance (DeFi) boom. Some, like Phil Daian and numerous other researchers, were forward-looking enough to expect it unlike most; extensively documenting early signs of MEV back in 2019 [and even as far back as 2014!], preparing the research community for the mempool war that would ensue.

But what is MEV?

MEV is fundamental to blockchains. Any decentralized network that has some mechanism that allows users to express some economic preference for their transaction’s importance through, say, a miner tip, or generally any network where any validator receives a transaction and propagates that transaction or a block consisting of that transaction to other network participants will have MEV.

MEV is comprehensive, but one simple way of looking at MEV is as the value that is extracted by participants in the network that have a preference for how events take place on-chain, and where they can economically express that preference. This dynamic itself is a necessity in a public blockchain, as economic preference for transaction inclusion is how parties differentiate their transactions from spam transactions, which protects the network from denial of service attacks. Let’s consider the following.

Since DeFi exploded in 2020, the ecosystem has expanded to encompass many different decentralized exchanges, lending platforms, and other decentralized applications with growing complexity, interactivity, and ever-increasing pockets of liquidity.

• Across different decentralized exchanges (DEXes), arbitrage opportunities exist when there is a price imbalance.
• Across lending protocols, when a flash crash occurs in crypto prices, the difference between a borrower increasing the health factor of a loan and a borrower being liquidated can ultimately come down to transaction ordering, and some party may be able to profit by pushing for the latter.
• Across protocols that utilize price oracles in general, there is a time window in the lag between oracle updates that some profit can be extracted.

Quantitative pros have designed bots to take advantage of arbitrage opportunities, which has had negative impacts primarily on decentralized exchange users; they are able to either steal a good price on one exchange that a user was trying to buy at then sell it on another. Since the bot steals the price that the user was buying at, the user’s transaction fails, meaning they would have wasted gas fees on nothing.

The bots have also taken advantage of the [typically 2%] slippage included in a transaction to buy the asset at the quoted price before the user, then sell it to them right at the top of their slippage limit (also known as a sandwich attack).

Let’s take a look at what kinds of actors are in an advantageous position in the blockspace marketplace and have access to more MEV opportunities.

• Validators (miners) who both (a) produce blocks by picking and ordering from a pool of pending transactions and (b) propose blocks to add to the Ethereum network. This gives them the ability to extract value by ordering transactions in a way that maximizes the profits that their bots are generating.
• Arbitrage bot operators, who devise MEV strategies for frontrunning, backrunning, and sandwich attacks, then making sure their transactions get included before the user they’re trading against by paying a higher gas fee or by sending the transaction to the miner themselves. The miner could be the one to operate a bot, but this is a game of know-how and expertise; some miners won’t be as sophisticated as the quantitative actors extracting MEV.

MEV is commonly misconceived as Miner Extractable Value, but as we discovered above, this is not accurate: MEV is value extracted by those taking advantage of some mechanism can dictate transaction ordering such as Ethereum’s Priority Gas Auctions (PGAs).

As of today, MEV has allowed actors to extract over $750M. The profits per arbitrage range from miniscule amounts to millions of dollars.

It’s worth noting that MEV as a concept is not new. MEV exists on and across centralized exchanges; it exists across clearing houses in the traditional banking world. What is different with on-chain MEV is the rate and scale at which it becomes available in highly complex permissionless systems. If the Ethereum ecosystem is still small and MEV is large now, imagine how significant MEV will be when Ethereum grows by a factor of 10x.

MEV can be thought of as a security budget for a system. MEV acts as extra incentive to validate the chain, and as we’ll discover further in this article, it can drastically increase revenue for validators and further drive security and decentralization as a result. Also, generalized arbitrage is important in balancing prices across exchanges and reducing spreads.

However, there are some negative externalities to MEV, so let’s go over some of the initiatives from the community to mitigate them.

Flashbots

Flashbots is an MEV-focused company that has synthesized live on-chain data to categorize and sort rogue MEV transactions for the community, on their MEV Explore page (although, it is likely that this page underreports total MEV or will significantly underreport in the future, as the Ethereum ecosystem grows in complexity).

However, their main contribution to the ecosystem is in Flashbots Alpha which consists of MEV-Geth, an updated version of the Go-Ethereum node client that includes a direct channel for traders to bid for transaction inclusion with miners; and MEV-Relay, a centrally-operated relay network that aims to protect miners from spam related to MEV-Geth. MEV-Geth solved a critical problem at the time, where frontrunning and backrunning arbitrage bots were congesting the Ethereum network. Miners making use of Flashbots organize compatible arbitrage transactions into ‘bundles’ which get included at the top of the block; and in their v.02 release, Flashbots released bundle merging, which allows miners to include multiple bundles at the top of each block, improving efficiency. The Flashbots Dashboard shows what is happening within the Flashbots ecosystem, and the Flashbots Bundle Explorer shows Flashbots bundle transactions as they go live. Over 86% of the Ethereum hashrate has onboarded onto Flashbots to date.

Flashbots aims to create more transparency in the industry and democratize MEV extraction; where before, backchannel deals between traders and miners would allow certain actors to have an edge in the MEV game, Flashbots aims to create a network effect that convinces arbitrageurs and miners that it is better to play in a fair marketplace rather than doing secretive deals with specific arbitrageurs which could erode transparency, permissionlessness, and trust in Ethereum’s fairness. Flashbots Alpha in its current form is imperfect and is not permissionless, as MEV-Relay requires miners to be manually whitelisted to receive Flashbots bundles. It’s also not private, as it allows miners to see transactions in bundles before being included, which could raise concerns of miners executing transactions for themselves (although they would likely be blacklisted from Flashbots for now if they did choose to do this). It also does not offer finality protection against chain reorgs. Flashbots aims to solve the first two in the future by incorporating secure enclave attestations using Intel SGX as a replacement for a centralized MEV-Relay, called MEV-SGX. MEV-SGX would allow for separating the block construction and block proposal process, by allowing MEV extractors to construct and store a block with their transaction bundle at the top inside an enclave, which can attest to a miner that the block isn’t spam and that the MEV exists. The miner would use the truncated block header hash to compute the block’s proof of work, then receive the block which would be propagated to the network. MEV-SGX is still much of a work in progress, so for now, MEV-Geth is what the community has to work with.

A few teams have been creative enough to make use of MEV-Geth to power more efficient and user-friendly decentralized exchanges. Let’s go over them now.

ArcherSwap

ArcherSwap is a DEX aggregator (controlled by a mining pool DAO) that is compatible with Flashbots, which aims to offer optimal pricing (like most DEX aggregators), but also zero slippage and protection from being frontrun, backrun, or being sandwich attacked. When a user selects their trade parameters, they sign a transaction that contains data, allowing the ArcherSwap Relayer contract to pull the assets from their account and then submit the trade to Flashbots’ private mempool or to their own mining pool, rather than the public mempool. The transaction sits in the private Flashbots mempool, so it won’t be executed unless it can be executed successfully, which allows users to set very low slippage tolerances (basically, asking ArcherSwap to stick very close to the quoted price) without fears of wasting gas on a failed transaction. Since the transaction is being communicated directly with the miner through MEV-Relay, the transaction won’t be frontrun, backrun, or sandwich attacked by anyone else, or even the miner, who would get booted from MEV-Relay if they did. ArcherSwap transactions may be routed through Flashbots directly to the Archer DAO mining pool.

MistX

MistX is another DEX aggregator (controlled by the Alchemist DAO) that functions similarly to how ArcherSwap does in utilizing Flashbots, however it does not use a Relay contract and instead allows the user to interact with Flashbots more directly. Hasu’s article on MistX walks the user through a typical MistX transaction.

As a private organization, Flashbots could eventually extract a small fee from transactions for itself. The organization’s technology is already having profound network effects, and it intends to bolster its position in the MEV game by deploying solutions onto other blockchains when MEV plays a role on them. Flashbots seems like a unicorn in the making; another for-profit technology provider, like Starkware, that will play a significant role in the DeFi future; though there is room for other players to come in and disrupt their business with the right solution.

What is certain is that MEV exists, and in great scale. Flashbots miners alone are earning over 500 ETH in fee revenue on a bad day through MEV-Geth, which was around 22% of their total fee revenue (excluding block subsidies). The MEV that MEV exploiters are earning, based on Flashbots’ MEV Explore page, is ranging from $1M to $5M per day, and even then this could be significantly underreporting the scale of MEV.

Flashbots.tools is another app that lets anyone submit custom transactions directly to Flashbots’ MEV-Relay, which could be useful for the everyday layman in submitting a private transaction to a miner to rescue ERC20s from a compromised wallet, without alerting the compromiser by broadcasting a transaction to the public mempool. It requires users to paste in a private key, so it is advised to build the tool from source before using it.

CowSwap (Gnosis)

CowSwap is a DEX and DEX aggregator hybrid, launched by the Gnosis team, which uses the Gnosis Protocol V2 to first check if it can match submitted transactions peer-to-peer without having to go through a regular AMM like Uniswap or Sushiswap. If two simultaneous users of CowSwap can have their orders matched together in some way, the Gnosis protocol uses its off-chain batch auction scheme to execute that trade. If it cannot, then it forwards the transaction to a decentralized network of solvers who compete to submit the transaction to the best-priced DEX available. Since solvers take on the risk of any potential failed transactions, the user doesn’t face any of the pitfalls of setting a low slippage transaction, and the low slippage itself supposedly makes the transaction protected against sandwich attacks. However, it is unclear if the Gnosis Protocol is enough to protect transactions from being frontrun or backrun, especially since CowSwap’s solvers would have to submit transactions to the public mempool rather than to a private mempool like Flashbots. The Gnosis team states that Flashbots is more expensive than what they are currently offering; it certainly seems like some competition is in the air.

This article breaks down individual transactions, and goes over the pros and cons of each solution.

BackRunMe (bloXroute)

BackRunMe is another solution to the MEV issue being offered by bloXroute, conceptually similar to Flashbots in that it offers a direct channel with validators, but very different. BackRunMe suggests that users should submit DEX trades and other transactions to BackRunMe and deliberately allow someone to backrun them, with the promise that they won’t be frontrun. BackRunMe shares a quarter of the profits that an MEV exploiter gets by backrunning them with the user, half of the profits with the miner, five percent with bloXroute, and gives only 20% to the arbitrageur. BackRunMe is another project aiming to have network effects that benefit both users, traders, and miners, but it’s incentives need to be studied further and pales today in the traction that Flashbots has.

The Hiding Book (KeeperDAO)

KeeperDAO offers another solution to MEV extraction, similar to the solutions outlined above, where users or integrated dApps submit transactions to KeeperDAO’s private mempool. Submitting these transactions off-chain directly to a miner obviously brings in the other benefits such as no slippage transactions and no failed transactions, effectively allowing transactions sent to act as limit orders. Whitelisted arbitrage bots in the KeeperDAO ecosystem deliberately frontrun and/or backrun their transactions, and get them included in the block by a miner. All parties involved, including the user, the bot, and the integrated dApp earn ROOK token rewards. 10% of the MEV is given to the arbitrage bot and 90% of the MEV is given to the miner. Also, KeeperDAO uses a mechanism called The Coordination Game aiming to avoid competitive PGAs, by encouraging arbitrage bots to take turns in performing MEV extraction by sharing profits between them.

TaiChi Network (SparkPool)

TaiChi Network is a private transaction service offered by SparkPool, one of the largest miners by hashrate distribution on Ethereum, where again, transactions are submitted to a private mempool rather than the public mempool where it avoids arbitrage bots. Transactions may still experience MEV extraction from SparkPool miners, but SparkPool claims that it is not for selfish use and rather meant as a public good. The service is more advanced and technical than those above.

MEV on Ethereum 2.0

On ETH2, the same MEV opportunities of today will exist and can be extracted by validators selected to propose a beacon block which contains a fresh ETH1 block queried from the ETH1 chain. Since the issuance rate of new ETH supply will be going down significantly after the merge, MEV extraction can have a pronounced effect in increasing validator staking yields. Across the current 160,000 validators, MEV can increase validator rewards by 75.3% excluding transaction fees in ETH2, versus the 5.6% boost in revenue from MEV being seen on ETH1 now by Flashbots miners actively extracting MEV. MEV will only be extracted for validators chosen to propose a block, which can come down to luck. The mean number of blocks any one validator will propose is 26; with an average of 0.185 ETH in real extracted value today from MEV by Flashbots miners, this will significantly increase yields.

There will not be an even distribution of MEV yields on ETH2; there may be unlucky validators who win far fewer blocks with less MEV, while lucky validators may win far more blocks with more MEV. This variance will be smoothed out through validator pools that may distribute MEV evenly, which at the same time could be a centralizing force for consensus voting power since solo validation becomes disincentivized. MEV could amplify oligopolic dynamics in ETH2 by enriching entities with the most 32 ETH stakes faster than the ones with less, who will have the numbers to give them better block proposal luck and hence better yields. Exchanges, protocol treasures, investment funds, and validator pools will be in a position to extract the most MEV.

Exchanges like Binance, Coinbase, and Kraken will control the largest amount of validator slots. How they extract MEV will depend on the regulatory forces that dictate the type of MEV they accept revenue from. Some exchanges can use MEV synergistically; expediting transactions by including withdrawals immediately in the chain as the block proposer, reducing on-chain fees for its clients. This will give exchanges with high stakes in ETH2 a competitive edge which may hurt exchanges that don’t because of capital constraints or regulation.

The validators that propose each of the up-to-32 blocks in an epoch to the wider set of attesters will be known one epoch in advance, meaning that those proposers will have 6.4 minutes while the current epoch [that their validators aren’t a part of] is underway to compute optimal MEV extraction on the transactions in the ETH1 mempool, while also making simulation and execution easier because of the predictable guarantee of when each validator will be proposing what block. Some validators could be lucky enough to be the proposer of multiple blocks or multiple consecutive blocks within one epoch. Large validator pools and exchanges will hold multiple consecutive block proposal slots across their validator sets, which will allow for sophisticated multi-block MEV extraction.

When transaction flow ports over to L2s like ZK-rollups or optimistic rollups, that MEV will not be available for L1 validators. New forms of MEV on L2s, across L2s, or between L1 and L2 will need to be devised. MEV may be extracted by ordering batch proofs submitted by L2s in an advantageous way, while also combining that with L1 strategies.

Also, when sharding goes live later into Ethereum’s scaling journey, the order of shards may reveal MEV possibilities.

The implications of MEV

Ethics

The MEV debate is contentious and controversial. Some have suggested that MEV is outright theft and certain proposals that were made to hold MEV auctions to make MEV extraction more transparent and fair were labelled as glorified ‘frontrunning-as-a-service.’ Some have suggested that fair transaction ordering needs to be implemented on the Ethereum protocol itself. On the other hand, some argue that MEV is inherent in Ethereum as a permissionless and decentralized network, and even implementing some sort of fair ordering scheme will still leave MEV on the table and not properly address the issue, and that we need to think deeper to truly address the issue. Still, it’s clear that the solutions that teams have put forth are having a positive impact in reducing the negative externalities of MEV, at least by reducing the UX hazards that arbitrage bots were causing.

Some differentiate between “good MEV” described as general market arbitrage which tightens market spreads, or a liquidator that saves someone's loan from being liquidated; and “bad MEV” described as someone frontrunning a borrower’s loan repayment with a liquidation, or a miner censoring transactions on an L2 that uses fraud proofs, or if a miner sees an MEV opportunity 5 blocks in the past and implements a time-bandit attack to rewrite the blockchain’s history to take that profit.

Remove MEV?

Attempts to remove MEV from the protocol will simply result in miners and traders establishing off-protocol channels for facilitating MEV extraction. In the past, attempts to stop sandwich attacks — in which miners would both frontrun, then spam a transaction behind a users’ transaction in a backrun in that same block using the same gas price by updating the Geth client — simply propelled miners further towards using custom mining software, which pushes us further from a solution. All of this establishes a greater case for embracing MEV and focusing on democratizing its profits, whilst making it as permissionless and transparent as possible

Phil Daian argues that implementing some sort of fair ordering scheme will still leave MEV on the table and not properly address the issue, and that we need to think deeper to truly address the issue. Still, it’s clear that the solutions that teams have put forth are having a positive impact in reducing the negative externalities of MEV, at least by reducing the UX hazards that arbitrage bots were causing.

MEV on other blockchains

Emin Gun Sirer, the CEO of Ava Labs, argues that no protocol-level MEV exists on Avalanche, in which he likely assumes the notion of ‘Miner’ Extractable Value rather than ‘Maximal’ Extractable Value. There seems to be merit to his claims in that the same transaction prioritization mechanics and block latencies from Ethereum may differ Avalanche, which may vastly reduce the unfairness of any MEV. Phil Daian argues that while Avalanche makes good strides in the right direction, there still exists a power structure that allows certain parties to behave in a way that suits their preference; those participants in that validator set can still bias which transactions they propagate first and they can still influence outcomes probabilistically. Networks like Avalanche still have the same economic properties and implications on their consensus game as Ethereum.

MEV will exist in some form on Avalanche, as well as all other public blockchains and L2s. Cross-chain AMMs like Anyswap and Thorchain will also still leave MEV on the table for certain parties to extract from Avalanche, even if it isn’t extracted solely on Avalanche. Also, Avalanche has not yet seen the scale of DeFi volume that Ethereum has, so much remains to be seen with regards to how MEV manifests itself on Avalanche (and other chains) in the future.

MEV post-EIP-1559

Also, there are some minor implications for MEV when the EIP-1559 upgrade completes. Currently, direct-to-miner transactions on ArcherSwap, MistX, and BackRunMe, as well as peer-to-peer trades on CowSwap consume zero gas, instead directing fees via the smart contract to cover for the computation expenses. The base fee in EIP-1559 will force these transactions to pay a small gas fee, which will result in a minor change in how these transactions occur.

Conclusion

MEV is important, and particularly revealing of how unexpected dilemmas appear when we begin coordinating en masse in a radically new way. It is likely that more dilemmas like MEV will present themselves, and the community will need to be ready to react. The teams that got together and quickly devised strategies to mitigate the negative effects of MEV deserve applause, as their work has benefited everyone. More teams need to get involved in the MEV discussion, and more awareness needs to be raised about the solutions discussed in this article, as there is still lots of rogue MEV activity arguably resulting in some level of unfairness on Ethereum.

I encourage you to participate in the MEV discussions taking place on ethresear.ch and crypto Twitter. If you’ve got any questions, feel free to DM me on Twitter, and I’ll try my best to help out.