Increasing Tor relays per IP address

Previously Tor restricted relay operators to a maximum of two Tor relays per IPv4 address, resulting in high costs for medium scale and large scale relay operators. Nothing to hide teamed up with the Foundation for Applied Privacy and Artikel 10 to improve this situation, resulting in a increase to 8 relays per IP address. In this blog we provide some more background to this change.

Tor is single threaded

The server market has been focused on increasing the amount of threads instead of increasing clockspeeds for more than a decade now. And with great success: AMD64 CPUs with 128 threads are commonplace now and it won’t be long before CPU’s with 256 threads will make their debut.

While this is great in general, it also adds significant complexity for Tor operators since Tor does not scale on multi-core CPUs. In order to effectively use a modern server with an large amount of threads, you would need to run many Tor relays. This has many downsides, such as complex system/relay/process management and additional OS/system overhead.

But the biggest challenge by far is overcoming the restriction of two Tor relays per IPv4 address. The amount of IPv4 addresses Tor operators need to acquire in order to saturate a modern multi-core CPU is disproportionately high, resulting in very high TCO. As a result this places a unnecessary financial burden on medium and large scale Tor operators and stifles growth of the Tor network.

IPv4 exhaustion

The reason for this high cost is - of course - the exhaustion of available IPv4 address space, increasing cost consistently for the past 10 years. In 2013 the price of one IPv4 address was around € 6,00. At the time of writing this blog, the price fluctuates between € 50,00 and € 55,00 per IPv4 address. This steady increase in cost is passed on to Tor operators one way or another. For Tor operators using colo/DPS/VPS the increase in cost is reflected in the colo/cloud providers’ rates. Tor operators running their own autonomous system/datacenter have to spent a large amount of money on IPv4 address space or need to spent considerable time on lobbying for a sympathetic party that wants to sponsor IPv4 addresses. The latter is getting more difficult by the year, the IPv4 address space is running dry for pretty much everyone.

Sybil attacks

The original intent of restricting the amount of Tor relays per IP address is to increase the cost for a Sybil attack. We felt the “two relays per IP address” restriction is likely to impede poorly organized adversaries’ efforts, but on the other hand it’s hard to imagine that more organized adversaries would be held back by this limitation. Despite the restriction, Sybil attacks have been detected in the past, proving our point. Also with the current cost of IPv4 address space, Sybil attacks are already much more expensive than they used to be. This is even the case when the amount of Tor relays per IP address would be increased.

Solution

All in all we felt the aforementioned restriction is a disproportionate measure in terms of benefits vs. cost for Tor operators. It’s safe to say that this restriction limited the amount of CPU cycles and bandwidth Tor operators can contribute to the Tor network (and make contributing to the network significantly more expensive). This will only become worse with time.

On 23-01-2023 we published a proposal to increase the amount of Tor relays per IP address. This proposal was further discussed with other operators and the people from the Tor Project on the Tor Operators meetup of 28-01-2023. Fortunately many Tor operators showed their support for this proposal and the Tor Project came up with a few scenarios. The easiest scenario was to just increase the AuthDirMaxServersPerAddr limit. And that’s what happened on 02-02-2023! AuthDirMaxServersPerAddr 4 was added by the Tor network directory authorities, doubling the amount of Tor relays per IP address.

But we weren’t there yet. While the increase from two to four was a welcome change, it was certainly not enough to reduce the cost for large scale operators meaningfully. Our aim was to increase it to 8 or 16 relays per IP address. Meanwhile the Tor project wanted to investigate the impact of this change a bit more, so we waited on that. And then on 28-06-2023 it finally happened: AuthDirMaxServersPerAddr was increased to 8 \0/.

Future

The increase to 8 Tor relays per IP address alleviates some of the pain that’s caused by Tor’s single threaded architecture. But in the end it’s just a stopgap measure and some real change is necessary. Fortunately the Tor project is putting effort in a multi-threaded rust implementation of Tor named Arti. However, Arti is still a long way (2-3 years) off so we sadly have to deal with Tor’s current limitations for the foreseeable future.