Re: [tor-dev] Proposal Waterfilling

[A. Johnson]

Hi Florentin,

Thanks for the thoughtful response!


> So why is it working? I come up the following conclusion: OVH is a big enough 
> company not to lie with "unlimited, unmetered 100Mbits". I did not try other 
> big providers, but that would be likely the same result.
> 
> Conclusion: we can run many Gbps of bandwidth with the price I gave above, 
> for now.

I wonder how confident we can be about this situation. If we are most worried 
about an attacker trying to get, say, 10% of the network, would the provider be 
as oblivious/generous? Your numbers below (10% = 15Gbps) would require running 
15*(3/2) / 0.1 = 225 relays at 3 euros/month each. Would OVH still ignore 225 
cheap VPSs at 100% bandwidth utilization? Would they still be able to provide 
100Mbps at that number?

> Yes, you are right. This is insane price and theoretically stronger against 
> Waterfilling. But let me count the number of relays needed to achieve, let's 
> say 10% of bandwidth with that provider, and let's suppose 10% is 15 Gbps 
> (https://metrics.torproject.org/bandwidth-flags.html 
> ). Waterfilling reduces 
> the bandwidth that the adversary needs by (currently) a 2/3 ratio. So, the 
> adversary needs 10 Gbits:
> 
> 1/6 = 1666 relays.
> 
> From this number, I wonder the following things:
> 
> Can an adversary puts 1666 Guard relays in the network such that this 
> community would not notice that something strange is happening? Given the 
> fact that we don't even have 2000 Guards by now.

Again, I don’t see how this would be more noticeable or alarming than a single 
entity providing 10% of the guard bandwidth. Moreover, the security argument 
that “someone will surely notice and do something” doesn’t have a good track 
record. Absent a specific plan of how to notice it and respond automatically, I 
wouldn’t want to rely on it.

> Does the provider have enough IPv4? Are they the same /16?

Are you sure there aren’t many providers with such cheap deals (I fairly easily 
found another with $9/year for an IP and a 2TB cap: 
)? Being in the same /16 
won’t make a difference in their guard probability.

>> Alternatively, I bet you could get bulk IPv4 addresses for much cheaper than 
>> the $3/month that OVH charges for its SSD VPS, which you could then 
>> potentially apply to your 100Mbps (or larger) server to get 10Mbps and more 
>> cheaply attack waterfilling. For example, OVH provides 256 IP addresses for 
>> its dedicated servers at no monthly cost 
>> (https://www.ovh.co.uk/dedicated_servers/details-servers-range-GAME-id-MC-64-OC.xml
>>  
>> ).
>>  These servers can be had for at least 55 euros/month, which provides 
>> 500Mbps unlimited. With two of those, you could achieve the above attack on 
>> waterfilling for 110 euros = $136.36/month instead of 300 euros/month = 
>> $371.92/month.
> 
> You're right. But you're also having the same /24 for all your relays running 
> on this machine. Some easy rule on the directory server can prevent this to 
> happen. Limiting the number of relays over a same /24 for example.

Incorporating IP prefix diversity in Tor’s path selection does seem like a good 
idea in general. It sounds like you are suggesting that waterfilling should 
include a fixed limit on the number of relays in a /24. This is now a new 
scheme that would need its security analyzed. A few things that come to mind:
  1. Would there be limits for larger prefixes than an adversary might obtain 
(e.g. /16)? If not, the limit is only effective for adversaries without 
resources to obtain a larger prefix.
  2. Wouldn’t this allow an adversary to squat on a prefix? For example, he 
could run a bunch of cheap relays on prefixes owned by the Tor-friendly ISPs 
and keep anybody else from contributing more resources using that ISP.
  3. If resource limits are a reasonable strategy, instead of waterfilling, why 
not apply such limits to bandwidth (e.g. no more than 10Gbps per /24)? It seems 
simpler. It is also not susceptible to an attack on water filling in which the 
water level is raised by contributing to both guard and exit bandwidth.

Best,
Aaron___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hi all :)


On 2018-03-08 00:31, A. Johnson wrote:


On Mar 7, 2018, at 5:12 PM, Florentin Rochet 
> wrote:


Hello,


On 2018-03-07 14:31, Aaron Johnson wrote:

Hello friends,


1) The cost of IPs vs. bandwidth is definitely a function of market
offers. Your $500/Gbps/month seems quite expensive compared to what
can be found on OVH (which is hosting a large number of relays): they
ask ~3 euros/IP/month, including unlimited 100 Mbps traffic. If we
assume that wgg = 2/3 and a water level at 10Mbps, this means that,
if you want to have 1Gbps of guard bandwidth,
- the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 
euros/month

- the waterfilling mechanism would cost you 3 * 100 = 300 euros/month


The question of what the cheapest attack is can indeed be estimated by
looking at market prices for the required resources. Your cost
estimate of 3.72 USD/Gbps/month for bandwidth seems off by two orders
of magnitude.



Let me merge your second answer here:

I see that I misread your cost calculation, and that you estimated 
$37.20/Gbps/month instead of $3.72/Gbps/month. This still seems low 
by an order of magnitude. Thus, my argument stands: waterfilling 
would appear to decrease the cost to an adversary of getting guard 
probability compared to Tor’s current weighting scheme.


There is still something wrong.


What’s wrong? $37.20Gbps/month = 30 Euros/Gbps/month, which is what 
you are claiming.


I am sorry, "wrong" was a bad chosen word. It is just that we are not 
comparing the same bandwidth. What is written above is 1Gbps of *guard* 
bandwidth, which means 1,5 Gbps of bandwidth due to the 2/3 ratio on 
vanilla Tor. Either one are fine, but since we started with 1Gbps of 
*guard* bandwidth, let's keep using this baseline not to get confused :)



This would be the lowest price for a sustained Gbps transfer by a 
significant margin among all of the deals that have appeared on this 
thread. The other lowest was from Alex, who found $100/Gbps/month. I 
somewhat doubt that you could actually achieve 1Gbps sustained for 30 
Euros/month on a shared VPS or that OVH would actually tolerate using 
this much bandwidth at this little cost.


Rob and s7r also raised the same argument. So, let me share my complete 
experience regarding this topic:


I decided some time ago to invest 500$ in running relays, I did some 
research to look for the cheapest offers and also to try to setup my 
relays in different AS, if possible. I did find some interesting deals 
in different countries, with different providers and I made a list to 
try them all. All of the deals were quite similar: 100 Mbits unlimited, 
at an insane low price. So insane that I was suspicious as you are all. 
I started my relays and got a few bad experiences that I can list here:


- One of the deal was 50€/year for an unlimited 100Mbits in Sweden. 
After 3 or 4 weeks, my access got simply revoked with no warning or 
message. I contacted the support and got some clumsy arguments about the 
fact that I was running an hacking tool. Needless to say, the probable 
reason was my bandwidth consumption.
- Another one was an unlimited 100 Mbits in UK for 4pounds/month. The 
first few days were nice, relaying ~70Mbits. Then I got throttled to 
8Mbits until the end of the month.
- Another one was a reseller. I managed to run 200Mbits during a few 
days of Exit bandwidth on 1 machine, for less than 8€/month. Then, my 
access were revoked due to some external complain. The funny things was 
that I did ask if I could run an Exit Tor relay before and the support 
answered that they had no problems with Tor relays.


The list can go on, I had the same kind of problems with other 
providers. All of them have something is common, they are all small 
companies using what Rob said "unlimited bandwidth as marketing term".


Hopefully I had some good experience too (all of them are exit relays):

- I run a few relays at OVH (France, Poland), 100 Mbits for 3€/month 
like the offer linked in this thread. A different datacenter for each. 
No complain from the provider and the relays are used since months.

- I run one unlimited 100Mbits relay in Moldova since months
- I run one unlimited 100Mbits relay in Canada since months

Now, If we take the /16 prefix of the IP I got from my 3 OVH European 
relays: "54.37", "137.74", "145.239", and if we do some atlas relay search:


https://metrics.torproject.org/rs.html#search/137.74
https://metrics.torproject.org/rs.html#search/%20%0954.37
https://metrics.torproject.org/rs.html#search/145.239

All relays appearing to advertise around 10~12 MiB/s are *probably* the 
offer I linked in this thread. These relays even have a huge consensus 
weight :(.


Moreover, there is some people running more than 1Gbps with this method, 
such as this relay operator: 
https://metrics.torproject.org/rs.html#details/117B99D5CE22174DEA7F1AD3BE25ECE993F486B5 
and this guy is doing 

Re: [tor-dev] Proposal Waterfilling

[A. Johnson]

> On Mar 7, 2018, at 5:12 PM, Florentin Rochet  
> wrote:
> 
> Hello,
> 
> 
> On 2018-03-07 14:31, Aaron Johnson wrote:
>> Hello friends,
>> 
>>> 1) The cost of IPs vs. bandwidth is definitely a function of market
>>> offers. Your $500/Gbps/month seems quite expensive compared to what
>>> can be found on OVH (which is hosting a large number of relays): they
>>> ask ~3 euros/IP/month, including unlimited 100 Mbps traffic. If we
>>> assume that wgg = 2/3 and a water level at 10Mbps, this means that,
>>> if you want to have 1Gbps of guard bandwidth,
>>> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
>>> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month
>> 
>> The question of what the cheapest attack is can indeed be estimated by
>> looking at market prices for the required resources. Your cost
>> estimate of 3.72 USD/Gbps/month for bandwidth seems off by two orders
>> of magnitude.
>> 
> 
> Let me merge your second answer here:
> 
>> I see that I misread your cost calculation, and that you estimated 
>> $37.20/Gbps/month instead of $3.72/Gbps/month. This still seems low by an 
>> order of magnitude. Thus, my argument stands: waterfilling would appear to 
>> decrease the cost to an adversary of getting guard probability compared to 
>> Tor’s current weighting scheme.
> 
> There is still something wrong.

What’s wrong? $37.20Gbps/month = 30 Euros/Gbps/month, which is what you are 
claiming. This would be the lowest price for a sustained Gbps transfer by a 
significant margin among all of the deals that have appeared on this thread. 
The other lowest was from Alex, who found $100/Gbps/month. I somewhat doubt 
that you could actually achieve 1Gbps sustained for 30 Euros/month on a shared 
VPS or that OVH would actually tolerate using this much bandwidth at this 
little cost. It would at least be a notable new record for the cheapest 
possible Tor bandwidth, as far as I can tell.

> With Waterfilling, we assume above a water level of 10 Mbits, so we need:
> 
> 100 VPS SSD 1 relaying 1Gbps at the guard position, which the cost turns
> to be 3*100 = 300 euros/month.

This calculation is much too kind to waterfilling :-) Why pay for a full 
100Mbps with only 1 IPv4 address when you only need 10Mbps/IP to achieve the 
same guard probability? Earlier I showed an example of a cheaper VPS 
(https://my.hiformance.com/cart.php?a=add=165 
) that appears to provide for 
just $0.63/month a VPS with an IPv4 address that is capped at 6Mbps sustained 
througput. This would be a more economical way (3.5x cheaper) to attack 
waterfilling. Alternatively, I bet you could get bulk IPv4 addresses for much 
cheaper than the $3/month that OVH charges for its SSD VPS, which you could 
then potentially apply to your 100Mbps (or larger) server to get 10Mbps and 
more cheaply attack waterfilling. For example, OVH provides 256 IP addresses 
for its dedicated servers at no monthly cost 
(https://www.ovh.co.uk/dedicated_servers/details-servers-range-GAME-id-MC-64-OC.xml
 
).
 These servers can be had for at least 55 euros/month, which provides 500Mbps 
unlimited. With two of those, you could achieve the above attack on 
waterfilling for 110 euros = $136.36/month instead of 300 euros/month = 
$371.92/month. Once we’re talking about trying to achieve a large fraction of 
the Tor network, which requires many Gbps in vanilla Tor, the fixed cost of a 
server becomes a smaller fraction of the total cost and the savings from the 
free extra IPs become greater.

> That depends on the kind of policy that the Tor network could put in
> place. If we decide that large families become a threat in
> end-positions, we may just aggregate all the bandwidth of the family,
> and apply Waterfilling. That would not kick them off, but would create a
> kind of 'quarantine'. Same kind of suggestion than the one just below.

This seems to be a different argument than you were making, which was that the 
many servers must appear to be run independently, which I still disagree with.

> This is what Waterfilling does: increase the cost of a well-defined
> attacker and offer clients to choose into a more "diverse" network.

Sorry, I still don’t agree. It increases the cost in terms of number of IP 
addresses required and causes clients to spread out more across guards with 
different IP addresses. This is a narrow notion of diversity and not one that I 
think is useful as a design principle.

Best,
Aaron___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[s7r]

Hello

Florentin Rochet wrote:
> Hello,
> 
> 
> On 2018-03-07 14:31, Aaron Johnson wrote:
>> Hello friends,
>>
>>> 1) The cost of IPs vs. bandwidth is definitely a function of market
>>> offers. Your $500/Gbps/month seems quite expensive compared to what
>>> can be found on OVH (which is hosting a large number of relays): they
>>> ask ~3 euros/IP/month, including unlimited 100 Mbps traffic. If we
>>> assume that wgg = 2/3 and a water level at 10Mbps, this means that,
>>> if you want to have 1Gbps of guard bandwidth,
>>> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
>>> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month
>>
>> The question of what the cheapest attack is can indeed be estimated by
>> looking at market prices for the required resources. Your cost
>> estimate of 3.72 USD/Gbps/month for bandwidth seems off by two orders
>> of magnitude.
>>
> 
> Let me merge your second answer here:
> 
>> I see that I misread your cost calculation, and that you estimated 
>> $37.20/Gbps/month instead of $3.72/Gbps/month. This still seems low by an 
>> order of magnitude. Thus, my argument stands: waterfilling would appear to 
>> decrease the cost to an adversary of getting guard probability compared to 
>> Tor’s current weighting scheme.
> 
> There is still something wrong.  Let's assume the adversary wants to run
> 1 Gbps of real guard bandwidth.
> 
> With vanilla Tor, the cheapest (considering only OVH) is:
> 
> VPS SSD 1 (https://www.ovh.com/fr/vps/vps-ssd.xml): You need 10 of them
> to reach 1Gbps of bandwidth, but you need 15 of them to actually relay 1
> Gbps in the guard position (due to wgg = 2/3 roughly). This is our
> calculation above: 3*10*3/2 = 45 euros/month (or a few more dollars).
> 
> With Waterfilling, we assume above a water level of 10 Mbits, so we need:
> 
> 100 VPS SSD 1 relaying 1Gbps at the guard position, which the cost turns
> to be 3*100 = 300 euros/month.
> 
[]


A VPS is a shared resource environment. All VPSes on a single physical
server share the same NIC(s). While they do advertise a port speed (like
unlimited traffic at 100 mbps, 250 mbps, 1gbps, etc) they actually refer
to the theoretical physical NIC speed. Absolutely all of them have
something like a 'fair usage policy', which means that if you use more
than n % of your port's theoretical max speed during m % of time, they
will either:

a) throttle your VPS to something they find reasonable, like 5mbps or
10mbps maximum (could be far less);

b) suspend your service and force you to get dedicated hardware +
dedicated switch port and bandwidth.

I can guarantee you will never ever _ever_ run 1gpbs of total real
effective bandwidth at the guard position at the cost of 45 euros /
month nowhere in the world (doesn't matter if it's Europe, US or
whatever). Try getting a 3 euros VPS and you'll see that you won't be
able to saturate its port for too long.



signature.asc
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hi Rob,

Thank you for your comments!


On 07/03/18 18:34, Rob Jansen wrote:
> Hi Florentin,
>
> I've added some comments below.
>
> Overall, I think a useful discussion for the community to have is to discuss 
> whether or not we think Waterfilling is even a good idea in the first place, 
> before you go ahead and do a bunch of work writing and fixing a proposal that 
> may just end up in the pile of old grad student research ideas. (Maybe I'm 
> too late, or maybe you want a proposal out there in any case.)

Agree. Most of the work is already done, though. Details remain, but
let's try to reach a consensus before :)

>> On Mar 7, 2018, at 3:28 AM, Florentin Rochet  
>> wrote:
>>
>> Hi Aaron,
>>
>> Thanks for your comments, you are definitely touching interesting aspects.
>>
>> Here are thoughts regarding your objections:
>>
>> 1) The cost of IPs vs. bandwidth is definitely a function of market offers. 
>> Your $500/Gbps/month seems quite expensive compared to what can be found on 
>> OVH (which is hosting a large number of relays): they ask ~3 euros/IP/month, 
>> including unlimited 100 Mbps traffic. If we assume that wgg = 2/3 and a 
>> water level at 10Mbps, this means that, if you want to have 1Gbps of guard 
>> bandwidth,
>> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
>> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month
>>
>> We do not believe that this is conclusive, as the market changes, and there 
>> certainly are dozens of other providers.
>>
> Have you purchased service from OVH and run relays yourself? Have you talked 
> to anyone who has? I strongly believe that you will not find a provider that 
> legitimately offers you continuous 100 Mbit/s over a long period of time for 
> 3 euros. Providers tend to use "unmetered" and "unlimited" bandwidth as 
> marketing terms, but they don't actually mean what you think unlimited means.

I do run several 100 Mbits relay, all used at full 100 Mbits with
unmetered and unlimited traffic, for 3€/month. One of them is online
since almost a year, no problem from the provider. I must have relayed
thousands of TB, and still there.

I agree with your arguments, some providers are indeed using that as a
marketing term: the small ones. I don't think OVH or other big providers
bother that you fully use what they *must* give you, and my relays
uptime confirms that. Probably because they have enough clients paying
VPS which do nothing. Basically, we're running cheap Tor relays thanks
to all other people buying OVH services and not really using them.

>  What they mean is that you have a 100 MBit/s network card, and they allow 
> you to burst up to the full 100 MBit/s. However, they usually have a total 
> bandwidth cap on such service, or become angry and threaten to disconnect 
> your service if you don't cut down your usage (this has happened to me).
>
> It is far more expensive to obtain *continuous*, i.e., *sustained* bandwidth 
> usage over time. Generally, it's cheaper to buy in bulk. In the US, the 
> cheapest bandwidth service we found (that also allows us to run Tor relays) 
> was one that offers sustained 1 Gbit/s for an average of $500/month 
> (including service fees).
>
>> The same applies for 0-day attacks: if you need to buy them just for 
>> attacking Tor, then they are expensive. If you are an organization in the 
>> business of handling 0-day attacks for various other reasons, then the costs 
>> are very different. And it may be unclear to determine if it is 
>> easier/cheaper to compromise 1 top relay or 20 mid-level relays.
>>
> It's hard to reason about this, since I'm not in the business. However, it 
> you already have a zero-day, why would you want to waste it on a Tor relay? 
> You would risk being discovered accessing the machine of a likely 
> security-consious relay operator, and you could just run your own relays. 
> Running your own relays does have some cost, but is far easier to manage and 
> more reliable since you don't have to worry about being discovered or losing 
> access because the software is patched.
>
>> And we are not sure that the picture is so clear about botnets either: bots 
>> that can become guards need to have high availability (in order to pass the 
>> guard stability requirements), and such high availability bots are also 
>> likely to have a bandwidth that is higher than the water level (abandoned 
>> machines in university networks, ...). As a result, waterfilling would 
>> increase the number of high availability bots that are needed, which is 
>> likely to be hard.
>>
> I think its much more likely that bots are running on my parents Windows 
> machines than on high-bandwidth University machines.

Sure, but all those windows computers have no chance to get the guard
flag due to the likely diurnal behavior.

>  Sure, there might be some machines with outdated OSes out there on 
> University networks, but they are also monitored 

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hello,


On 2018-03-07 14:31, Aaron Johnson wrote:
> Hello friends,
>
>> 1) The cost of IPs vs. bandwidth is definitely a function of market
>> offers. Your $500/Gbps/month seems quite expensive compared to what
>> can be found on OVH (which is hosting a large number of relays): they
>> ask ~3 euros/IP/month, including unlimited 100 Mbps traffic. If we
>> assume that wgg = 2/3 and a water level at 10Mbps, this means that,
>> if you want to have 1Gbps of guard bandwidth,
>> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
>> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month
>
> The question of what the cheapest attack is can indeed be estimated by
> looking at market prices for the required resources. Your cost
> estimate of 3.72 USD/Gbps/month for bandwidth seems off by two orders
> of magnitude.
>

Let me merge your second answer here:

> I see that I misread your cost calculation, and that you estimated 
> $37.20/Gbps/month instead of $3.72/Gbps/month. This still seems low by an 
> order of magnitude. Thus, my argument stands: waterfilling would appear to 
> decrease the cost to an adversary of getting guard probability compared to 
> Tor’s current weighting scheme.

There is still something wrong.  Let's assume the adversary wants to run
1 Gbps of real guard bandwidth.

With vanilla Tor, the cheapest (considering only OVH) is:

VPS SSD 1 (https://www.ovh.com/fr/vps/vps-ssd.xml): You need 10 of them
to reach 1Gbps of bandwidth, but you need 15 of them to actually relay 1
Gbps in the guard position (due to wgg = 2/3 roughly). This is our
calculation above: 3*10*3/2 = 45 euros/month (or a few more dollars).

With Waterfilling, we assume above a water level of 10 Mbits, so we need:

100 VPS SSD 1 relaying 1Gbps at the guard position, which the cost turns
to be 3*100 = 300 euros/month.

> The numbers I gave ($2/IP/month and $500/Gbps/month) are the amounts
> currently charged by my US hosting provider. At the time that I
> shopped around (which was in 2015), it was by far the best bandwidth
> cost that I was able to find, and those costs haven’t changed much
> since then.
>
> Currently on OVH the best I could find for hosting just now was
> $93.02/per month for 250Mbps unlimited
> (https://www.ovh.co.uk/dedicated_servers/hosting/1801host01.xml). This
> yields $372.08/Gbps/month. I am far from certain that this is the best
> price that one could find - please do point me to better pricing if
> you have it!
>
> I also just looked at Hetzter - another major Tor-friendly hosting
> provider. The best I could find was 1Gbps link capped at 100TB/month
> for $310.49 (https://wiki.hetzner.de/index.php/Traffic/en). 1Gbps
> sustained upload is 334.8Terabytes (i.e. 1e12 bytes) for a 31-day
> month. If you exceed that limit, you can arrange to pay $1.24/TB.
> Therefore I would estimate the cost to be $601.64/Gbps/month. Again, I
> maybe missing an option more tailored to a high-traffic server, and I
> would be happy to be pointed to it :-)
>
> Moreover, European bandwidth costs are among the lowest in the world.
> Other locations are likely to have even higher bandwidth costs
> (Australia, for example, has notoriously high bandwidth costs).
>
>> We do not believe that this is conclusive, as the market changes, and
>> there certainly are dozens of other providers.
>
> I do agree that the market changes, and in fact I expect the cost fo
> IPs to plummet as the shift to IPv6 becomes pervasive. The current
> high cost of IPv4 addresses is due to their recent scarcity. In any
> case, a good question to ask would be how Tor should adjust to changes
> in market pricing over time.
>
>> The same applies for 0-day attacks: if you need to buy them just for
>> attacking Tor, then they are expensive. If you are an organization in
>> the business of handling 0-day attacks for various other reasons,
>> then the costs are very different. And it may be unclear to determine
>> if it is easier/cheaper to compromise 1 top relay or 20 mid-level relays.
>
> I agree that the cost of compromising machines is unclear. However, we
> should guess, and the business of 0-days has provided some signals of
> their value in terms of their price. 0-days for the Tor software stack
> are expensive, as, for security reasons, (well-run) Tor relays run few
> services other than the tor process. I haven’t seen great data on
> Linux zero-days, but recently a Windows zero-day (Windows being the
> second most-common Tor relays OS) appeared to cost $90K
> (https://www.csoonline.com/article/3077447/security/cost-of-a-windows-zero-day-exploit-this-one-goes-for-9.html).
> Deploying a zero-day does impose a cost, as it increases the chance of
> that exploit being discovered and its value lost. Therefore, such
> exploits are likely to be deployed only on high-value targets. I would
> argue that Tor relays are unlikely to be such a target because it is
> so much cheaper to simply run your own relays. An exception could 

Re: [tor-dev] Proposal Waterfilling

[A. Johnson]

Sorry, that link should have been 
>.

Best,
Aaron

> On Mar 7, 2018, at 4:18 PM, A. Johnson  wrote:
> 
> OVH and OVH resellers do seem to have some insane prices.
> 
> On the other end, the waterfilling assumption we were working off of was a 
> water level of 10Mbps. A server that can sustain that seems quite cheap. In 
> fact, a quick Google search for “cheap vps” yielded this offer of a VPS with 
> one IPv4 address and a 1Gbps port capped at 2TB/month for $0.63/month: 
>  >. 2TB/month is about 
> 6Mbps sustained, which falls below the supposed water level and thus gets 
> fully allocated to guard probability. Thus to achieve in waterfilling a total 
> guard probability equal to that of 1Gbps relay in today’s Tor (taking into 
> account the 1/3 loss of bandwidth to the middle position in today’s Tor), one 
> could run 1000*(2/3)/6 ~= 112 of these at $71/month. This would be cheaper 
> than the price below of $100/month for 1Gbps.
> 
> Can we get even lower attacking either system…? :-)
> 
> Best,
> Aaron
> 
>> On Mar 7, 2018, at 4:01 PM, Alexander Nasonov > > wrote:
>> 
>> Aaron Johnson wrote:
>>> Currently on OVH the best I could find for hosting just now was
>>> $93.02/per month for 250Mbps unlimited
>>> (https://www.ovh.co.uk/dedicated_servers/hosting/1801host01.xml 
>>> ).
>> 
>> https://www.ovh.com/world/discover/poland.xml 
>> 
>> 
>> $49.99/per month, 500 Mbps bandwidth (burst   1 Gbps ) unlimited
>> 
>> My new relay runs there
>> http://rougmnvswfsmd4dq.onion/rs.html#details/2235E316DF8E737081A365A1386F36035592A6BD
>> 
>> but I'm not very happy with the setup because this particular offer
>> doesn't come with IPMI console and I had to install Proxmox Linux
>> and run NetBSD in a virtual machine. Clock isn't very stable even
>> with ntpd, I setup a cronjob to hard reset it periodically.
>> 
>> Alex
>> ___
>> tor-dev mailing list
>> tor-dev@lists.torproject.org
>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
> 

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[A. Johnson]

OVH and OVH resellers do seem to have some insane prices.

On the other end, the waterfilling assumption we were working off of was a 
water level of 10Mbps. A server that can sustain that seems quite cheap. In 
fact, a quick Google search for “cheap vps” yielded this offer of a VPS with 
one IPv4 address and a 1Gbps port capped at 2TB/month for $0.63/month: 
>. 2TB/month is about 
6Mbps sustained, which falls below the supposed water level and thus gets fully 
allocated to guard probability. Thus to achieve in waterfilling a total guard 
probability equal to that of 1Gbps relay in today’s Tor (taking into account 
the 1/3 loss of bandwidth to the middle position in today’s Tor), one could run 
1000*(2/3)/6 ~= 112 of these at $71/month. This would be cheaper than the price 
below of $100/month for 1Gbps.

Can we get even lower attacking either system…? :-)

Best,
Aaron

> On Mar 7, 2018, at 4:01 PM, Alexander Nasonov  wrote:
> 
> Aaron Johnson wrote:
>> Currently on OVH the best I could find for hosting just now was
>> $93.02/per month for 250Mbps unlimited
>> (https://www.ovh.co.uk/dedicated_servers/hosting/1801host01.xml).
> 
> https://www.ovh.com/world/discover/poland.xml
> 
> $49.99/per month, 500 Mbps bandwidth (burst   1 Gbps ) unlimited
> 
> My new relay runs there
> http://rougmnvswfsmd4dq.onion/rs.html#details/2235E316DF8E737081A365A1386F36035592A6BD
> 
> but I'm not very happy with the setup because this particular offer
> doesn't come with IPMI console and I had to install Proxmox Linux
> and run NetBSD in a virtual machine. Clock isn't very stable even
> with ntpd, I setup a cronjob to hard reset it periodically.
> 
> Alex
> ___
> tor-dev mailing list
> tor-dev@lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Alexander Nasonov]

Aaron Johnson wrote:
> Currently on OVH the best I could find for hosting just now was
> $93.02/per month for 250Mbps unlimited
> (https://www.ovh.co.uk/dedicated_servers/hosting/1801host01.xml).

https://www.ovh.com/world/discover/poland.xml

$49.99/per month, 500 Mbps bandwidth (burst   1 Gbps ) unlimited

My new relay runs there
http://rougmnvswfsmd4dq.onion/rs.html#details/2235E316DF8E737081A365A1386F36035592A6BD

but I'm not very happy with the setup because this particular offer
doesn't come with IPMI console and I had to install Proxmox Linux
and run NetBSD in a virtual machine. Clock isn't very stable even
with ntpd, I setup a cronjob to hard reset it periodically.

Alex


signature.asc
Description: PGP signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[teor]

> On 7 Mar 2018, at 18:34, Rob Jansen  wrote:
> 
> It is far more expensive to obtain *continuous*, i.e., *sustained* bandwidth 
> usage over time. Generally, it's cheaper to buy in bulk. In the US, the 
> cheapest bandwidth service we found (that also allows us to run Tor relays) 
> was one that offers sustained 1 Gbit/s for an average of $500/month 
> (including service fees).

In 2016, OVH was offering a discount for 1 Gbps servers in Europe
for USD 200/month. The ordinary price was USD 300/month.

We can also purchase a 250 Mbps server in Canada for USD 60/month,
but it doesn't get as much Guard bandwidth, because of its location or
connectivity.

T
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Rob Jansen]

> On Mar 7, 2018, at 12:34 PM, Rob Jansen  wrote:
> 
> Hi Florentin,
> 
> I've added some comments below.

(I just found out that Aaron responded to your reply this morning, but I didn't 
get that email (it probably got stuck somewhere in the NRL email filters). 
Sorry if I made any points that were already made, but they were made 
independently.)

Best,
Rob


signature.asc
Description: Message signed with OpenPGP
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Rob Jansen]

Hi Florentin,

I've added some comments below.

Overall, I think a useful discussion for the community to have is to discuss 
whether or not we think Waterfilling is even a good idea in the first place, 
before you go ahead and do a bunch of work writing and fixing a proposal that 
may just end up in the pile of old grad student research ideas. (Maybe I'm too 
late, or maybe you want a proposal out there in any case.)

> On Mar 7, 2018, at 3:28 AM, Florentin Rochet  
> wrote:
> 
> Hi Aaron,
> 
> Thanks for your comments, you are definitely touching interesting aspects.
> 
> Here are thoughts regarding your objections:
> 
> 1) The cost of IPs vs. bandwidth is definitely a function of market offers. 
> Your $500/Gbps/month seems quite expensive compared to what can be found on 
> OVH (which is hosting a large number of relays): they ask ~3 euros/IP/month, 
> including unlimited 100 Mbps traffic. If we assume that wgg = 2/3 and a water 
> level at 10Mbps, this means that, if you want to have 1Gbps of guard 
> bandwidth,
> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month
> 
> We do not believe that this is conclusive, as the market changes, and there 
> certainly are dozens of other providers.
> 

Have you purchased service from OVH and run relays yourself? Have you talked to 
anyone who has? I strongly believe that you will not find a provider that 
legitimately offers you continuous 100 Mbit/s over a long period of time for 3 
euros. Providers tend to use "unmetered" and "unlimited" bandwidth as marketing 
terms, but they don't actually mean what you think unlimited means. What they 
mean is that you have a 100 MBit/s network card, and they allow you to burst up 
to the full 100 MBit/s. However, they usually have a total bandwidth cap on 
such service, or become angry and threaten to disconnect your service if you 
don't cut down your usage (this has happened to me).

It is far more expensive to obtain *continuous*, i.e., *sustained* bandwidth 
usage over time. Generally, it's cheaper to buy in bulk. In the US, the 
cheapest bandwidth service we found (that also allows us to run Tor relays) was 
one that offers sustained 1 Gbit/s for an average of $500/month (including 
service fees).

> The same applies for 0-day attacks: if you need to buy them just for 
> attacking Tor, then they are expensive. If you are an organization in the 
> business of handling 0-day attacks for various other reasons, then the costs 
> are very different. And it may be unclear to determine if it is 
> easier/cheaper to compromise 1 top relay or 20 mid-level relays.
> 

It's hard to reason about this, since I'm not in the business. However, it you 
already have a zero-day, why would you want to waste it on a Tor relay? You 
would risk being discovered accessing the machine of a likely security-consious 
relay operator, and you could just run your own relays. Running your own relays 
does have some cost, but is far easier to manage and more reliable since you 
don't have to worry about being discovered or losing access because the 
software is patched.

> And we are not sure that the picture is so clear about botnets either: bots 
> that can become guards need to have high availability (in order to pass the 
> guard stability requirements), and such high availability bots are also 
> likely to have a bandwidth that is higher than the water level (abandoned 
> machines in university networks, ...). As a result, waterfilling would 
> increase the number of high availability bots that are needed, which is 
> likely to be hard.
> 

I think its much more likely that bots are running on my parents Windows 
machines than on high-bandwidth University machines. Sure, there might be some 
machines with outdated OSes out there on University networks, but they are also 
monitored pretty heavily for suspicious activity by the University IT folks, 
who regularly check in with the machine owners with anything suspicious occurs 
on the network.

> 2) Waterfilling makes it necessary for an adversary to run a larger number of 
> relays. Apart from the costs of service providers, this large number of 
> relays need to be managed in an apparently independent way, otherwise they 
> would become suspicious to community  members, like nusenu who is doing a 
> great job spotting all anomalies. It seems plausible that running 100 relays 
> in such a way that they look independent is at least as difficult as doing 
> that with 10 relays.
> 

But not much more difficult, and not difficult enough that an intern could not 
whip up a managed deployment in a few weeks. There are various tools out there 
that can automate software installation and configuration. Ansible, Chef, and 
Puppet are popular ones, but here is a longer list:

https://en.wikipedia.org/wiki/Comparison_of_open-source_configuration_management_software

I would be 

Re: [tor-dev] Proposal Waterfilling

[Aaron Johnson]

>> 1) The cost of IPs vs. bandwidth is definitely a function of market offers. 
>> Your $500/Gbps/month seems quite expensive compared to what can be found on 
>> OVH (which is hosting a large number of relays): they ask ~3 euros/IP/month, 
>> including unlimited 100 Mbps traffic. If we assume that wgg = 2/3 and a 
>> water level at 10Mbps, this means that, if you want to have 1Gbps of guard 
>> bandwidth,
>> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
>> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month
> 
> The question of what the cheapest attack is can indeed be estimated by 
> looking at market prices for the required resources. Your cost estimate of 
> 3.72 USD/Gbps/month for bandwidth seems off by two orders of magnitude.

I see that I misread your cost calculation, and that you estimated 
$37.20/Gbps/month instead of $3.72/Gbps/month. This still seems low by an order 
of magnitude. Thus, my argument stands: waterfilling would appear to decrease 
the cost to an adversary of getting guard probability compared to Tor’s current 
weighting scheme.

Best,
Aaron
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Aaron Johnson]

Hello friends,

> 1) The cost of IPs vs. bandwidth is definitely a function of market offers. 
> Your $500/Gbps/month seems quite expensive compared to what can be found on 
> OVH (which is hosting a large number of relays): they ask ~3 euros/IP/month, 
> including unlimited 100 Mbps traffic. If we assume that wgg = 2/3 and a water 
> level at 10Mbps, this means that, if you want to have 1Gbps of guard 
> bandwidth,
> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month

The question of what the cheapest attack is can indeed be estimated by looking 
at market prices for the required resources. Your cost estimate of 3.72 
USD/Gbps/month for bandwidth seems off by two orders of magnitude.

The numbers I gave ($2/IP/month and $500/Gbps/month) are the amounts currently 
charged by my US hosting provider. At the time that I shopped around (which was 
in 2015), it was by far the best bandwidth cost that I was able to find, and 
those costs haven’t changed much since then.

Currently on OVH the best I could find for hosting just now was $93.02/per 
month for 250Mbps unlimited 
(https://www.ovh.co.uk/dedicated_servers/hosting/1801host01.xml). This yields 
$372.08/Gbps/month. I am far from certain that this is the best price that one 
could find - please do point me to better pricing if you have it!

I also just looked at Hetzter - another major Tor-friendly hosting provider. 
The best I could find was 1Gbps link capped at 100TB/month for $310.49 
(https://wiki.hetzner.de/index.php/Traffic/en 
). 1Gbps sustained upload is 
334.8Terabytes (i.e. 1e12 bytes) for a 31-day month. If you exceed that limit, 
you can arrange to pay $1.24/TB. Therefore I would estimate the cost to be 
$601.64/Gbps/month. Again, I maybe missing an option more tailored to a 
high-traffic server, and I would be happy to be pointed to it :-)

Moreover, European bandwidth costs are among the lowest in the world. Other 
locations are likely to have even higher bandwidth costs (Australia, for 
example, has notoriously high bandwidth costs).

> We do not believe that this is conclusive, as the market changes, and there 
> certainly are dozens of other providers.

I do agree that the market changes, and in fact I expect the cost fo IPs to 
plummet as the shift to IPv6 becomes pervasive. The current high cost of IPv4 
addresses is due to their recent scarcity. In any case, a good question to ask 
would be how Tor should adjust to changes in market pricing over time.

> The same applies for 0-day attacks: if you need to buy them just for 
> attacking Tor, then they are expensive. If you are an organization in the 
> business of handling 0-day attacks for various other reasons, then the costs 
> are very different. And it may be unclear to determine if it is 
> easier/cheaper to compromise 1 top relay or 20 mid-level relays.

I agree that the cost of compromising machines is unclear. However, we should 
guess, and the business of 0-days has provided some signals of their value in 
terms of their price. 0-days for the Tor software stack are expensive, as, for 
security reasons, (well-run) Tor relays run few services other than the tor 
process. I haven’t seen great data on Linux zero-days, but recently a Windows 
zero-day (Windows being the second most-common Tor relays OS) appeared to cost 
$90K 
(https://www.csoonline.com/article/3077447/security/cost-of-a-windows-zero-day-exploit-this-one-goes-for-9.html
 
).
 Deploying a zero-day does impose a cost, as it increases the chance of that 
exploit being discovered and its value lost. Therefore, such exploits are 
likely to be deployed only on high-value targets. I would argue that Tor relays 
are unlikely to be such a target because it is so much cheaper to simply run 
your own relays. An exception could be a specific targeted investigation in 
which some suspect is behind a known relay (say, a hidden service behind a 
guard), because running other relays doesn’t help dislodge the target from 
behind its existing guard.

> And we are not sure that the picture is so clear about botnets either: bots 
> that can become guards need to have high availability (in order to pass the 
> guard stability requirements), and such high availability bots are also 
> likely to have a bandwidth that is higher than the water level (abandoned 
> machines in university networks, ...). As a result, waterfilling would 
> increase the number of high availability bots that are needed, which is 
> likely to be hard.

This doesn’t seem like a good argument to me: “bots that become guards must 
have high availability, and thus they likely have high bandwidth”. How many 
bots would become guards in the first place? And why would availability (by 
which I understand you to mean 

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hi Aaron,

Thanks for your comments, you are definitely touching interesting aspects.

Here are thoughts regarding your objections:

1) The cost of IPs vs. bandwidth is definitely a function of market 
offers. Your $500/Gbps/month seems quite expensive compared to what can 
be found on OVH (which is hosting a large number of relays): they ask ~3 
euros/IP/month, including unlimited 100 Mbps traffic. If we assume that 
wgg = 2/3 and a water level at 10Mbps, this means that, if you want to 
have 1Gbps of guard bandwidth,

- the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 euros/month
- the waterfilling mechanism would cost you 3 * 100 = 300 euros/month

We do not believe that this is conclusive, as the market changes, and 
there certainly are dozens of other providers.


The same applies for 0-day attacks: if you need to buy them just for 
attacking Tor, then they are expensive. If you are an organization in 
the business of handling 0-day attacks for various other reasons, then 
the costs are very different. And it may be unclear to determine if it 
is easier/cheaper to compromise 1 top relay or 20 mid-level relays.


And we are not sure that the picture is so clear about botnets either: 
bots that can become guards need to have high availability (in order to 
pass the guard stability requirements), and such high availability bots 
are also likely to have a bandwidth that is higher than the water level 
(abandoned machines in university networks, ...). As a result, 
waterfilling would increase the number of high availability bots that 
are needed, which is likely to be hard.


2) Waterfilling makes it necessary for an adversary to run a larger 
number of relays. Apart from the costs of service providers, this large 
number of relays need to be managed in an apparently independent way, 
otherwise they would become suspicious to community  members, like 
nusenu who is doing a great job spotting all anomalies. It seems 
plausible that running 100 relays in such a way that they look 
independent is at least as difficult as doing that with 10 relays.


3) The question of the protection from relays, ASes or IXPs is puzzling, 
and we do not have a strong opinion about it. We focused on relays 
because they are what is available to any attacker, compared to ASes or 
IXPs which are more specific adversaries. But, if there is a consensus 
that ASes or IXPs should rather be considered as the main target, it is 
easy to implement waterfilling at the AS or IXP level rather than at the 
IP level: just aggregate the bandwidth relayed per AS or IXP, and apply 
the waterfilling level computation method to them. Or we could mix the 
weights obtained for all these adversaries, in order to get some 
improvement against all of them instead of an improvement against only 
one and being agnostic about the others.


4) More fundamentally, since the fundamental idea of Tor is to mix 
traffic through a large number of relays, it seems to be a sound design 
principle to make the choice of the critical relays as uniform as 
possible, as Waterfilling aims to do. A casual Tor user may be concerned 
to see that his traffic is very likely to be routed through a very small 
number of top relays, and this effect is likely to increase as soon as 
a  multi-cores compliant implementation of Tor rises (rust dev). Current 
top relays which suffer from the main CPU bottleneck will probably be 
free to relay even more bandwidth than they already do, and gain an even 
more disproportionate consensus weight. Waterfilling might prevent that, 
and keep those useful relays doing their job at the middle position of 
paths.


We hope those thoughts can help, and thanks again for sharing yours.

Best,

Florentin and Olivier

On 2018-03-05 23:30, Aaron Johnson wrote:

Hello,

I recently took the time to read the waterfilling paper. I’m not sure its a 
good idea even for the goal of increasing the cost of traffic correlation 
attacks. It depends on whether it is easier for an adversary to run many small 
relays of total weight x or a few large relays of total weight y, where x = y*c 
with c the fraction of a Guard-flagged relay used in the guard position (I 
believe that c=2/3 currently, as Wgg=7268 and Wmg=2732). Just to emphasize it: 
waterfilling requires *less bandwidth* to achieve a given guard probability as 
is needed in Tor currently.

Based on prices I’ve seen (~$2/IP/month vs. ~$500/Gbps/month), its 
significantly cheaper to add a new relay than it is to add bandwidth 
commensurate with the highest-bandwidth relays. If an adversary finds it easier 
to compromise machines, then waterfilling might help as it lowers the guard 
probability of high-bandwidth relays. However, for adversaries with the 
resources to posses zero-day vulnerabilities against the well-run 
high-bandwidth relays, it seems to me that those adversaries would easily also 
have the resources to run relays instead, and in fact it would probably be 
cheaper for them to 

Re: [tor-dev] Proposal Waterfilling

[Aaron Johnson]

Hello,

I recently took the time to read the waterfilling paper. I’m not sure its a 
good idea even for the goal of increasing the cost of traffic correlation 
attacks. It depends on whether it is easier for an adversary to run many small 
relays of total weight x or a few large relays of total weight y, where x = y*c 
with c the fraction of a Guard-flagged relay used in the guard position (I 
believe that c=2/3 currently, as Wgg=7268 and Wmg=2732). Just to emphasize it: 
waterfilling requires *less bandwidth* to achieve a given guard probability as 
is needed in Tor currently.

Based on prices I’ve seen (~$2/IP/month vs. ~$500/Gbps/month), its 
significantly cheaper to add a new relay than it is to add bandwidth 
commensurate with the highest-bandwidth relays. If an adversary finds it easier 
to compromise machines, then waterfilling might help as it lowers the guard 
probability of high-bandwidth relays. However, for adversaries with the 
resources to posses zero-day vulnerabilities against the well-run 
high-bandwidth relays, it seems to me that those adversaries would easily also 
have the resources to run relays instead, and in fact it would probably be 
cheaper for them to run relays as zero-days are expensive. Adversaries with 
botnets, which have many IPs but generally low bandwidth, would benefit from 
waterfilling, as it would increase the number of clients choosing them as 
guards that they can then attack. Waterfilling doesn’t clearly make things 
better or worse against network-level adversaries.

Thus, it doesn’t seem to me that waterfilling protects Tor’s users against 
their likely adversaries, and in fact is likely to make things less secure in a 
few important cases.

Best,
Aaron

> On Jan 31, 2018, at 5:01 PM, teor  wrote:
> 
> 
> On 1 Feb 2018, at 07:15, Florentin Rochet  
> wrote:
> 
>>> On 18/01/18 01:03, teor wrote:
>>> 
 I've added this concern within the 'unanswered questions' section. This
 proposal assumes relay measurement are reliable (consensus weight).
>>> How reliable?
>>> 
>>> Current variance is 30% - 40% between identical bandwidth authorities, and
>>> 30% - 60% between all bandwidth authorities.
>>> 
>>> Sources:
>>> https://tomrittervg.github.io/bwauth-tools/#apples-to-apples-comparison
>>> https://tomrittervg.github.io/bwauth-tools/#updated-01
>>> 
>>> Is this sufficient?
>> 
>> My apologies, I was not enough specific: we assume bandwidth
>> measurements reliable as an hypothesis to make the claim that
>> Waterfilling is not going to reduce or improve the performance. If these
>> measurements are not reliable enough, then Waterfilling might make
>> things better, worse or both compared to the current bandwidth-weights
>> is some unpredictable way.
> 
> This variance is measurement error. In this case, discretization error is
> less than 1%.
> 
> We need to know whether measurement inaccuracy makes the network
> weights converge or diverge under your scheme.
> 
> It looks like they converge on the current network with the current
> bandwidth authorities. This is an essential property we need to keep.
> 
>> All of this depends on the bandwidth
>> authority. Anyway, I willingly agree that we need some kind of tools
>> able to report on such modification. Besides, those tools could be
>> reused for any new proposal impacting the path selection, such as
>> research protecting against network adversaries or even some of the
>> changes you already plan to do (such as Prop 276).
> 
> Yes, we are hoping to introduce better tools over time.
> 
>>> 
 …
 
 - The upper bound in (a) is huge, and would be appreciated for an
 adversary running relays. The adversary could manage to set relays with
 almost 2 times the consensus weight of the water level, and still being
 used at 100% in the entry position. This would reduce a lot the benefits
 of this proposal, right?
>>> I do not know how much the benefits of the proposal depend on the exact
>>> water level, and how close relays are to the water level.
>>> 
>>> …
>>> 
>>> How much variance will your proposal tolerate?
>>> Because current variance is 30% - 60% anyway (see above).
>> 
>> The variance is not a problem if the water level is adapted
>> (re-computed) at each consensus.
> 
> I'm not sure we're talking about the same thing here.
> The variance I am talking about here is measurement error and
> discretization error. Re-computation doesn't change the error.
> (And going from relay measurement to consensus bandwidth can take hours.)
> 
> See my comment above about convergence: we need to converge in
> the presence of discretization error, too.
> 
>>> …
>>> 
 With your explanations below (weight change on clients), and given that
 the consensus diff size is a thing, I am leaning to believe that the
 weight calculation should be done on clients. Anyway, I have added a
 remark about this possibility within the proposal.
>>> 

Re: [tor-dev] Proposal Waterfilling

[teor]

Hi,

> On 4 Mar 2018, at 02:37, Florentin Rochet  
> wrote:
> 
> I have a few questions about convergence/divergence of weights, but
> maybe we could take advantage of the meeting in Rome to discuss this avenue?

I was wrong. The current network doesn't attempt to converge
on a stable set of weights, because the feedback loop is too
weak. So I think we can disregard this question.

>> On 28/01/18 23:40, teor wrote:
>> 
>> I'm going to re-ask this questions, in light of the extra middle load from
>> Tor2web clients:
>> 
>> Does the waterfilling proposal make excessive load on middles worse, by
>> allocating more middle weight to higher capacity relays?
> 
> If bwauths overestimate top relays, or if we reach some soft limit, yes.
> But the reverse would be true too: if we have excessive load of guards,
> then this proposal will make things better.

I'm not sure that this statement is true.

Rather:
If we have excessive load on middles, then top relays will suffer more.
If we have excessive load on guards, then lower relays will suffer more.

You can actually lose both ways when you unbalance network load.
It depends on the type of load.

>> In particular, connections are limited by file descriptors, and file
>> descriptor
>> limits typically don't scale with the bandwidth of the relay. As far
>> as I can tell,
>> waterfilling would have directed additional Tor2web traffic to large
>> guards.
>> It would have brought down my guards faster, and made it much harder
>> for me
>> to keep them up.
>> 
>> If we had implemented waterfilling before this attack, would it have
>> lead to
>> cascading failures on our top guards? They would have been carrying
>> significantly more middle load, and mine barely managed to cope.
>> 
> 
> Probably yes, but they would also carrying less load at the guard
> position from normal Tor users. In normal condition, that should tie. In
> a DDoS situation, I would say we face difficulties no matter what we do.

I think that unbalanced networks tend to suffer more during a DDoS.

> …
>> 
>>> Bandwidth-weights and measurements (consensus weights) are two different
>>> things that solve 2 different problems. So, we can work independently on
>>> improving measurements (like what is currently done with bwscanner) and
>>> improving Tor's balancing (bandwidth-weights) with this proposal.
>> 
>> I don't think this is realistic.
>> There is always contention for shared resources.
>> 
>> Integrating and testing new code, and monitoring its effects, will
>> take effort from
>> the teams I mentioned above. This takes away from the urgent work of
>> fixing the
>> bandwidth authority system. Which also takes effort from the Core Tor and
>> Metrics teams.
> 
> Right, totally agree that the first focus should be on bwauths. Could we
> try to make plan, or at least move this proposal into the todo list?

While we're talking priorities, I would also prioritise fixing the
guardfraction code over this proposal. When we work out how to test
guardfraction, we can use similar tests for this proposal.

At the moment, this proposal is "OPEN", because it is still under
discussion:
https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt#n148

If you want to move it to "ACCEPTED":
* answer as many open questions as you can
* reply to this thread with a link to the final version of the proposal
* we will open a ticket for the proposal
* we will schedule a proposal meeting on IRC to discuss the proposal

Please be aware:
* It sometimes takes years for research to make it into Tor.
* Some research is good research, but it is not a good fit for the public
  Tor network.

T

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hi teor,

Sorry about the huge delay :)

I've added your following idea to the proposal (seems we come up to the
right way to do it :-)):

> Why not list the waterfilling level on a single line in the consensus?
>
> That way:
> * authorities do the expensive calculation
> * clients can re-weight relays using a simple calculation:
>
> if it is less than or equal to the waterfilling level:
>   use the relay's weight as its guard weight
>   use 0 as its middle weight
> otherwise:
>   use the waterfilling level as the relay's guard weight
>   use the relay's weight minus the waterfilling level as its middle weight
>
> This is O(n) and requires one comparison and one subtraction in the worst 
> case.

I have a few questions about convergence/divergence of weights, but
maybe we could take advantage of the meeting in Rome to discuss this avenue?

On 28/01/18 23:40, teor wrote:
> 
> I'm going to re-ask this questions, in light of the extra middle load from
> Tor2web clients:
>
> Does the waterfilling proposal make excessive load on middles worse, by
> allocating more middle weight to higher capacity relays?
>

If bwauths overestimate top relays, or if we reach some soft limit, yes.
But the reverse would be true too: if we have excessive load of guards,
then this proposal will make things better.

> In particular, connections are limited by file descriptors, and file
> descriptor
> limits typically don't scale with the bandwidth of the relay. As far
> as I can tell,
> waterfilling would have directed additional Tor2web traffic to large
> guards.
> It would have brought down my guards faster, and made it much harder
> for me
> to keep them up.
>
> If we had implemented waterfilling before this attack, would it have
> lead to
> cascading failures on our top guards? They would have been carrying
> significantly more middle load, and mine barely managed to cope.
>

Probably yes, but they would also carrying less load at the guard
position from normal Tor users. In normal condition, that should tie. In
a DDoS situation, I would say we face difficulties no matter what we do.

> Can you redesign the proposal so there is some limit on the extra
> middle load
> assigned to a guard? Or does this ruin the security properties?
>
> Is there a compelling argument for security over network robustness?
>

Questions added to the proposal :)

> 
>> Could you be
>> more specific about your concerns with the bandwidth authorities and
>> this proposal?
>
> It takes time and effort from Tor people to integrate and maintain the
> code and monitoring for a new proposal like this one.
>
> We will need to take extra time on this proposal, because we already need
> more monitoring for the current bandwidth authority system. And only then
> would we have time to build monitoring specific to this proposal.
>
> Also, when we change bandwidth measurement or allocation, we need to
> change one thing at a time, and then monitor the change. So depending
> on our priorities, this proposal may need to wait until after we implement
> and monitor other urgent fixes.
>

Yes, this proposal can wait, it's totally up to you. I agree that fixing
the current bwauthorities should be somewhat on the top of priorities.
But nothing prevents us to further discuss this proposal and make plans.

>>> Who will maintain the new code we add to Tor to implement waterfilling?
>>
>> I would volunteer to that.
>
> Typically, experienced Core Tor team members review and maintain code.
>
> And there's still a lot of development and testing work to be done before
> the code is ready to merge. Are you able to do this development?
>

If I can make this as a research project, that can be very fast. If I
have to do this on my spare time, that's going to be a bit slower.

> How much help will you need to write a new consensus method?
> How much help will you need to write unit tests?
> (This help will come from existing team members.)
>

I should be fine with both. Adding a new consensus method does not seem
too much difficult at first glance, but removing one looks a bit more
challenging. Hopefully, I just need to add one :)

> Does your current code pass:
> * make check
> * make test-network-all
>   * in particular, any new consensus method must pass the "mixed" network,
>     with an unpatched Tor version in your path as "tor-stable"
>

As much as I recall, make check passed as well as chutney networks. I
don't remember what test I did back at that time, though. If we go for
an up-to-date implementation, I willl make sure everything's ok :)

> 
>> Bandwidth-weights and measurements (consensus weights) are two different
>> things that solve 2 different problems. So, we can work independently on
>> improving measurements (like what is currently done with bwscanner) and
>> improving Tor's balancing (bandwidth-weights) with this proposal.
>
> I don't think this is realistic.
> There is always contention for shared resources.
>
> Integrating and testing new code, and monitoring its 

Re: [tor-dev] Proposal Waterfilling

[teor]

On 1 Feb 2018, at 07:15, Florentin Rochet  wrote:

>> On 18/01/18 01:03, teor wrote:
>> 
>>> I've added this concern within the 'unanswered questions' section. This
>>> proposal assumes relay measurement are reliable (consensus weight).
>> How reliable?
>> 
>> Current variance is 30% - 40% between identical bandwidth authorities, and
>> 30% - 60% between all bandwidth authorities.
>> 
>> Sources:
>> https://tomrittervg.github.io/bwauth-tools/#apples-to-apples-comparison
>> https://tomrittervg.github.io/bwauth-tools/#updated-01
>> 
>> Is this sufficient?
> 
> My apologies, I was not enough specific: we assume bandwidth
> measurements reliable as an hypothesis to make the claim that
> Waterfilling is not going to reduce or improve the performance. If these
> measurements are not reliable enough, then Waterfilling might make
> things better, worse or both compared to the current bandwidth-weights
> is some unpredictable way.

This variance is measurement error. In this case, discretization error is
less than 1%.

We need to know whether measurement inaccuracy makes the network
weights converge or diverge under your scheme.

It looks like they converge on the current network with the current
bandwidth authorities. This is an essential property we need to keep.

> All of this depends on the bandwidth
> authority. Anyway, I willingly agree that we need some kind of tools
> able to report on such modification. Besides, those tools could be
> reused for any new proposal impacting the path selection, such as
> research protecting against network adversaries or even some of the
> changes you already plan to do (such as Prop 276).

Yes, we are hoping to introduce better tools over time.

>> 
>>> …
>>> 
>>> - The upper bound in (a) is huge, and would be appreciated for an
>>> adversary running relays. The adversary could manage to set relays with
>>> almost 2 times the consensus weight of the water level, and still being
>>> used at 100% in the entry position. This would reduce a lot the benefits
>>> of this proposal, right?
>> I do not know how much the benefits of the proposal depend on the exact
>> water level, and how close relays are to the water level.
>> 
>> …
>> 
>> How much variance will your proposal tolerate?
>> Because current variance is 30% - 60% anyway (see above).
> 
> The variance is not a problem if the water level is adapted
> (re-computed) at each consensus.

I'm not sure we're talking about the same thing here.
The variance I am talking about here is measurement error and
discretization error. Re-computation doesn't change the error.
(And going from relay measurement to consensus bandwidth can take hours.)

See my comment above about convergence: we need to converge in
the presence of discretization error, too.

>> …
>> 
>>> With your explanations below (weight change on clients), and given that
>>> the consensus diff size is a thing, I am leaning to believe that the
>>> weight calculation should be done on clients. Anyway, I have added a
>>> remark about this possibility within the proposal.
>> Another alternative is to apply proposal 276 weight rounding to these
>> weights as well.
>> 
>> https://gitweb.torproject.org/torspec.git/tree/proposals/276-lower-bw-granularity.txt
>> 
>> I think this may be our best option. Because running all these divisions on
>> some mobile clients will be very slow and cost a lot of power.
> 
> Added this to the proposal. We might also "divide" the algorithm: what
> about computing the weights on dirauths but broadcasting only the pivot
> (the index of the relay at the water level). Clients can then resume the
> computation and produce the weights themselves with a reduced cost.
> Strength:
>   - The weight calculation would be O(n) on clients (n being the size of
> the guard set) instead of O(n*log(n))
>   - No impact on the consensus diff (well, except 1 line, the pivot value).
> Weakness:
>   - We still have O(n) divisions on the client, each time we download a
> new consensus.

Why not list the waterfilling level on a single line in the consensus?

That way:
* authorities do the expensive calculation
* clients can re-weight relays using a simple calculation:

if it is less than or equal to the waterfilling level:
  use the relay's weight as its guard weight
  use 0 as its middle weight
otherwise:
  use the waterfilling level as the relay's guard weight
  use the relay's weight minus the waterfilling level as its middle weight

This is O(n) and requires one comparison and one subtraction in the worst case.

T
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hi,

I updated the proposal with some more of your advises, questions and
concerns.

On 18/01/18 01:03, teor wrote:
>
>> I've added this concern within the 'unanswered questions' section. This
>> proposal assumes relay measurement are reliable (consensus weight).
> How reliable?
>
> Current variance is 30% - 40% between identical bandwidth authorities, and
> 30% - 60% between all bandwidth authorities.
>
> Sources:
> https://tomrittervg.github.io/bwauth-tools/#apples-to-apples-comparison
> https://tomrittervg.github.io/bwauth-tools/#updated-01
>
> Is this sufficient?

My apologies, I was not enough specific: we assume bandwidth
measurements reliable as an hypothesis to make the claim that
Waterfilling is not going to reduce or improve the performance. If these
measurements are not reliable enough, then Waterfilling might make
things better, worse or both compared to the current bandwidth-weights
is some unpredictable way. All of this depends on the bandwidth
authority. Anyway, I willingly agree that we need some kind of tools
able to report on such modification. Besides, those tools could be
reused for any new proposal impacting the path selection, such as
research protecting against network adversaries or even some of the
changes you already plan to do (such as Prop 276).

> 
>> Wow, so first of all, thanks for suggesting new ways of resolving
>> issues. I really appreciate your consideration for this work.
>>
>> This suggestion indeed reduces the diff problem, and I find it elegant.
>> But, I have the following concerns:
>>
>>  - The upper bound in (a) is huge, and would be appreciated for an
>> adversary running relays. The adversary could manage to set relays with
>> almost 2 times the consensus weight of the water level, and still being
>> used at 100% in the entry position. This would reduce a lot the benefits
>> of this proposal, right?
> I do not know how much the benefits of the proposal depend on the exact
> water level, and how close relays are to the water level.
>
> I chose the exponent "2" as an example that is easy to calculate.
> Smaller exponents are possible.
>
> How much variance will your proposal tolerate?
> Because current variance is 30% - 60% anyway (see above).

The variance is not a problem if the water level is adapted
(re-computed) at each consensus.

> (I think using the median reduces this, but I can't see it going below 40%,
> because that's what identical authorities get.)
>
>>  - The analysis in our paper works for equations we showed. Even if
>> these ones are very similar, I am not sure we can say that it would be
>> good without re-doing the research.
> I do not know enough to answer this.
>
>> With your explanations below (weight change on clients), and given that
>> the consensus diff size is a thing, I am leaning to believe that the
>> weight calculation should be done on clients. Anyway, I have added a
>> remark about this possibility within the proposal.
> Another alternative is to apply proposal 276 weight rounding to these
> weights as well.
>
> https://gitweb.torproject.org/torspec.git/tree/proposals/276-lower-bw-granularity.txt
>
> I think this may be our best option. Because running all these divisions on
> some mobile clients will be very slow and cost a lot of power.

Added this to the proposal. We might also "divide" the algorithm: what
about computing the weights on dirauths but broadcasting only the pivot
(the index of the relay at the water level). Clients can then resume the
computation and produce the weights themselves with a reduced cost.
Strength:
  - The weight calculation would be O(n) on clients (n being the size of
the guard set) instead of O(n*log(n))
  - No impact on the consensus diff (well, except 1 line, the pivot value).
Weakness:
  - We still have O(n) divisions on the client, each time we download a
new consensus.

> 
> Unanswered questions:
>
> What about the feedback loop between this new allocation system
> and the bandwidth authorities?
>
 I am sorry, I don't really understand why a feedback loop is needed. 
 Measuring bandwidth and producing bandwidth-weights seems orthogonal to me.
>>> You do not need to add a feedback loop, one already exists:
>>> 1. Consensus weights on guards and middles change
>>> 2. Client use of guards and middles change
>>> 3. Bandwidth authority measurements of guards and middles change
>>> 4. Repeat from 1
>>>
>>> My question is:
>>>
>>> How does this existing feedback loop affect your proposal?
>>> Does it increase or reduce the size of the guard and middle weight changes?
>> I have added those questions to the proposal. This looks difficult to know.
> Can shadow simulate this?

I am not fully sure about this. Shadow's topology is static, meaning
that a change in advertised bandwidth does not change the uplink and
downlink bandwidth configured by the initial advertised bandwidth. But,
Shadow can use a Torflow plugin and continuously measure the virtual
network, which would 

Re: [tor-dev] Proposal Waterfilling

[teor]

> On 29 Jan 2018, at 08:00, Florentin Rochet  
> wrote:
> 
> Hello,
> 
>> On 28/01/18 11:52, teor wrote:
>> Hi,
>> 
>> I have some more questions:
> 
> Nice, thanks! I still have to answer your previous email and push an
> update to the proposal. I should do it this week, sorry for late answers :)
> 
> See inline a few answers to your questions:
> 
>> 
>> On 18 Jan 2018, at 11:03, teor  wrote:
>> 
>>> Unanswered questions:
>> The Tor network has been experiencing excessive load on guards and 
>> middles since December 2017.
> 
> If I have correctly followed what was happening: around 1M tor2web
> clients appeared at OVH

Not just OVH, at least 3 different providers.

And not just Tor2web, either.
There are onion services which are overloading the network
as well, probably in response to these clients. The onion services
are mostly overloading guard-weighted nodes.

> and started to overload the network with circuit
> creation requests using the old and costly TAP handshake.

Not just TAP. The sheer number of entry connections, extend requests,
and destroy cells is also creating overloads on some relays.

> Tor2web
> clients make direct connections to the intro point and to the rendezvous
> point, right?

Yes.

> And, looking into the code right now, it does not looks
> like Tor2webs make any distinction to flags. So, basically, the Tor2web
> load is only weighted by consensus weight (bandwidth-weights have no
> impact) on the overall network (exits too).

This only applies if Tor2webRendezvousPoints is set.
Otherwise, the nodes are middle-weighted.

> Guess: shouldn't that the reason why all exits logs are flooded with the
> message "[warn] Tried to establish rendezvous on non-OR circuit with
> purpose Acting as rendevous (pending)"? Those messages would be caused
> by tor2web clients picking exit relays as rendezvous node :/ I started
> to see them increasing more and more since August 2017.

No, this is a different issue.
Exit relays are allowed as rendezvous nodes.

> So basically, I *think* we can drop the questions below because
> bandwidth-weights do not play any role in the excessive load that the
> network is handling with those tor2webs.

Guard weights are used by overloading onion services, and middle
weights are used by overloading Tor2web clients.

>> Does the waterfilling proposal make excessive load on guards worse, by
>> allocating more guard weight to lower capacity relays?
>> Is the extra security worth the increased risk of failure?

We want to design a network that can handle different kinds of extra load.
So these questions are important, even if they don't apply right now.

>> Does the waterfilling proposal make excessive load on middles better, by
>> allocating more middle weight to higher capacity relays?
>> Is there a cascading failure mode, where excess middle weight overwhelms
>> our top relays one by one? (It seems unlikely.)

I'm going to re-ask this questions, in light of the extra middle load from
Tor2web clients:

Does the waterfilling proposal make excessive load on middles worse, by
allocating more middle weight to higher capacity relays?

In particular, connections are limited by file descriptors, and file descriptor
limits typically don't scale with the bandwidth of the relay. As far as I can 
tell,
waterfilling would have directed additional Tor2web traffic to large guards.
It would have brought down my guards faster, and made it much harder for me
to keep them up.

If we had implemented waterfilling before this attack, would it have lead to
cascading failures on our top guards? They would have been carrying
significantly more middle load, and mine barely managed to cope.

Can you redesign the proposal so there is some limit on the extra middle load
assigned to a guard? Or does this ruin the security properties?

Is there a compelling argument for security over network robustness?

>> I also have another practical question:
>> 
>> We struggle to have time to maintain the current bandwidth authority
>> system.
>> 
>> Is it a good idea to make it more complicated?
> 
> Hm, I don't see how Waterfilling plays any role with torflow or
> bwscanner? I mean, there is still this feedback loop thing but it has no
> impact on the design of the current torflow or bwscanner?

I can't really say.
I look forward to your explanation of the feedback loop. 

> Could you be
> more specific about your concerns with the bandwidth authorities and
> this proposal?

It takes time and effort from Tor people to integrate and maintain the
code and monitoring for a new proposal like this one.

We will need to take extra time on this proposal, because we already need
more monitoring for the current bandwidth authority system. And only then
would we have time to build monitoring specific to this proposal.

Also, when we change bandwidth measurement or allocation, we need to
change one thing at a time, and then monitor the change. So depending

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hello,


On 28/01/18 11:52, teor wrote:
> Hi,
>
> I have some more questions:

Nice, thanks! I still have to answer your previous email and push an
update to the proposal. I should do it this week, sorry for late answers :)

See inline a few answers to your questions:

>
> On 18 Jan 2018, at 11:03, teor  wrote:
>
>> Unanswered questions:
> The Tor network has been experiencing excessive load on guards and 
> middles since December 2017.

If I have correctly followed what was happening: around 1M tor2web
clients appeared at OVH and started to overload the network with circuit
creation requests using the old and costly TAP handshake. Tor2web
clients make direct connections to the intro point and to the rendezvous
point, right? And, looking into the code right now, it does not looks
like Tor2webs make any distinction to flags. So, basically, the Tor2web
load is only weighted by consensus weight (bandwidth-weights have no
impact) on the overall network (exits too).
Guess: shouldn't that the reason why all exits logs are flooded with the
message "[warn] Tried to establish rendezvous on non-OR circuit with
purpose Acting as rendevous (pending)"? Those messages would be caused
by tor2web clients picking exit relays as rendezvous node :/ I started
to see them increasing more and more since August 2017.

So basically, I *think* we can drop the questions below because
bandwidth-weights do not play any role in the excessive load that the
network is handling with those tor2webs.


> Does the waterfilling proposal make excessive load on guards worse, by
> allocating more guard weight to lower capacity relays?
> Is the extra security worth the increased risk of failure?
>
> Does the waterfilling proposal make excessive load on middles better, by
> allocating more middle weight to higher capacity relays?
> Is there a cascading failure mode, where excess middle weight overwhelms
> our top relays one by one? (It seems unlikely.)
>
>
> I also have another practical question:
>
> We struggle to have time to maintain the current bandwidth authority
> system.
>
> Is it a good idea to make it more complicated?

Hm, I don't see how Waterfilling plays any role with torflow or
bwscanner? I mean, there is still this feedback loop thing but it has no
impact on the design of the current torflow or bwscanner? Could you be
more specific about your concerns with the bandwidth authorities and
this proposal?

> Who will maintain the new code we add to Tor to implement waterfilling?

I would volunteer to that.

> Who will build the analysis tools to show that waterfilling benefits the
> network?

Volunteers or master students. I can definitely suggest this topic in my
university.

> Do the benefits of waterfilling justify this extra effort?

Question for the other Tor devs :) I am definitely biased towards the "yes"

> And even if they do, should we focus on getting the bandwidth authorities
> in a maintainable state, before adding new features?
> (I just gave similar advice to another developer who has some great ideas
> about improving bandwidth measurement.)
>

Bandwidth-weights and measurements (consensus weights) are two different
things that solve 2 different problems. So, we can work independently on
improving measurements (like what is currently done with bwscanner) and
improving Tor's balancing (bandwidth-weights) with this proposal.

>> What about the feedback loop between this new allocation system
>> and the bandwidth authorities?
>>
> I am sorry, I don't really understand why a feedback loop is needed. 
> Measuring bandwidth and producing bandwidth-weights seems orthogonal to 
> me.
 You do not need to add a feedback loop, one already exists:
 1. Consensus weights on guards and middles change
 2. Client use of guards and middles change
 3. Bandwidth authority measurements of guards and middles change
 4. Repeat from 1

 My question is:

 How does this existing feedback loop affect your proposal?
 Does it increase or reduce the size of the guard and middle weight changes?
>>> I have added those questions to the proposal. This looks difficult to know.
>> Can shadow simulate this?
> I am still interested in this feedback loop.
> If it fails to converge, the system will break down.

Yup. Going to answer this on your previous email.

Best,
Florentin

>
> T
> ___
> tor-dev mailing list
> tor-dev@lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[teor]

Hi,

I have some more questions:

On 18 Jan 2018, at 11:03, teor  wrote:

> Unanswered questions:

The Tor network has been experiencing excessive load on guards and 
middles since December 2017.

Does the waterfilling proposal make excessive load on guards worse, by
allocating more guard weight to lower capacity relays?
Is the extra security worth the increased risk of failure?

Does the waterfilling proposal make excessive load on middles better, by
allocating more middle weight to higher capacity relays?
Is there a cascading failure mode, where excess middle weight overwhelms
our top relays one by one? (It seems unlikely.)


I also have another practical question:

We struggle to have time to maintain the current bandwidth authority
system.

Is it a good idea to make it more complicated?
Who will maintain the new code we add to Tor to implement waterfilling?
Who will build the analysis tools to show that waterfilling benefits the
network?

Do the benefits of waterfilling justify this extra effort?

And even if they do, should we focus on getting the bandwidth authorities
in a maintainable state, before adding new features?
(I just gave similar advice to another developer who has some great ideas
about improving bandwidth measurement.)

> What about the feedback loop between this new allocation system
> and the bandwidth authorities?
> 
 I am sorry, I don't really understand why a feedback loop is needed. 
 Measuring bandwidth and producing bandwidth-weights seems orthogonal to me.
>>> You do not need to add a feedback loop, one already exists:
>>> 1. Consensus weights on guards and middles change
>>> 2. Client use of guards and middles change
>>> 3. Bandwidth authority measurements of guards and middles change
>>> 4. Repeat from 1
>>> 
>>> My question is:
>>> 
>>> How does this existing feedback loop affect your proposal?
>>> Does it increase or reduce the size of the guard and middle weight changes?
>> 
>> I have added those questions to the proposal. This looks difficult to know.
> 
> Can shadow simulate this?

I am still interested in this feedback loop.
If it fails to converge, the system will break down.

T
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[teor]

> On 18 Jan 2018, at 08:10, Florentin Rochet  
> wrote:
> ...
>> It might help to provide a procedural algorithm for assigning
>> bandwidth, as well as equations. It would resolve some ambiguity.
> 
> Not sure to understand what 'bandwidth' signifies here. I can provide a
> procedural algorithm to compute the waterfilling bandwidth-weights, if
> this is what you meant?

Yes.

> ...
> 
 ...
 
> 2.1 Going further by tweaking original bandwidth-weights computation
> 
>  As explained above, our Waterfilling equations are based on: 1) the
>  Wgg weight computed by Tor 2) the assumption that the bandwidth
>  available in exit is scarce, i.e., it is lower than the one
>  available for guard (and middle) positions.
> 
>  The second point is satisfied most of the time in Tor, and we do take
>  it for granted here.
> 
>  We, however, observe that Waterfilling could be made even more
>  effective by applying a minor change in the way Tor computes the
>  Wgg.  For the moment, Tor computes Wgg in such a way that the same
>  bandwidth is allocated to the guard and middle positions. As a
>  result, both positions are in excess compared to the exit position.
> 
>  The water level could be decreased and, as a result, the uniformity
>  of the guard selection process could be improved, by computing Wgg
>  in a way that allocates the same bandwidth to the guard and exit
>  positions, putting the middle position as the only position in
>  excess.
 No, this would slow down Guards, and everything that isn't an exit circuit:
 * directory fetches (3% of total bandwidth to guard position)
>> If we reduce guard weights on large relays, does this slow down Tor 
>> bootstrap?
>> (Because smaller relays with higher latency get more micro descriptor 
>> requests.)
>> 
>> Does it slow down directory fetches in general?
> 
> Assuming smallest guards have an higher latency in average, this could
> indeed slow down directory fetches and then, maybe it would slow down
> Tor bootstrapping. I guess that measuring this with Shadow is possible,
> as well as collecting statistics on a small set of deployed Tor clients.

This would be helpful.
Although the overall effect might not be measurable.
If it is, we could use the unmodified guard weights for directory fetches.

 * onion services (rend is 4% of total bandwidth to guard and middle)
 * HSDir is unweighted, and we don't know how much bandwidth it uses
 * FallbackDir is unweighted, but mostly Guards, and small
 
>>> That's difficult to predict, I cannot be sure if it is better or worse for 
>>> that type of traffic since internal circuits use at least 2 middle relays + 
>>> the guard and sometimes, even not the guard. Hence we might also think that 
>>> pushing a bit more to the middle position could be a good thing to do. 
>>> Moreover, middle relays are unstable and often at the edge of the internet, 
>>> while guard are stable and most of them within the core of the internet. 
>>> Hence, a little more of them within the middle position *could* be a good 
>>> thing, especially if it makes entry's selection probability more uniform. 
>>> Anyway, I don't have any proof to assert this, as well that I don't have 
>>> any proof to assert that this optimization could be bad. What I got, is 
>>> that, for exit circuits, it does not slow down anything.
>>> 
>>> This optimization is not mandatory, and could also be enabled/disabled at 
>>> will by the directory auths.
>>> 
>  We show in the performance section of the Waterfilling paper that
>  scarcity on two positions does not reduce performance compared to
>  vanilla bandwidth-weights.
 What about guards that have low consensus weight due to latency,
 rather than available bandwidth?
 
 I think this could also cause you huge latency issues as you push more
 bandwidth away from fast relays. I'm not sure if shadow captures this
 accurately.
 
>>> If it happens that any bandwidth is pushed away from fast relays within the 
>>> entry position and make the entry position slower, at average, then it will 
>>> make the middle position faster (because it got that bandwidth pushed 
>>> away). Since the latency of your traffic flow just care about the global 
>>> latency of the circuit, this will not appear to be slower or faster, on 
>>> average. This is exactly what we observe in Shadow, and yes, it captures 
>>> latency accurately. At least, better than any other simulator.
>> I am also concerned that this proposal may overload small guards and
>> underload large guards. Because it does not consider advertised
>> bandwidth.
> 
> I've added this concern within the 'unanswered questions' section. This
> proposal assumes relay measurement are reliable (consensus weight).

How reliable?

Current variance is 30% - 40% between identical bandwidth authorities, and
30% - 60% 

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hello,

Thank you again for your valuable comments and efforts for improving!

I have added the proposal here: https://github.com/frochet/wf_proposal.
And, btw, the slides from the talk might help to understand the proposal
(for anyone that hesitates to dive into it: You can browse the slide at
http://ndouuqkqdsd6v56h.onion/and get the big picture. Especially from
slides 14.1, 14.2, .3, .4 and .5).


On 15/01/18 03:26, teor wrote:
>> On 13 Jan 2018, at 01:17, Florentin Rochet  
>> wrote:
>>
>> Hello,
>>
>> Thank you for your helpful review, teor.
>>
>> I updated the proposal from most of your comments (see attached .txt) and I 
>> respond inline to add some precisions relative to a few of your questions.
>>
>> Btw, I've mirrored my private repo to github 
>> https://github.com/frochet/Waterfilling, such that you have the proper 
>> commit history.
> Thanks!
>
> I replied below in context.
>
>> On 2018-01-11 14:47, teor wrote:
>>> Hi Florentin,
>>>
>>> I have copied your proposal below, so I can respond to it inline.
>>> ...
   Hence, in addition to select paths in a probabilistic manner, the
   path selection algorithm is responsible for balancing the network,
   that is, making sure that enough bandwidth is made available in all
   the positions.  The current balance of the network is decided by the
   bandwidth-weights (see dir-spec.txt section 3.8.3.  or/and the
   Waterfilling PETS2017 paper
   https://petsymposium.org/2017/papers/issue2/paper05-2017-2-source.pdf).
   This balance does not achieve maximum diversity in end-positions of
   user paths: the same network throughput could be achieved by
   decreasing the use of high-bandwidth relays and increasing the use
   of lower-bandwidth relays in the guard position, instead of using
   these relays in a way that is just proportional to their bandwidth.
>>> When you say "bandwidth", it's not clear whether you mean consensus
>>> weight (measured bandwidth) or advertised bandwidth (bandwidth
>>> capacity). They're not the same.
>>>
>>> I'm going to assume consensus weight from now on.
>>> Please fix all uses of bandwidth in the rest of the proposal.
>> Yes, we mean consensus weight :) I did s/bandwidth/consensus weight within 
>> the proposal
> In some cases, you seem to expect the consensus weight to be the available
> bandwidth (or advertised bandwidth) of the relay. I'll try to find them,
> but you should check as well.


Ok, I will look into it. If you see some, let me know :)

>>> ...
>>> 
 Compute a "pivot" N and the weight Wgg_i assigned to
   relay_i in such a way that:

  (a) Wgg_i * BW_i == Wgg_i+1 * BW_i+1 forall i in [1, N]
>>> subscripts in brackets, otherwise it's ambiguous:
>>>
 Wgg_(i+1) * BW_(i+1)
  (b) Wgg_i == 1 forall i in [N+1, K]
  (c) sum_{i=1}^{K} Wgg_i*BW_i = Wgg*G  (Wgg is provided by Tor)
>>> These equations are under-specified, because they allow solutions with:
>>> Wgg*G > 0
>>> Wgg_1 == 0
>>> That is, no guard selections for high-bandwidth relays.
> This is a really important point:
>
> Your equations allow the largest guard to have zero guard weighting and zero
> guard bandwidth. They should require it to have at least as much guard
> bandwidth as the water level.
>
>>> From the descriptions, I think the missing condition is:
>>> Wgg_N * BW_N >= Wgg_(N+1) * BW_(N+1)
>> Yes, this can be added. But I think that this condition is redundant, since 
>> BW_i are sorted in decreasing order.
> No, it's not, because it's a condition on the *guard-weighted* consensus
> weights. The sorting is a condition on the *original* consensus weights.


I have added the condition, thanks :)

> It might help to provide a procedural algorithm for assigning
> bandwidth, as well as equations. It would resolve some ambiguity.

Not sure to understand what 'bandwidth' signifies here. I can provide a
procedural algorithm to compute the waterfilling bandwidth-weights, if
this is what you meant?

>>> Also, Wgg is provided by the Tor directory authorities based on
>>> consensus weights from the bandwidth authorities.
>>>
>>> And what happens to any remainder in the calculations?
>> This is a very good question. Currently in my implementation, I ignore the 
>> remainder. This is negligible for large network but can be weird for small 
>> one (of a few relays).
>>
>> A possible solution would be to use floating precision for consensus weights.
> No, this is not a good solution. It simply passes the issue to the
> floating-point implementation. And the result will depend on the
> libraries, processor, and compiler.
>
> For this reason, we work very hard to use as little floating point as
> possible in Tor. We really don't want to use it when generating the
> consensus, because mismatches will break consensus.
>
> Instead, I suggest you apply this missing condition:
>
>>> From the descriptions, I think the missing condition is:
>>> Wgg_N * 

Re: [tor-dev] Proposal Waterfilling

[teor]

> On 13 Jan 2018, at 01:17, Florentin Rochet  
> wrote:
> 
> Hello,
> 
> Thank you for your helpful review, teor.
> 
> I updated the proposal from most of your comments (see attached .txt) and I 
> respond inline to add some precisions relative to a few of your questions.
> 
> Btw, I've mirrored my private repo to github 
> https://github.com/frochet/Waterfilling, such that you have the proper commit 
> history.

Thanks!

I replied below in context.

> On 2018-01-11 14:47, teor wrote:
>> Hi Florentin,
>> 
>> I have copied your proposal below, so I can respond to it inline.
>> ...
> 
> 
>>>   Hence, in addition to select paths in a probabilistic manner, the
>>>   path selection algorithm is responsible for balancing the network,
>>>   that is, making sure that enough bandwidth is made available in all
>>>   the positions.  The current balance of the network is decided by the
>>>   bandwidth-weights (see dir-spec.txt section 3.8.3.  or/and the
>>>   Waterfilling PETS2017 paper
>>>   https://petsymposium.org/2017/papers/issue2/paper05-2017-2-source.pdf).
>>>   This balance does not achieve maximum diversity in end-positions of
>>>   user paths: the same network throughput could be achieved by
>>>   decreasing the use of high-bandwidth relays and increasing the use
>>>   of lower-bandwidth relays in the guard position, instead of using
>>>   these relays in a way that is just proportional to their bandwidth.
>> 
>> When you say "bandwidth", it's not clear whether you mean consensus
>> weight (measured bandwidth) or advertised bandwidth (bandwidth
>> capacity). They're not the same.
>> 
>> I'm going to assume consensus weight from now on.
>> Please fix all uses of bandwidth in the rest of the proposal.
> 
> Yes, we mean consensus weight :) I did s/bandwidth/consensus weight within 
> the proposal

In some cases, you seem to expect the consensus weight to be the available
bandwidth (or advertised bandwidth) of the relay. I'll try to find them,
but you should check as well.

>> ...
> 
>> 
>>> Compute a "pivot" N and the weight Wgg_i assigned to
>>>   relay_i in such a way that:
>>> 
>>>  (a) Wgg_i * BW_i == Wgg_i+1 * BW_i+1 forall i in [1, N]
>> 
>> subscripts in brackets, otherwise it's ambiguous:
>> 
>>> Wgg_(i+1) * BW_(i+1)
>> 
>>>  (b) Wgg_i == 1 forall i in [N+1, K]
>>>  (c) sum_{i=1}^{K} Wgg_i*BW_i = Wgg*G  (Wgg is provided by Tor)
>> 
>> These equations are under-specified, because they allow solutions with:
>> Wgg*G > 0
>> Wgg_1 == 0
>> That is, no guard selections for high-bandwidth relays.

This is a really important point:

Your equations allow the largest guard to have zero guard weighting and zero
guard bandwidth. They should require it to have at least as much guard
bandwidth as the water level.

>> From the descriptions, I think the missing condition is:
>> Wgg_N * BW_N >= Wgg_(N+1) * BW_(N+1)
> 
> Yes, this can be added. But I think that this condition is redundant, since 
> BW_i are sorted in decreasing order.

No, it's not, because it's a condition on the *guard-weighted* consensus
weights. The sorting is a condition on the *original* consensus weights.

It might help to provide a procedural algorithm for assigning
bandwidth, as well as equations. It would resolve some ambiguity.

>> Also, Wgg is provided by the Tor directory authorities based on
>> consensus weights from the bandwidth authorities.
>> 
>> And what happens to any remainder in the calculations?
> 
> This is a very good question. Currently in my implementation, I ignore the 
> remainder. This is negligible for large network but can be weird for small 
> one (of a few relays).
> 
> A possible solution would be to use floating precision for consensus weights.

No, this is not a good solution. It simply passes the issue to the
floating-point implementation. And the result will depend on the
libraries, processor, and compiler.

For this reason, we work very hard to use as little floating point as
possible in Tor. We really don't want to use it when generating the
consensus, because mismatches will break consensus.

Instead, I suggest you apply this missing condition:

>> From the descriptions, I think the missing condition is:
>> Wgg_N * BW_N >= Wgg_(N+1) * BW_(N+1)

And then allocate the remainder to the highest-bandwidth relays, perhaps
in proportion to their consensus weight. Please allocate bandwidth to
equally weighted relays using a stable sort that is the same on all
directory authorities. (The existing relay sorting code might already do
this for you.)

>> (This is most important for small, low bandwidth test networks.)
>> 
>> For example, if:
>> G = 10
>> Wgg = 0.6
>> BW_i = 6, 2, 2
>> What are the final weighted bandwidths?
>> 2, 2, 2?
> 
> From my note, my current implementation would crash if the water level 
> reaches the smallest relay. Since it was prototype code, I didn't mind to 
> think about it, and I let it that way.

We would need a solution for this crash as part of 

Re: [tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hello,

Thank you for your helpful review, teor.

I updated the proposal from most of your comments (see attached .txt) 
and I respond inline to add some precisions relative to a few of your 
questions.


Btw, I've mirrored my private repo to github 
https://github.com/frochet/Waterfilling, such that you have the proper 
commit history.


On 2018-01-11 14:47, teor wrote:

Hi Florentin,

I have copied your proposal below, so I can respond to it inline.

On 11 Jan 2018, at 21:00, Florentin Rochet 
> 
wrote:



1  Overview

  The current Tor path selection algorithm is designed to satisfy two
  important properties:

  1) Each relay should handle a fraction of connections that is
  proportional to its bandwidth.

  2) The nodes in each circuit position (entry-middle-exit) should
  be able to handle the same volume of connections.

What about onion service circuits?

They consist of entry - middle - middle, and for the purposes of this
analysis, make up about 4% of the network.
(2% of traffic at rend points, going through 2 x 3-hop circuits.)

https://metrics.torproject.org/hidserv-rend-relayed-cells.html



That's a good point. Waterfilling uses the current bandwidth-weights 
logic as a basis and they doesn't account for onion service circuit, 
hence it also ignore this sort of traffic. Prop 265 tries to address 
that problem when producing the bandwidth-weights; Since our method 
achieves the same total consensus weight balance between position as the 
one produced by bandwidth-weights, Waterfilling would directly inherit 
Prop 265's properties if this proposal is merged.




  Hence, in addition to select paths in a probabilistic manner, the
  path selection algorithm is responsible for balancing the network,
  that is, making sure that enough bandwidth is made available in all
  the positions.  The current balance of the network is decided by the
  bandwidth-weights (see dir-spec.txt section 3.8.3.  or/and the
  Waterfilling PETS2017 paper
https://petsymposium.org/2017/papers/issue2/paper05-2017-2-source.pdf).
  This balance does not achieve maximum diversity in end-positions of
  user paths: the same network throughput could be achieved by
  decreasing the use of high-bandwidth relays and increasing the use
  of lower-bandwidth relays in the guard position, instead of using
  these relays in a way that is just proportional to their bandwidth.


When you say "bandwidth", it's not clear whether you mean consensus
weight (measured bandwidth) or advertised bandwidth (bandwidth
capacity). They're not the same.

I'm going to assume consensus weight from now on.
Please fix all uses of bandwidth in the rest of the proposal.


Yes, we mean consensus weight :) I did s/bandwidth/consensus weight 
within the proposal






2  Design

  Correlation attacks require to control guard and exit nodes, but the
  scarcity of exit bandwidth is such that there is no real freedom in
  the way to use it.

As a result, the Waterfilling proposal focuses
  on the guard position. However, it could be easily extended to the
  exit position if, someday, nodes in that position happen not to be
  exploited to their full capacity.


No, exit bandwidth is only exploited to its full capacity on
high-bandwidth exits in the northern EU and North America.



Well, I changed my wording here to avoid ambiguity. I was talking about 
relays flagged as Exits being used only in the exit position (Wee and 
Wed have the max value), which means that we cannot apply our method 
over those relays with the current state of the Tor network.



Select "Consensus Weight vs Bandwidth" on this map:
https://atlas.torproject.org/#map/flag:Exit
All the exits in all the purple countries are probably under-loaded.
And some exits elsewhere are under-loaded.
(That's why we let Exits be fallback directories.)

So the network might actually benefit the most from a reallocation
of Exit bandwidth. But you'd have to use the advertised bandwidth
rather than Wee and Wed.

What would it look like if we used waterfilling on the advertised
bandwidths of Exits?


The problem your mention is more a measurement problem that would not 
exist if the bwauths were perfect, within a perfect network. I believe 
that research such as Peerflow[0] is the right path to track down such 
issue, and this is compatible to our proposal.


[0] www.robgjansen.com/publications/peerflow-popets2017.pdf

Is there a way to do this that avoids gaming the system by
increasing advertised bandwidth?
Does the feedback loop with bandwidth authority measurements
mitigate this risk?


  _Recall_: Tor currently computes bandwidth-weights in order to balance
  the bandwidth made available by nodes between the different path
  positions. In particular the Wgg weight indicates to each guard which
  proportion of its bandwidth should be used for entry traffic (the rest
  being normally devoted to the middle position).  This proportion is
  the 

Re: [tor-dev] Proposal Waterfilling

[Justin Tracey]

On 2018-01-11 08:47 AM, teor wrote:
> What about guards that have low consensus weight due to latency,
> rather than available bandwidth?
> 
> I think this could also cause you huge latency issues as you push more
> bandwidth away from fast relays. I'm not sure if shadow captures this
> accurately.

FWIW, Shadow does accurately reflect latencies, assuming you supply a
realistic network topology file and used relatively recent Tor metrics.

https://github.com/shadow/shadow/wiki/3.2-Network-Config
https://github.com/shadow/shadow-plugin-tor/wiki#generating-a-new-tor-network

 - Justin
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal Waterfilling

[teor]

Hi Florentin,

I have copied your proposal below, so I can respond to it inline.

> On 11 Jan 2018, at 21:00, Florentin Rochet  
> wrote:
> 
> Hello everyone,
> 
> In order that our paper does not fall into the list of "yet another 
> seems-to-be-cool-feature is that never going to be discussed because 
> researchers moved on another topic", here is an attached proposal in which we 
> summarize our work published in the last PETS event and how it can be 
> implemented.
> 
> And, if you find this interesting, I would be glad to submit a patch :)
> 
> Any kind of feedbacks is more than welcome!
> 
> Cheers!
> 
> Florentin
> 
> Filename: waterfilling-balancing-with-max-diversity.txt
> Title: Waterfilling
> Authors: Florentin Rochet and Olivier Pereira
> Reviewed by (thanks!): George Kadianakis, Edouard Cuvelier
> Created:  Jan 2018
> Status: Open  
> 
> 0  Motivation
> 
>   An adversary who monitors the entry and exit of a Tor communication
>   path is usually able to de-anonymize that communication by traffic
>   correlation. In order to limit the number of users that a single
>   corrupted entry node could attack, the users keep using the same entry
>   node, also called a "guard" for long periods of time: since guard
>   rotation is limited, the users are less likely to use a corrupted
>   guard at some point in their communication. In the current design, the
>   amount of traffic that a given guard sees is directly proportional to
>   the bandwidth that is provided by this guard. As a result, the few
>   guards offering the highest amount of bandwidth become very attractive
>   targets for an attacker.
> 
>   Waterfilling is a new path selection mechanism designed to make the
>   guard selection even more efficient: if an adversary wants to profile
>   more users, she has to increase her bandwidth _and_ increase the
>   number of relays injected/hacked into the network.
> 
>   Waterfilling mitigates the risks of end-to-end traffic correlation
>   by balancing the load as evenly as possible on endpoints of user
>   circuits. More precisely, waterfilling modifies the probability
>   distribution according to which users select guard nodes my making
>   that distribution closer to the uniform distribution, without
>   impacting the performance of the Tor network.
> 
> 1  Overview
> 
>   The current Tor path selection algorithm is designed to satisfy two
>   important properties:
> 
>   1) Each relay should handle a fraction of connections that is
>   proportional to its bandwidth.
> 
>   2) The nodes in each circuit position (entry-middle-exit) should
>   be able to handle the same volume of connections.

What about onion service circuits?

They consist of entry - middle - middle, and for the purposes of this
analysis, make up about 4% of the network.
(2% of traffic at rend points, going through 2 x 3-hop circuits.)

https://metrics.torproject.org/hidserv-rend-relayed-cells.html

>   Hence, in addition to select paths in a probabilistic manner, the
>   path selection algorithm is responsible for balancing the network,
>   that is, making sure that enough bandwidth is made available in all
>   the positions.  The current balance of the network is decided by the
>   bandwidth-weights (see dir-spec.txt section 3.8.3.  or/and the
>   Waterfilling PETS2017 paper
>   https://petsymposium.org/2017/papers/issue2/paper05-2017-2-source.pdf).
>   This balance does not achieve maximum diversity in end-positions of
>   user paths: the same network throughput could be achieved by
>   decreasing the use of high-bandwidth relays and increasing the use
>   of lower-bandwidth relays in the guard position, instead of using
>   these relays in a way that is just proportional to their bandwidth.

When you say "bandwidth", it's not clear whether you mean consensus
weight (measured bandwidth) or advertised bandwidth (bandwidth
capacity). They're not the same.

I'm going to assume consensus weight from now on.
Please fix all uses of bandwidth in the rest of the proposal.

>   Such a change would make top relays less attractive targets to
>   adversaries, and would increase the number of relays that need to be
>   compromised in order to obtain a given probability of mounting a
>   successful correlation attack.
> 
>   Our proposal only modifies the balance between the relays in a given
>   position in the network. It does not modify, and actually takes as
>   its starting point, any allocation mechanism used to decide the
>   bandwidth that is allocated in guard, middle and exit positions.  As
>   a consequence, the changes that we propose are quite minimal in
>   terms of code base and performance and, in particular, they do not
>   interfere with prop 265.
> 
> 2  Design
> 
>   Correlation attacks require to control guard and exit nodes, but the
>   scarcity of exit bandwidth is such that there is no real freedom in
>   the way to use it.
> 
>