Re: [tor-relays] Exit relay operators please help test #2667 branch

[Roger Dingledine]

On Thu, Feb 04, 2021 at 01:20:58AM +0200, s7r wrote:
> Indeed the defense is triggered more often than I expected. Very nice.

Btw, a better version of the #2667 patch is now included in all of the
current Tor releases: 0.3.5.13, 0.4.3.8, 0.4.4.7, 0.4.5.5-rc.

So if you are still trying out my experiment patch, thank you, but now
please stop doing that and move to one of the actual releases. :)

> This defense is only implemented at the Exit relay, and it blocks all the
> relay:ORPort / DirPort combinations that exit knows about according to its
> version of the consensus?

The design that we decided on was "block relay:ORPort, dirauth:ORPort,
and dirauth:DirPort". That is, it doesn't try to block relay:DirPort
connections.

We chose that balance because if we'd blocked all relay dirports too,
we would have broken relay dirport reachability tests, because the
way those tests work is by building an exit circuit and then "exiting"
back to the relay's dirport to see if it works.

Ultimately I expect we're going to phase out the concept of a dirport
on non-dir-auth relays, since they mostly go unused so it's just yet
another thing to maintain that isn't worth it. But now we can do that
phase-out completely separate from this topic.

> What happens if the abuser has a different valid consensus (newer/older)
> that has some relays which are not known by the Exit?

If there are edge cases where different sides have different knowledge,
then those edge cases will 'get through'. But hopefully they'll be rare,
or even if they're not rare, hopefully they will be low-impact.

> Or if the abuser uses
> for re-entry a relay configured with `PublishServerDescriptor 0`? How does
> it treat bridges?

They are all allowed.

In fact, one of the Tor use cases we're going to break here is people
who torify all their traffic (at the network or host level) and then
run Tor Browser also. The suggested workaround for them is that if they
really need to do this, they should configure their Tor Browser to use
a bridge. There will be a bunch of confused users who don't know they're
doing this, and don't know why it broke, though.

I've been working on
https://gitlab.torproject.org/tpo/core/tor/-/issues/40271
to do a little bit toward that explanation side.

> And most important, does this means that an Exit will no longer try to
> connect to other middle relay? At this moment can't an exit relay (of course
> with a small probability) be used as a middle or entry? Won't this become a
> mess if we have 80% of relays Exits and thus used more as 1st or 2nd hops,
> or in a perfect future where 100% or relays are all exits?

No, exiting is different from extending. Exit relays can still extend
to other relays -- this is a part of the Tor protocol that lets a relay
make a TLS connection with another relay. The difference is in who the
"ends" of that TLS connection are: in the extend case, the ends are the
two relays. In the 'exit and reenter' case, one end is the client and
the other is the destination relay.

--Roger

___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

Re: [tor-relays] Exit relay operators please help test #2667 branch

[s7r]

Roger Dingledine wrote:

Hello friendly relay operators,

Another day, another weird thing with the Tor network. This time we
have some jerk bombing the directory authorities with directory fetches,
and doing it via exits:
https://lists.torproject.org/pipermail/network-health/2021-January/000661.html

The network is mostly holding together, but I wouldn't say it is pretty.

One of the long-term fixes will be ticket #2667:
https://gitlab.torproject.org/tpo/core/tor/-/issues/2667
where exit relays refuse to let users connect back into the Tor network.

David and I made a branch this evening that implements #2667, and it
could use some testing. If you're comfortable building your exit relay
from a git branch, please do, and let us know how it goes. It is the
"ticket2667" branch on either
https://git.torproject.org/user/arma/tor
or
https://gitlab.torproject.org/arma/tor/

And if your relay is currently using 100% cpu and/or way more bandwidth
than usual, you might be especially excited to try out this patch. :)

When the defense triggers, you will see an info-level log line like
"%s tried to connect back to a known relay address. Closing."
(where %s is the destination, so don't get upset at them. :)

You can let us know how it's going either by mail just to me, or by a
reply on the list, whichever you prefer. Once we know that you're running
the branch, we can also probe your relay remotely to verify that it is
correctly refusing those connections.

Thanks!
--Roger



Hello,

Indeed the defense is triggered more often than I expected. Very nice.

I tried to read on that ticket that's a decade old but didn't understand 
what was the final resolution.


This defense is only implemented at the Exit relay, and it blocks all 
the relay:ORPort / DirPort combinations that exit knows about according 
to its version of the consensus?


What happens if the abuser has a different valid consensus (newer/older) 
that has some relays which are not known by the Exit? Or if the abuser 
uses for re-entry a relay configured with `PublishServerDescriptor 0`? 
How does it treat bridges?


And most important, does this means that an Exit will no longer try to 
connect to other middle relay? At this moment can't an exit relay (of 
course with a small probability) be used as a middle or entry? Won't 
this become a mess if we have 80% of relays Exits and thus used more as 
1st or 2nd hops, or in a perfect future where 100% or relays are all exits?




OpenPGP_signature
Description: OpenPGP digital signature
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

Re: [tor-relays] Exit relay operators please help test #2667 branch

[nusenu]

Roger Dingledine:
> On Fri, Jan 29, 2021 at 12:34:28AM +0100, nusenu wrote:
>> If dir auths (some or all) are willing to share (privately or publicly) the 
>> distribution of
>> attack load (frequency, bandwidth, ...) by exit source IP in total or 
>> relative values
>> I can correlate this data to strengthen a hypothesis that 
>> malicious/suspicious
>> exits are involved to a greater extend than well-known long term exits.
> 
> I'll send you out-of-band a little snapshot of requests from relay
> IP addresses -- 160k requests over a 24 minute period from yesterday
> early evening.

I've looked at the data and found no clear indicators to support a
hypothesis that malicious/suspicious exit operators are involved to a greater 
extend then others, 
but I'm not sure if 24 minutes is enough to draw any conclusions. 
24 hours would probably more suitable. 

Expected request frequency for exit IPs would also be interesting when looking
and evaluating such data.

> At one point later in the evening I was getting several tens of millions
> of requests per hour. That's when I started to realize that exit relay
> operators were probably seeing this increased load too.

Did any exit operator actually see increased load on their exits?

kind regards,
nusenu

-- 
https://nusenu.github.io
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

Re: [tor-relays] Exit relay operators please help test #2667 branch

[Roger Dingledine]

On Fri, Jan 29, 2021 at 12:34:28AM +0100, nusenu wrote:
> If dir auths (some or all) are willing to share (privately or publicly) the 
> distribution of
> attack load (frequency, bandwidth, ...) by exit source IP in total or 
> relative values
> I can correlate this data to strengthen a hypothesis that malicious/suspicious
> exits are involved to a greater extend than well-known long term exits.

I'll send you out-of-band a little snapshot of requests from relay
IP addresses -- 160k requests over a 24 minute period from yesterday
early evening.

At one point later in the evening I was getting several tens of millions
of requests per hour. That's when I started to realize that exit relay
operators were probably seeing this increased load too.

> That could mean that they are not (exclusively) attacking via but _from_ 
> servers that also happen to
> run tor exits. 

Well, there are definitely other addresses -- the overload from last
week was non-relay addresses, and that's still going.

It's possible that there are exits that are generating more than their
"fair" share of requests. I didn't see that pattern obviously happening,
and confirming it would be complicated by the fact that some relays
probably have less or more congestion, which would cause the attacks to
be more efficient or less efficient through them.

We had a long debugging session in #tor-dev on irc last night, and there
will be more of those as we proceed. We've found a bunch of short-term
fragile distinguishers, which we could use to block the "bad" traffic
right now, but which wouldn't hold up if the bad traffic adapts a bit.

More broadly, we're trying to walk the fine line between doing our
analysis and patches in public (yay transparency), vs being aware
that whoever is doing this is probably reading these threads too. The
destination we want is that we have defenses that are robust to the
attacker knowing about them, but things will be a bit bumpy as we get
to that destination.

I'm also trying to make sure everybody continues to think about the
privacy side -- the directory authorities or fallbackdirs don't know
what paths clients build, or what destinations they reach with them,
but they can know at what timestamps some IP addresses seemed to be using
Tor. And like most things, that information is better private by default.

> From another angle this is an interesting precedence
> because the tor project uses it's access to protect dir auths
> from exit relays. Why is that interesting? Because no one else
> that gets attacked via exit relays has that "luxury" to "filter"
> it at the "source" (exits).

Actually, the #2667 patch protects all relays from exit relays. That is,
exit relays will decline to exit to known ORPorts or DirPorts of any
relay. There are two benefits here: (a) people can't do circuit-level
amplification attacks (happy to elaborate on these once the defense is
more in place), and (b) people can't create directory requests which
blend with the directory requests that the relay itself does.

These two issues are Tor-specific, and the second one is an especially
good argument I think, because the relay is reserving for itself the
ability to make its dir connections in a way that the destination can
know that the relay is the one making the connection. (Another option
would be to add more authentication to the connection, but most ways of
doing that are heavier-weight, not lighter-weight.)

--Roger

___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

Re: [tor-relays] Exit relay operators please help test #2667 branch

[nusenu]

Unless you already ruled out that hypothesis by looking at the attack
distribution by source IP:

If dir auths (some or all) are willing to share (privately or publicly) the 
distribution of
attack load (frequency, bandwidth, ...) by exit source IP in total or relative 
values
I can correlate this data to strengthen a hypothesis that malicious/suspicious
exits are involved to a greater extend than well-known long term exits.
That could mean that they are not (exclusively) attacking via but _from_ 
servers that also happen to
run tor exits. 


>From another angle this is an interesting precedence
because the tor project uses it's access to protect dir auths
from exit relays. Why is that interesting? Because no one else
that gets attacked via exit relays has that "luxury" to "filter"
it at the "source" (exits).


-- 
https://nusenu.github.io
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

Re: [tor-relays] Exit relay operators please help test #2667 branch

[Chris Dagdigian]

Have a tor exit running in the US ; fingerprint is 
3DE567C1350C0E858C6147AECB06EA9B3EAF3261 and OR address is 
71.174.105.126:9001


Just built and launched the ticket-2667 branch; came up as:

[notice] Tor 0.4.6.0-alpha-dev running on Linux with Libevent 
2.0.21-stable, OpenSSL 1.0.1t, Zlib 1.2.8, Liblzma N/A, Libzstd N/A and 
Glibc 2.19 as libc.


I'll monitor the log notices but feel free to probe to test. Thanks!





Roger Dingledine 
January 28, 2021 at 1:40 AM
Hello friendly relay operators,

Another day, another weird thing with the Tor network. This time we
have some jerk bombing the directory authorities with directory fetches,
and doing it via exits:
https://lists.torproject.org/pipermail/network-health/2021-January/000661.html

The network is mostly holding together, but I wouldn't say it is pretty.

One of the long-term fixes will be ticket #2667:
https://gitlab.torproject.org/tpo/core/tor/-/issues/2667
where exit relays refuse to let users connect back into the Tor network.

David and I made a branch this evening that implements #2667, and it
could use some testing. If you're comfortable building your exit relay
from a git branch, please do, and let us know how it goes. It is the
"ticket2667" branch on either
https://git.torproject.org/user/arma/tor
or
https://gitlab.torproject.org/arma/tor/

And if your relay is currently using 100% cpu and/or way more bandwidth
than usual, you might be especially excited to try out this patch. :)

When the defense triggers, you will see an info-level log line like
"%s tried to connect back to a known relay address. Closing."
(where %s is the destination, so don't get upset at them. :)

You can let us know how it's going either by mail just to me, or by a
reply on the list, whichever you prefer. Once we know that you're running
the branch, we can also probe your relay remotely to verify that it is
correctly refusing those connections.

Thanks!
--Roger

___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays


___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

[tor-relays] Exit relay operators please help test #2667 branch

[Roger Dingledine]

Hello friendly relay operators,

Another day, another weird thing with the Tor network. This time we
have some jerk bombing the directory authorities with directory fetches,
and doing it via exits:
https://lists.torproject.org/pipermail/network-health/2021-January/000661.html

The network is mostly holding together, but I wouldn't say it is pretty.

One of the long-term fixes will be ticket #2667:
https://gitlab.torproject.org/tpo/core/tor/-/issues/2667
where exit relays refuse to let users connect back into the Tor network.

David and I made a branch this evening that implements #2667, and it
could use some testing. If you're comfortable building your exit relay
from a git branch, please do, and let us know how it goes. It is the
"ticket2667" branch on either
https://git.torproject.org/user/arma/tor
or
https://gitlab.torproject.org/arma/tor/

And if your relay is currently using 100% cpu and/or way more bandwidth
than usual, you might be especially excited to try out this patch. :)

When the defense triggers, you will see an info-level log line like
"%s tried to connect back to a known relay address. Closing."
(where %s is the destination, so don't get upset at them. :)

You can let us know how it's going either by mail just to me, or by a
reply on the list, whichever you prefer. Once we know that you're running
the branch, we can also probe your relay remotely to verify that it is
correctly refusing those connections.

Thanks!
--Roger

___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays