Re: any middlemen seeing DoS currently?
Il 11/11/2008 15:23, Geoff Down ha scritto: Crashed again after only 2 hours: I had to shut down my node temporarily due to high load. Jan
Re: any middlemen seeing DoS currently?
Crashed again after only 2 hours: This was about 20 minutes beforehand, %CPU %MEM VSZRSS TT STAT STARTED TIME 0.0 1.639784 10400 ?? S 4:03AM 1:32.40 Nov 11 04:03:06.129 [Notice] Tor v0.2.0.31 (r16744). This is experimental software. Do not rely on it for strong anonymity. (Running on Darwin Power Macintosh) Nov 11 04:03:06.177 [Notice] Initialized libevent version 1.4.7-stable using method kqueue. Good. Nov 11 04:03:06.198 [Notice] Opening OR listener on 0.0.0.0:9001 Nov 11 04:03:06.219 [Notice] Opening Socks listener on 127.0.0.1:9050 Nov 11 04:03:06.299 [Notice] Opening Control listener on 127.0.0.1:9051 Nov 11 04:04:23.566 [Notice] Self-testing indicates your ORPort is reachable from the outside. Excellent. Publishing server descriptor. Nov 11 04:04:53.299 [Notice] Performing bandwidth self-test...done. Nov 11 06:05:20.894 [Notice] We tried for 15 seconds to connect to '[scrubbed]' using exit 'johndoe'. Retrying on a new circuit. Should I be logging at info level? It's a lot of data... GD On 10 Nov 2008, at 03:19, Nick Mathewson wrote: On Fri, Nov 07, 2008 at 01:38:28PM +0100, Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? Any alternative explanation to DoS? (ISP throttling?). Judging by the timing, I'd think it might be related to a bug we only uncovered on Friday. Why Friday? That was the first time that a directory authority's certificate expired before it could be replaced. The bug was that clients repeatedly asked directory caches for a new certificate over and over, without noticing that they were getting something expired and deciding to wait for a while. That bug should be fixed in newer versions of Tor. Also, all the authority operators should (if we can make them) get way more careful about checking certificate expiry times. -- Nick
Re: any middlemen seeing DoS currently?
On Fri, Nov 07, 2008 at 01:38:28PM +0100, Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? Any alternative explanation to DoS? (ISP throttling?). Judging by the timing, I'd think it might be related to a bug we only uncovered on Friday. Why Friday? That was the first time that a directory authority's certificate expired before it could be replaced. The bug was that clients repeatedly asked directory caches for a new certificate over and over, without noticing that they were getting something expired and deciding to wait for a while. That bug should be fixed in newer versions of Tor. Also, all the authority operators should (if we can make them) get way more careful about checking certificate expiry times. -- Nick
Re: any middlemen seeing DoS currently?
Geoff Down wrote: My PC crashed overnight a couple of times now with a relay running - is this the same thing? my tor process' workspace increased from about 400 MB to 1200 MB over the last hours. Fortuntely anonymizer.blutmagie.de is equipped with 4 GB memory. Olaf I'm seeing the same as well. 11k tcp connections and 1.5GB of memory used by tor. Router name rXTXo1bxEP.
Re: any middlemen seeing DoS currently?
Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? yes, the same here Olaf
Re: any middlemen seeing DoS currently?
Could the No current certificate known for authority ides; launching request. message that my client's been displaying every minute or so for the last 4 hours be related to that, or is my problem just a coincident? ___ Sent by ePrompter, the premier email notification software. Free download at http://www.ePrompter.com.
Re: any middlemen seeing DoS currently?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CyberRax wrote: Could the No current certificate known for authority ides; launching request. message that my client's been displaying every minute or so for the last 4 hours be related to that, or is my problem just a coincident? This might be the problem, yes. The reason is that ides' authority certificate has expired. But clients do not differentiate between a (temporary) download problem and the situation that a certificate has expired and a new one needs to be created (which is rather unlikely to happen within the next few minutes). So, clients send another request for a certificate every minute. All clients running 0.2.0.x or higher do this, which is why there is so much additional traffic in the network. The problem of clients downloading certificates that often will be solved with the next alpha. But the main solution is to upgrade the authority certificate which should happen some time today. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJFGJI0M+WPffBEmURAs5cAJ97vQ6F+WZKLjux4ubjWirV3KyIfACggWiN emK+fqenVFWlN5aOcxkPo2M= =A9gY -END PGP SIGNATURE-
Re: any middlemen seeing DoS currently?
On Fri, Nov 07, 2008 at 02:49:47PM +0100, Eugen Leitl wrote: On Fri, Nov 07, 2008 at 02:10:32PM +0100, Olaf Selke wrote: Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? yes, the same here Anyone knows which kind of attack that is? Any suggestions how to block it (pf here) yet? you may set the timeout values in pf.conf to rather low values. Actually I start wondering if larger values are of any use anyway. maybe like: - set timeout interval 2 set timeout frag 5 set timeout tcp.first 5 set timeout tcp.opening 5 set timeout tcp.established 600 set timeout tcp.closing 5 set timeout tcp.finwait 3 set timeout tcp.closed 5 -- besides the default. this will kick yourself too if the line is idle for too long. Hans
Re: any middlemen seeing DoS currently?
On Fri, Nov 07, 2008 at 02:10:32PM +0100, Olaf Selke wrote: Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? yes, the same here Anyone knows which kind of attack that is? Any suggestions how to block it (pf here) yet? -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
any middlemen seeing DoS currently?
I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? Any alternative explanation to DoS? (ISP throttling?). Thanks. -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Re: any middlemen seeing DoS currently?
On Fri, Nov 07, 2008 at 01:38:28PM +0100, Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? Any alternative explanation to DoS? (ISP throttling?). Thanks. Yes, now roughly 4.5 hours ago states went to the upper limit 10k. Flushed tables the hard way but ten minutes later they were up on 10k again. Talked to my ISP's helpline and according to them it may not be their responsibility, but h !?!
Re: any middlemen seeing DoS currently?
Same at IdentityHog. Number of TCP connections steadily increased to ~10k and then the server crashed. I have unfortunately had to shut it down for now. Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? Any alternative explanation to DoS? (ISP throttling?). Thanks.
Re: any middlemen seeing DoS currently?
My PC crashed overnight a couple of times now with a relay running - is this the same thing? OSX 10.3.9 Vidalia 0.1.9 Tor 0.2.0.31 r16744 GD On 7 Nov 2008, at 18:25, Martin Hodge wrote: Same at IdentityHog. Number of TCP connections steadily increased to ~10k and then the server crashed. I have unfortunately had to shut it down for now. Eugen Leitl wrote: I've seen continuous table state increase since about 3.5 hours. It went up from 1 k baseline to 5 k. Anyone else seeing this? Any alternative explanation to DoS? (ISP throttling?). Thanks.
Re: any middlemen seeing DoS currently?
Geoff Down wrote: My PC crashed overnight a couple of times now with a relay running - is this the same thing? my tor process' workspace increased from about 400 MB to 1200 MB over the last hours. Fortuntely anonymizer.blutmagie.de is equipped with 4 GB memory. Olaf
Re: any middlemen seeing DoS currently?
On Fri, Nov 7, 2008 at 12:52 PM, Olaf Selke [EMAIL PROTECTED] wrote: Geoff Down wrote: My PC crashed overnight a couple of times now with a relay running - is this the same thing? my tor process' workspace increased from about 400 MB to 1200 MB over the last hours. Fortuntely anonymizer.blutmagie.de is equipped with 4 GB memory. Olaf FWIW: On an old HP Kayak (2x500mhz PIIIs, 768mb) running Ubuntu 7.10 and tor built from the SVN a couple of weeks ago: open number of connections are hovering around 1200-1400 and no memory or CPU problems: Doing my best copy/paste from top: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 00 tor-tor 15 0 123m 81m 15m R2 9.2 362:54.22 tor -- [EMAIL PROTECTED]
Re: any middlemen seeing DoS currently?
On Fri, 07 Nov 2008 16:44:09 +0100 Karsten Loesing [EMAIL PROTECTED] wrote: The problem of clients downloading certificates that often will be solved with the next alpha. But the main solution is to upgrade the authority certificate which should happen some time today. I think that it might be an idea to send out an official announcement here on or-announce and perhaps on the website to tell people to stop their inactive tor clients (i.e. sudo /etc/init.d/tor stop ) to take the pressure off the exits and middles. When I read the above thread I checked and my computer has been pinging away all day looking for an updated certificate.. and I've only used Tor a little.. It's now turned off till I need it. or the problem is fixed. Take Care, Freemor -- [EMAIL PROTECTED] [EMAIL PROTECTED] This e-mail has been digitally signed with GnuPG - ( http://gnupg.org/ ) signature.asc Description: PGP signature
Re: any middlemen seeing DoS currently?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Freemor wrote: I think that it might be an idea to send out an official announcement here on or-announce and perhaps on the website to tell people to stop their inactive tor clients (i.e. sudo /etc/init.d/tor stop ) to take the pressure off the exits and middles. When I read the above thread I checked and my computer has been pinging away all day looking for an updated certificate.. and I've only used Tor a little.. It's now turned off till I need it. or the problem is fixed. A new certificate is now in place. This should clear things up really soon. The authorities should exchange the new certificate during the next voting process (in roughly 30 minutes). Then clients will be satisfied with the new certificate and stop requesting a new one repeatedly. That means there is not enough time for an official announcement. Or rather, the effect would not be as significant as compared to the resulting confusion. Sorry for the trouble! This will be fixed in future Tor versions. And we will pay more attention to expiring certificates. The next certificate expires on January 17, 2009. Mark the date. ;) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJFJXg0M+WPffBEmURAgj4AJ9viyb2hSad/dG4Ho2dgbB036MaBwCfXtjZ cOoynU24phoc1M7i7+FT7dQ= =Z+94 -END PGP SIGNATURE-
Re: any middlemen seeing DoS currently?
On Fri, Nov 07, 2008 at 07:52:06PM +0100, Olaf Selke wrote: Geoff Down wrote: My PC crashed overnight a couple of times now with a relay running - is this the same thing? my tor process' workspace increased from about 400 MB to 1200 MB over the last hours. Fortuntely anonymizer.blutmagie.de is equipped with 4 GB memory. Here's the plot from the state table (see URL). The horizontal line at 5 k is when the firewall's (128 kByte RAM, WRAP) state table ran over until I increased them by a factor of 6. The vertical lines were manual flushes. http://eugen.leitl.org/status_rrd_graph_img.php.png The server sees some 25 GBytes/day traffic. -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE - End forwarded message - -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Crashing relay (was any middlemen seeing DoS currently?)
Is there anything, in simple terms, that I can do to stop this? Bear in mind please that I'm an expert neither in Tor nor OSX, but I would like to contribute to the network. My torrc is the bare minimum generated by the Vidalia interface, apart from my specifying my Address to avoid a bug with my dynamic IP (I posted previously under thread 'Problem with dynamic IP'). Thanks, GD On 7 Nov 2008, at 19:51, Eugen Leitl wrote: On Fri, Nov 07, 2008 at 07:52:06PM +0100, Olaf Selke wrote: Geoff Down wrote: My PC crashed overnight a couple of times now with a relay running - is this the same thing? my tor process' workspace increased from about 400 MB to 1200 MB over the last hours. Fortuntely anonymizer.blutmagie.de is equipped with 4 GB memory. Here's the plot from the state table (see URL). The horizontal line at 5 k is when the firewall's (128 kByte RAM, WRAP) state table ran over until I increased them by a factor of 6. The vertical lines were manual flushes. http://eugen.leitl.org/status_rrd_graph_img.php.png The server sees some 25 GBytes/day traffic. -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE - End forwarded message - -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Re: any middlemen seeing DoS currently?
Olaf Selke schrieb: my tor process' workspace increased from about 400 MB to 1200 MB over the last hours. Fortuntely anonymizer.blutmagie.de is equipped with 4 GB memory. Similar here: from around 2k to 5.6k connections, cpu load tripled from 25% to 75%, memory consumption increased from 240MB to 840MB, swap file usage from 0 to 450MB. ;-) And traffic declines slowly over the last few hours, probably because my tor relay rejects most new connections now. But, kind of interesting that the network stills works more or less, with noticable higher latency though. ;-) I took the opportunity to reboot my machine after some updates, doesn't matter much now. ;-) Mhmm, I guess, it could be an opportunity to study the effects of a dDOS on the network, but probably there are no statistics detailed enough for enough relays available for researchers? Dominik
Re: Crashing relay (was any middlemen seeing DoS currently?)
On Fri, Nov 07, 2008 at 08:02:37PM +, Geoff Down wrote: Is there anything, in simple terms, that I can do to stop this? Bear in mind please that I'm an expert neither in Tor nor OSX, but I I'm running Tor on Leopard (G4 Mac mini, 1 GByte) behind a pfSense 1.2.1 firewall on a WRAP (state table size now up to 60 kStates, memory requirements and load negligible). I haven't done anything to the Tor process, which currently requires, according to top: 163 tor 12.8% 10:56:55 228 2599 135M+ 200K 150M- 230M+ So far it doesn't seem as if I'm running into any limits yet. I did run into firewall's state table limit at 10 kStates, which however didn't result in a crash. would like to contribute to the network. My torrc is the bare minimum generated by the Vidalia interface, apart from my specifying my Address to avoid a bug with my dynamic IP (I posted previously under thread 'Problem with dynamic IP'). Thanks, -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Re: any middlemen seeing DoS currently?
Hans Schnehl schrieb: you may set the timeout values in pf.conf to rather low values. Actually I start wondering if larger values are of any use anyway. maybe like: - set timeout interval 2 set timeout frag 5 set timeout tcp.first 5 set timeout tcp.opening 5 set timeout tcp.established 600 set timeout tcp.closing 5 set timeout tcp.finwait 3 set timeout tcp.closed 5 -- If I am not mistaken (don't know pf), this will terminate all idle connections after 600s. That means that your Tor relay will at least loose idle exit connections (maybe even circuits?) which are still valid. This is especially bad for people using long-lived connection to instant messaging services, IRC, and others for which Tor prefers relays with the stable flag. If your relay has that flag, it may be worth to keep that in mind. Dominik
Re: any middlemen seeing DoS currently?
On Fri, Nov 07, 2008 at 10:23:02PM +0100, Dominik Schaefer wrote: Hans Schnehl schrieb: you may set the timeout values in pf.conf to rather low values. Actually I start wondering if larger values are of any use anyway. maybe like: - set timeout interval 2 set timeout frag 5 set timeout tcp.first 5 set timeout tcp.opening 5 set timeout tcp.established 600 set timeout tcp.closing 5 set timeout tcp.finwait 3 set timeout tcp.closed 5 -- If I am not mistaken (don't know pf), this will terminate all idle connections after 600s. Correct. That means that your Tor relay will at least loose idle exit connections (maybe even circuits?) which are still valid. This is especially bad for people using long-lived connection to instant messaging services, IRC, and others for which Tor prefers relays with the stable flag. If your relay has that flag, it may be worth to keep that in mind. Thanks for bringing this up, these values are _not_ useful under normal circumstances. These settings will, under conditions like mentioned in this threat, reduce the number of states in pf's table. As soon as the state tables reach their upper limit no further connections are accepted until the number drops under the limit again. It is the choice of having useless states or trying to get rid of unused connections rather fast in order to provide access for 'real' connections. It reduced the number of states by 20% within an hour or so on vallenator. AFAIK this present situation will take some more time until the new certificate is through. _Until then_ , these 'emergency response settings' may be helpful if using pf ;) Hans
