Re: RFC: Established connections hash function

2007-03-29 Thread Evgeniy Polyakov
On Wed, Mar 28, 2007 at 04:52:55PM +0200, Andi Kleen ([EMAIL PROTECTED]) wrote: 3) We dont want to be 'totally secure'. We only want to raise the level, and eventually see if we have to spend more time on this next year(s). AFAIK we had two different reports from people being hit by the

Re: RFC: Established connections hash function

2007-03-28 Thread Andi Kleen
But I think it can be mostly ignored. With all due respect, it cannot. An attacker with a small-sized botnet (which is ~250 hosts) can create chains that contain well in excess of 3000 items. Most likely they can also easily generate enough latency data to crack any simple hash

Re: RFC: Established connections hash function

2007-03-28 Thread Evgeniy Polyakov
On Wed, Mar 28, 2007 at 11:29:43AM +0200, Andi Kleen ([EMAIL PROTECTED]) wrote: But I think it can be mostly ignored. With all due respect, it cannot. An attacker with a small-sized botnet (which is ~250 hosts) can create chains that contain well in excess of 3000 items. Most

Re: RFC: Established connections hash function

2007-03-28 Thread Andi Kleen
Evgeniy Polyakov [EMAIL PROTECTED] writes: Jenkins hash is far from being simple to crack, although with some knowledge it can be done faster. TCP tends to be initialized early before there is anything good in the entropy pool. static void init_std_data(struct entropy_store *r) {

Re: RFC: Established connections hash function II

2007-03-28 Thread Andi Kleen
Andi Kleen [EMAIL PROTECTED] writes: The only truly variable thing is the boot time, which can be guessed Actually you don't need to guess it. It's in any TCP timestamp. -Andi - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED]

Re: RFC: Established connections hash function

2007-03-28 Thread Eric Dumazet
On 28 Mar 2007 16:14:17 +0200 Andi Kleen [EMAIL PROTECTED] wrote: TCP tends to be initialized early before there is anything good in the entropy pool. static void init_std_data(struct entropy_store *r) { struct timeval tv; unsigned long flags;

Re: RFC: Established connections hash function

2007-03-28 Thread Andi Kleen
On Wed, Mar 28, 2007 at 03:50:47PM +0200, Eric Dumazet wrote: 1) We can now use struct timespec to get more bits in init_std_data() That would be a good change, but i don't think it would help that much. If you know the hardware (e.g. webhost farms tend to have quite predictive kit) and the

Re: RFC: Established connections hash function

2007-03-28 Thread David Miller
From: Andi Kleen [EMAIL PROTECTED] Date: 28 Mar 2007 16:14:17 +0200 Evgeniy Polyakov [EMAIL PROTECTED] writes: Jenkins hash is far from being simple to crack, although with some knowledge it can be done faster. TCP tends to be initialized early before there is anything good in the

Re: RFC: Established connections hash function

2007-03-28 Thread Andi Kleen
In my patch the random seed is initialized when the first TCP or DCCP socket is created, at which point we'll have sufficient entropy. See my discussion of this case in a later mail to Evgeniy -Andi - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to

Re: RFC: Established connections hash function

2007-03-27 Thread Andi Kleen
Nikolaos D. Bougalis [EMAIL PROTECTED] writes: I will be more than happy to provide a patch for this, but I figured I would solicit some input first. To truly defend against this you would likely need a cryptographic hash, which would be likely too slow. If it's a real problem the

Re: RFC: Established connections hash function

2007-03-27 Thread Nikolaos D. Bougalis
Andi Kleen ([EMAIL PROTECTED]) wrote: To truly defend against this you would likely need a cryptographic hash, which would be likely too slow. I do not think that a cryptographically secure hash is necessary for this. Using a better hash function (i.e. one which does a good job of

Re: RFC: Established connections hash function

2007-03-27 Thread David Miller
From: Nikolaos D. Bougalis [EMAIL PROTECTED] Date: Tue, 27 Mar 2007 22:01:38 -0700 What this boils down to is, yes, we can keep patching our own kernels to use tagged jenkins hashing if this affects us, waiting for something better to come along. You don't have to patch, I put jenkins

Re: RFC: Established connections hash function

2007-03-24 Thread linux
Result with jenkins: 1 23880 2 12108 3 4040 4 1019 5 200 6 30 7 8 8 1 Xor: 1 65536 Precisely. This means that the Xor hash SUCKS, because its output is conspicuously non-random. What you expect is a Poisson distribution, where the chance that a chain will contain k elements is

Re: RFC: Established connections hash function

2007-03-24 Thread Evgeniy Polyakov
On Sat, Mar 24, 2007 at 08:26:58AM -0400, [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: Result with jenkins: 1 23880 2 12108 3 4040 4 1019 5 200 6 30 7 8 8 1 Xor: 1 65536 Precisely. This means that the Xor hash SUCKS, because its output is conspicuously non-random.

Re: RFC: Established connections hash function

2007-03-23 Thread David Miller
From: Eric Dumazet [EMAIL PROTECTED] Date: Fri, 23 Mar 2007 09:00:08 +0100 I dont consider this new hash as bug fix at all, ie your patch might enter 2.6.22 normal dev cycle. Ok, I checked the patch into net-2.6.22 - To unsubscribe from this list: send the line unsubscribe netdev in the body

Re: RFC: Established connections hash function

2007-03-23 Thread David Miller
From: Eric Dumazet [EMAIL PROTECTED] Date: Thu, 22 Mar 2007 23:03:04 +0100 David Miller a écrit : From: Nikolaos D. Bougalis [EMAIL PROTECTED] Date: Thu, 22 Mar 2007 12:44:09 -0700 People _have_ had problems. _I_ have had problems. And when someone with a few thousand drones under

Re: RFC: Established connections hash function

2007-03-23 Thread Evgeniy Polyakov
On Thu, Mar 22, 2007 at 01:53:03PM -0700, Nikolaos D. Bougalis ([EMAIL PROTECTED]) wrote: Grrr, I think I pointed several times already, that properly distributed values do not change distribution after folding. And it can be seen in all tests (and in that you pointed too). Yes, I agree

Re: RFC: Established connections hash function

2007-03-23 Thread Eric Dumazet
David Miller a écrit : From: Eric Dumazet [EMAIL PROTECTED] Welcome to the club :) Ok, how about we put something like the following into 2.6.21? 2.6.21 really ? Just to be clear : I had an attack two years ago, I applied your patch, rebooted the machine, and since then the attackers

Re: RFC: Established connections hash function

2007-03-23 Thread Evgeniy Polyakov
On Thu, Mar 22, 2007 at 01:58:34PM -0700, David Miller ([EMAIL PROTECTED]) wrote: From: Nikolaos D. Bougalis [EMAIL PROTECTED] Date: Thu, 22 Mar 2007 12:44:09 -0700 People _have_ had problems. _I_ have had problems. And when someone with a few thousand drones under his control hoses

Re: RFC: Established connections hash function

2007-03-23 Thread Eric Dumazet
Evgeniy Polyakov a ecrit : Call me a loooser which mail will be deleted on arrival, but... jhash_2words(const, const, ((const 16) | $sport) ^ $random) where $sport is 1-65535 in a loop, and $random is pseudo-random number obtained on start. Which is exactly the case of web server and

Re: RFC: Established connections hash function

2007-03-23 Thread Evgeniy Polyakov
On Fri, Mar 23, 2007 at 09:17:19AM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: You have a machine somewhere that allows 65536 concurrent connections coming from the same IP address ? Attached png file of botnet scenario: 1000 addresses from the same network (class B for example), each one

Re: RFC: Established connections hash function

2007-03-23 Thread Evgeniy Polyakov
On Fri, Mar 23, 2007 at 11:33:32AM +0300, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: Eric, I agree, that XOR hash is not perfect, and it should be changed, but not blindly. Attached case of how broken can be xor in botnet scenario. -- Evgeniy Polyakov jhash_good.png Description:

XOR hash beauty solved [Was: RFC: Established connections hash function]

2007-03-23 Thread Evgeniy Polyakov
Please, do not apply patch as is, I will devote this day to find where jenkins has problems and try to fix distribution. If I will fail, then it is up to you to decide that above results are bad or good. I need to admit that I was partially wrong in my analysis of the Jenkins hash distribution

Re: RFC: Established connections hash function

2007-03-23 Thread Nikolaos D. Bougalis
Let me start off by saying that I hope I didn't come across as condenscending in my previous posts. If I did, then it wasn't intended. Now, on to more important things :) jhash_2words(const, const, ((const 16) | $sport) ^ $random) where $sport is 1-65535 in a loop, and $random is

Re: XOR hash beauty solved [Was: RFC: Established connections hash function]

2007-03-23 Thread Nikolaos D. Bougalis
So, briefly saying, jhash_2/3words have safe distribution, but have higher-number of elements waves as a result of folding which is unavoidable for general-purpose hash. Thanks for the analysis. -n - To unsubscribe from this list: send the line unsubscribe netdev in the body of a

RFC: Established connections hash function

2007-03-22 Thread Nikolaos D. Bougalis
Hello, I have noticed that the hash function that the kernel uses for established TCP/IP connections is rather simplistic, specifically: h = (local address ^ local_port) ^ (remote_address ^ remote_port); h ^= h 16; h ^= h 8; Now, simple is great, but this has a number of

Re: RFC: Established connections hash function

2007-03-22 Thread Evgeniy Polyakov
On Thu, Mar 22, 2007 at 08:39:04AM -0700, Nikolaos D. Bougalis ([EMAIL PROTECTED]) wrote: This particular hash seems to be the odd-man out, since most other network related hashes in the kernel seem to be Jenkins-based, and some use tagged hashing to defeat algorithmic complexity attacks.

Re: RFC: Established connections hash function

2007-03-22 Thread Nikolaos D. Bougalis
On Thu, March 22, 2007 at 8:52 AM, Evgeniy Polyakov [EMAIL PROTECTED] wrote: It seems you do not know a history... I know a lot about history. I may not know the specific history you had in mind though. I do see now that this has been brought up before. Before posting, I did search

Re: RFC: Established connections hash function

2007-03-22 Thread Evgeniy Polyakov
On Thu, Mar 22, 2007 at 10:32:44AM -0700, Nikolaos D. Bougalis ([EMAIL PROTECTED]) wrote: Utterly broken? Nonsense. I have tested the actual function I proposed (sans the __force and __u32 stuff, which weren't necessary in my test program), against real data, collected from various

Re: RFC: Established connections hash function

2007-03-22 Thread Nikolaos D. Bougalis
On Thu, Mar 22, 2007 11:21 AM, Evgeniy Polyakov [EMAIL PROTECTED] wrote: Utterly broken? Nonsense. I have tested the actual function I proposed (sans the __force and __u32 stuff, which weren't necessary in my test program), against real data, collected from various servers in real-time. It

Re: RFC: Established connections hash function

2007-03-22 Thread Evgeniy Polyakov
On Thu, Mar 22, 2007 at 12:44:09PM -0700, Nikolaos D. Bougalis ([EMAIL PROTECTED]) wrote: On Thu, Mar 22, 2007 11:21 AM, Evgeniy Polyakov [EMAIL PROTECTED] wrote: Utterly broken? Nonsense. I have tested the actual function I proposed (sans the __force and __u32 stuff, which weren't

Re: RFC: Established connections hash function

2007-03-22 Thread Nikolaos D. Bougalis
We started our discussion a bit wrong - let's start it again, ok? :) Fair enough. You do not want to read what was written - _if_ we use artificial data, then attacker can use it too, so if it is possible to break the system with artificial data, then it is possible it will be broken in

Re: RFC: Established connections hash function

2007-03-22 Thread David Miller
From: Nikolaos D. Bougalis [EMAIL PROTECTED] Date: Thu, 22 Mar 2007 12:44:09 -0700 People _have_ had problems. _I_ have had problems. And when someone with a few thousand drones under his control hoses your servers because he can do math and he leaves you with 2-item long chains,

Re: RFC: Established connections hash function

2007-03-22 Thread Eric Dumazet
David Miller a écrit : From: Nikolaos D. Bougalis [EMAIL PROTECTED] Date: Thu, 22 Mar 2007 12:44:09 -0700 People _have_ had problems. _I_ have had problems. And when someone with a few thousand drones under his control hoses your servers because he can do math and he leaves you with