Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-08 Thread Robert Olsson
David Miller writes: > Even a nearly perfect hash has small lumps in distribution, and we > should not penalize entries which fall into these lumps. > > Let us call T the threshold at which we would grow the routing hash > table. As we approach T we start to GC. Let's assume hash table >

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-07 Thread Nick Piggin
On Tue, Mar 06, 2007 at 02:20:55PM -0800, David Miller wrote: > From: Robert Olsson <[EMAIL PROTECTED]> > Date: Tue, 6 Mar 2007 14:26:04 +0100 > > > David Miller writes: > > > > > Actually, more accurately, the conflict exists in how this GC > > > logic is implemented. The core issue is that

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread David Miller
From: Robert Olsson <[EMAIL PROTECTED]> Date: Tue, 6 Mar 2007 14:26:04 +0100 > David Miller writes: > > > Actually, more accurately, the conflict exists in how this GC > > logic is implemented. The core issue is that hash table size > > guides the GC processing, and hash table growth therefo

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
Eric Dumazet writes: > With 2^20 entries, your actual limit of 2^19 entries in root node will > probably show us quite different numbers for order-1,2,3,4... tnodes Yeep trie will get deeper and lookup more costly as insert and delete. The 2^19 was that was getting memory alloction problem

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Eric Dumazet
On Tuesday 06 March 2007 18:05, Robert Olsson wrote: > Eric Dumazet writes: > > Indeed. It would be nice to see how it performs with say 2^20 > > elements... Because with your data, I wonder if the extra complexity of > > the trash is worth it (since most lookups are going to only hit the hash >

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
Eric Dumazet writes: > Indeed. It would be nice to see how it performs with say 2^20 elements... > Because with your data, I wonder if the extra complexity of the trash is > worth > it (since most lookups are going to only hit the hash and give the answer > without intermediate nodes) I

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Eric Dumazet
On Tuesday 06 March 2007 14:42, Robert Olsson wrote: > Eric Dumazet writes: > > Well, maybe... but after looking robert's trash, I discovered its model > > is essentially a big (2^18 slots) root node (our hash table), and very > > few order:1,2,3 nodes. > > It's getting "hashlike" yes. I guess

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
Eric Dumazet writes: > Well, maybe... but after looking robert's trash, I discovered its model is > essentially a big (2^18 slots) root node (our hash table), and very few > order:1,2,3 nodes. It's getting "hashlike" yes. I guess all effective algorithms today is doing some sort of "inde

[RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
David Miller writes: Interesting. > Actually, more accurately, the conflict exists in how this GC > logic is implemented. The core issue is that hash table size > guides the GC processing, and hash table growth therefore > modifies those GC goals. So with the patch below we'll just > k

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Nick Piggin
On Tue, Mar 06, 2007 at 10:23:44AM +0100, Eric Dumazet wrote: > On Tuesday 06 March 2007 10:11, Nick Piggin wrote: > > > Cool! I have some fixes for the rcu barrier issues, with some C-style > > comments and questions :) > > > > I was going to send you a fix first for the rcu barriers, then a > >

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Eric Dumazet
On Tuesday 06 March 2007 10:11, Nick Piggin wrote: > Cool! I have some fixes for the rcu barrier issues, with some C-style > comments and questions :) > > I was going to send you a fix first for the rcu barriers, then a > second to convert the read-side to a barrier-less one that I described, > ho

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Nick Piggin
On Tue, Mar 06, 2007 at 01:17:06AM -0800, David Miller wrote: > From: Nick Piggin <[EMAIL PROTECTED]> > Date: Tue, 6 Mar 2007 10:11:12 +0100 > > > @@ -1449,6 +1472,12 @@ static struct dst_entry *ipv4_negative_a > > "%u.%u.%u.%u/%02x dropped\n", > >

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread David Miller
From: Nick Piggin <[EMAIL PROTECTED]> Date: Tue, 6 Mar 2007 10:11:12 +0100 > @@ -1449,6 +1472,12 @@ static struct dst_entry *ipv4_negative_a > "%u.%u.%u.%u/%02x dropped\n", > NIPQUAD(rt->rt_dst), rt->fl.fl4_tos); > #endif > +

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Nick Piggin
On Mon, Mar 05, 2007 at 08:26:32PM -0800, David Miller wrote: > > This is essentially a "port" of Nick Piggin's dcache hash table > patches to the routing cache. It solves the locking issues > during table grow/shrink that I couldn't handle properly last > time I tried to code up a patch like thi

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread David Miller
From: Eric Dumazet <[EMAIL PROTECTED]> Date: Tue, 06 Mar 2007 08:58:45 +0100 > Yes, but on bootup you have an appropriate NUMA active policy. (Well... we > hope so, but it broke several time in the past) > I am not sure what kind of mm policy is active for scheduled works. Good point, that defin

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread Eric Dumazet
David Miller a écrit : From: Eric Dumazet <[EMAIL PROTECTED]> Date: Tue, 06 Mar 2007 08:14:46 +0100 I wonder... are you sure this has no relation with the size of rt_hash_locks / RT_HASH_LOCK_SZ ? One entry must have the same lock in the two tables when resizing is in flight. #define MIN_RTHAS

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread David Miller
From: Eric Dumazet <[EMAIL PROTECTED]> Date: Tue, 06 Mar 2007 08:14:46 +0100 > I wonder... are you sure this has no relation with the size of rt_hash_locks > / > RT_HASH_LOCK_SZ ? > One entry must have the same lock in the two tables when resizing is in > flight. > #define MIN_RTHASH_SHIFT LOG2

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread Eric Dumazet
David Miller a écrit : This is essentially a "port" of Nick Piggin's dcache hash table patches to the routing cache. It solves the locking issues during table grow/shrink that I couldn't handle properly last time I tried to code up a patch like this. But one of the core issues of this kind of c

[RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread David Miller
This is essentially a "port" of Nick Piggin's dcache hash table patches to the routing cache. It solves the locking issues during table grow/shrink that I couldn't handle properly last time I tried to code up a patch like this. But one of the core issues of this kind of change still remains. The