Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-07 Thread Nick Piggin
On Tue, Mar 06, 2007 at 02:20:55PM -0800, David Miller wrote: From: Robert Olsson [EMAIL PROTECTED] Date: Tue, 6 Mar 2007 14:26:04 +0100 David Miller writes: Actually, more accurately, the conflict exists in how this GC logic is implemented. The core issue is that hash table

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread David Miller
From: Eric Dumazet [EMAIL PROTECTED] Date: Tue, 06 Mar 2007 08:58:45 +0100 Yes, but on bootup you have an appropriate NUMA active policy. (Well... we hope so, but it broke several time in the past) I am not sure what kind of mm policy is active for scheduled works. Good point, that

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Nick Piggin
On Mon, Mar 05, 2007 at 08:26:32PM -0800, David Miller wrote: This is essentially a port of Nick Piggin's dcache hash table patches to the routing cache. It solves the locking issues during table grow/shrink that I couldn't handle properly last time I tried to code up a patch like this.

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread David Miller
From: Nick Piggin [EMAIL PROTECTED] Date: Tue, 6 Mar 2007 10:11:12 +0100 @@ -1449,6 +1472,12 @@ static struct dst_entry *ipv4_negative_a %u.%u.%u.%u/%02x dropped\n, NIPQUAD(rt-rt_dst), rt-fl.fl4_tos); #endif +

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Nick Piggin
On Tue, Mar 06, 2007 at 01:17:06AM -0800, David Miller wrote: From: Nick Piggin [EMAIL PROTECTED] Date: Tue, 6 Mar 2007 10:11:12 +0100 @@ -1449,6 +1472,12 @@ static struct dst_entry *ipv4_negative_a %u.%u.%u.%u/%02x dropped\n,

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Eric Dumazet
On Tuesday 06 March 2007 10:11, Nick Piggin wrote: Cool! I have some fixes for the rcu barrier issues, with some C-style comments and questions :) I was going to send you a fix first for the rcu barriers, then a second to convert the read-side to a barrier-less one that I described, however

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Nick Piggin
On Tue, Mar 06, 2007 at 10:23:44AM +0100, Eric Dumazet wrote: On Tuesday 06 March 2007 10:11, Nick Piggin wrote: Cool! I have some fixes for the rcu barrier issues, with some C-style comments and questions :) I was going to send you a fix first for the rcu barriers, then a second to

[RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
David Miller writes: Interesting. Actually, more accurately, the conflict exists in how this GC logic is implemented. The core issue is that hash table size guides the GC processing, and hash table growth therefore modifies those GC goals. So with the patch below we'll just keep

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
Eric Dumazet writes: Well, maybe... but after looking robert's trash, I discovered its model is essentially a big (2^18 slots) root node (our hash table), and very few order:1,2,3 nodes. It's getting hashlike yes. I guess all effective algorithms today is doing some sort of index

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Eric Dumazet
On Tuesday 06 March 2007 14:42, Robert Olsson wrote: Eric Dumazet writes: Well, maybe... but after looking robert's trash, I discovered its model is essentially a big (2^18 slots) root node (our hash table), and very few order:1,2,3 nodes. It's getting hashlike yes. I guess all

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
Eric Dumazet writes: Indeed. It would be nice to see how it performs with say 2^20 elements... Because with your data, I wonder if the extra complexity of the trash is worth it (since most lookups are going to only hit the hash and give the answer without intermediate nodes) I

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Eric Dumazet
On Tuesday 06 March 2007 18:05, Robert Olsson wrote: Eric Dumazet writes: Indeed. It would be nice to see how it performs with say 2^20 elements... Because with your data, I wonder if the extra complexity of the trash is worth it (since most lookups are going to only hit the hash and

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread Robert Olsson
Eric Dumazet writes: With 2^20 entries, your actual limit of 2^19 entries in root node will probably show us quite different numbers for order-1,2,3,4... tnodes Yeep trie will get deeper and lookup more costly as insert and delete. The 2^19 was that was getting memory alloction problem

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-06 Thread David Miller
From: Robert Olsson [EMAIL PROTECTED] Date: Tue, 6 Mar 2007 14:26:04 +0100 David Miller writes: Actually, more accurately, the conflict exists in how this GC logic is implemented. The core issue is that hash table size guides the GC processing, and hash table growth therefore

[RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread David Miller
This is essentially a port of Nick Piggin's dcache hash table patches to the routing cache. It solves the locking issues during table grow/shrink that I couldn't handle properly last time I tried to code up a patch like this. But one of the core issues of this kind of change still remains.

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread Eric Dumazet
David Miller a écrit : This is essentially a port of Nick Piggin's dcache hash table patches to the routing cache. It solves the locking issues during table grow/shrink that I couldn't handle properly last time I tried to code up a patch like this. But one of the core issues of this kind of

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread David Miller
From: Eric Dumazet [EMAIL PROTECTED] Date: Tue, 06 Mar 2007 08:14:46 +0100 I wonder... are you sure this has no relation with the size of rt_hash_locks / RT_HASH_LOCK_SZ ? One entry must have the same lock in the two tables when resizing is in flight. #define MIN_RTHASH_SHIFT

Re: [RFC PATCH]: Dynamically sized routing cache hash table.

2007-03-05 Thread Eric Dumazet
David Miller a écrit : From: Eric Dumazet [EMAIL PROTECTED] Date: Tue, 06 Mar 2007 08:14:46 +0100 I wonder... are you sure this has no relation with the size of rt_hash_locks / RT_HASH_LOCK_SZ ? One entry must have the same lock in the two tables when resizing is in flight. #define