Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/09/15 at 10:38am, Herbert Xu wrote: > On Wed, Dec 09, 2015 at 03:36:32AM +0100, Thomas Graf wrote: > > > > Without knowing your exact implementation plans: introducing an > > additional reference indirection for every lookup will have a > > huge performance penalty as well. > > > > Is your

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Wed, Dec 09, 2015 at 03:36:32AM +0100, Thomas Graf wrote: > > Without knowing your exact implementation plans: introducing an > additional reference indirection for every lookup will have a > huge performance penalty as well. > > Is your plan to only introduce the master table after an >

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/09/15 at 10:24am, Herbert Xu wrote: > On Wed, Dec 09, 2015 at 03:18:26AM +0100, Thomas Graf wrote: > > > > Assuming that we only encounter this scenario with very large > > table sizes, it might be OK to assume that deferring the actual > > resize via the worker thread while continuing to

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Wed, Dec 09, 2015 at 03:18:26AM +0100, Thomas Graf wrote: > > Assuming that we only encounter this scenario with very large > table sizes, it might be OK to assume that deferring the actual > resize via the worker thread while continuing to insert above > 100% utilization in atomic context is

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/05/15 at 03:06pm, Herbert Xu wrote: > Unless we can make __vmalloc work with BH disabled, I guess we'll > have to go back to multi-level lookups unless someone has a better > suggestion. Assuming that we only encounter this scenario with very large table sizes, it might be OK to assume

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/09/15 at 10:24am, Herbert Xu wrote: > On Wed, Dec 09, 2015 at 03:18:26AM +0100, Thomas Graf wrote: > > > > Assuming that we only encounter this scenario with very large > > table sizes, it might be OK to assume that deferring the actual > > resize via the worker thread while continuing to

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Wed, Dec 09, 2015 at 03:18:26AM +0100, Thomas Graf wrote: > > Assuming that we only encounter this scenario with very large > table sizes, it might be OK to assume that deferring the actual > resize via the worker thread while continuing to insert above > 100% utilization in atomic context is

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/05/15 at 03:06pm, Herbert Xu wrote: > Unless we can make __vmalloc work with BH disabled, I guess we'll > have to go back to multi-level lookups unless someone has a better > suggestion. Assuming that we only encounter this scenario with very large table sizes, it might be OK to assume

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Wed, Dec 09, 2015 at 03:36:32AM +0100, Thomas Graf wrote: > > Without knowing your exact implementation plans: introducing an > additional reference indirection for every lookup will have a > huge performance penalty as well. > > Is your plan to only introduce the master table after an >

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/09/15 at 10:38am, Herbert Xu wrote: > On Wed, Dec 09, 2015 at 03:36:32AM +0100, Thomas Graf wrote: > > > > Without knowing your exact implementation plans: introducing an > > additional reference indirection for every lookup will have a > > huge performance penalty as well. > > > > Is your

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

From: Thomas Graf Date: Mon, 7 Dec 2015 16:35:24 +0100 > Did you look into what __vmalloc prevents to work with BH disabled? You can't issue the cross-cpu TLB flushes from atomic contexts. It's the kernel page table updates that create the restriction. -- To unsubscribe from this list: send the

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/05/15 at 03:06pm, Herbert Xu wrote: > On Fri, Dec 04, 2015 at 07:15:55PM +0100, Phil Sutter wrote: > > > > > Only one should really do this, while others are waiting. > > > > Sure, that was my previous understanding of how this thing works. > > Yes that's clearly how it should be.

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On 12/05/15 at 03:06pm, Herbert Xu wrote: > On Fri, Dec 04, 2015 at 07:15:55PM +0100, Phil Sutter wrote: > > > > > Only one should really do this, while others are waiting. > > > > Sure, that was my previous understanding of how this thing works. > > Yes that's clearly how it should be.

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

From: Thomas Graf Date: Mon, 7 Dec 2015 16:35:24 +0100 > Did you look into what __vmalloc prevents to work with BH disabled? You can't issue the cross-cpu TLB flushes from atomic contexts. It's the kernel page table updates that create the restriction. -- To unsubscribe from this

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

From: Herbert Xu Date: Sat, 5 Dec 2015 15:03:54 +0800 > Sorry Dave but you'll have to revert this because I've been able > to trigger the following crash with the patch: > > Testing concurrent rhashtable access from 50 threads > [ cut here ] > kernel BUG at

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

From: Herbert Xu Date: Sat, 5 Dec 2015 15:03:54 +0800 > Sorry Dave but you'll have to revert this because I've been able > to trigger the following crash with the patch: > > Testing concurrent rhashtable access from 50 threads > [ cut here ]

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 07:15:55PM +0100, Phil Sutter wrote: > > > Only one should really do this, while others are waiting. > > Sure, that was my previous understanding of how this thing works. Yes that's clearly how it should be. Unfortunately while adding the locking to do this, I found out

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 04:53:34PM -0500, David Miller wrote: > From: Herbert Xu > Date: Fri, 4 Dec 2015 22:39:56 +0800 > > > When an rhashtable user pounds rhashtable hard with back-to-back > > insertions we may end up growing the table in GFP_ATOMIC context. > > Unfortunately when the table

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

From: Herbert Xu Date: Fri, 4 Dec 2015 22:39:56 +0800 > When an rhashtable user pounds rhashtable hard with back-to-back > insertions we may end up growing the table in GFP_ATOMIC context. > Unfortunately when the table reaches a certain size this often > fails because we don't have enough

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 09:45:20AM -0800, Eric Dumazet wrote: > On Fri, 2015-12-04 at 18:01 +0100, Phil Sutter wrote: > > On Fri, Dec 04, 2015 at 10:39:56PM +0800, Herbert Xu wrote: > > > On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > > > > > > > Anyway, __vmalloc() can be used

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, 2015-12-04 at 18:01 +0100, Phil Sutter wrote: > On Fri, Dec 04, 2015 at 10:39:56PM +0800, Herbert Xu wrote: > > On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > > > > > Anyway, __vmalloc() can be used with GFP_ATOMIC, have you tried this ? > > > > OK I've tried it and I

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 10:39:56PM +0800, Herbert Xu wrote: > On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > > > Anyway, __vmalloc() can be used with GFP_ATOMIC, have you tried this ? > > OK I've tried it and I no longer get any ENOMEM errors! I can't confirm this, sadly.

rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > Anyway, __vmalloc() can be used with GFP_ATOMIC, have you tried this ? OK I've tried it and I no longer get any ENOMEM errors! ---8<--- When an rhashtable user pounds rhashtable hard with back-to-back insertions we may end up

rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > Anyway, __vmalloc() can be used with GFP_ATOMIC, have you tried this ? OK I've tried it and I no longer get any ENOMEM errors! ---8<--- When an rhashtable user pounds rhashtable hard with back-to-back insertions we may end up

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, 2015-12-04 at 18:01 +0100, Phil Sutter wrote: > On Fri, Dec 04, 2015 at 10:39:56PM +0800, Herbert Xu wrote: > > On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > > > > > Anyway, __vmalloc() can be used with GFP_ATOMIC, have you tried this ? > > > > OK I've tried it and I

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 10:39:56PM +0800, Herbert Xu wrote: > On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > > > Anyway, __vmalloc() can be used with GFP_ATOMIC, have you tried this ? > > OK I've tried it and I no longer get any ENOMEM errors! I can't confirm this, sadly.

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

From: Herbert Xu Date: Fri, 4 Dec 2015 22:39:56 +0800 > When an rhashtable user pounds rhashtable hard with back-to-back > insertions we may end up growing the table in GFP_ATOMIC context. > Unfortunately when the table reaches a certain size this often > fails

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 07:15:55PM +0100, Phil Sutter wrote: > > > Only one should really do this, while others are waiting. > > Sure, that was my previous understanding of how this thing works. Yes that's clearly how it should be. Unfortunately while adding the locking to do this, I found out

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 04:53:34PM -0500, David Miller wrote: > From: Herbert Xu > Date: Fri, 4 Dec 2015 22:39:56 +0800 > > > When an rhashtable user pounds rhashtable hard with back-to-back > > insertions we may end up growing the table in GFP_ATOMIC context. > >

Re: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation

On Fri, Dec 04, 2015 at 09:45:20AM -0800, Eric Dumazet wrote: > On Fri, 2015-12-04 at 18:01 +0100, Phil Sutter wrote: > > On Fri, Dec 04, 2015 at 10:39:56PM +0800, Herbert Xu wrote: > > > On Thu, Dec 03, 2015 at 08:08:39AM -0800, Eric Dumazet wrote: > > > > > > > > Anyway, __vmalloc() can be used