Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-05 Thread Christoph Lameter
On Wed, 5 Jun 2013, JoonSoo Kim wrote: > @@ -2495,7 +2495,8 @@ static void __slab_free(struct kmem_cache *s, > struct page *page, > new.inuse--; > if ((!new.inuse || !prior) && !was_frozen) { > > - if (!kmem_cache_debug(s) && !prior) > +

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-05 Thread Christoph Lameter
On Wed, 5 Jun 2013, JoonSoo Kim wrote: > I knew that we need the other nodes pages if we consistently allocate > from there. > In allocation path, it is reasonable. > But, in free path, it is not reasonable that we put other nodes' page > to cpu partial list. It is reasonable. An "other nodes

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-05 Thread Christoph Lameter
On Wed, 5 Jun 2013, JoonSoo Kim wrote: I knew that we need the other nodes pages if we consistently allocate from there. In allocation path, it is reasonable. But, in free path, it is not reasonable that we put other nodes' page to cpu partial list. It is reasonable. An other nodes page on

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-05 Thread Christoph Lameter
On Wed, 5 Jun 2013, JoonSoo Kim wrote: @@ -2495,7 +2495,8 @@ static void __slab_free(struct kmem_cache *s, struct page *page, new.inuse--; if ((!new.inuse || !prior) !was_frozen) { - if (!kmem_cache_debug(s) !prior) +

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-04 Thread JoonSoo Kim
2013/6/4 Christoph Lameter : > On Tue, 4 Jun 2013, JoonSoo Kim wrote: > >> And I re-read Steven initial problem report in RT kernel and find that >> unfreeze_partial() do lock and unlock several times. This means that >> each page in cpu partial list doesn't come from same node. Why do we >> add

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-04 Thread JoonSoo Kim
2013/6/4 Christoph Lameter c...@linux.com: On Tue, 4 Jun 2013, JoonSoo Kim wrote: And I re-read Steven initial problem report in RT kernel and find that unfreeze_partial() do lock and unlock several times. This means that each page in cpu partial list doesn't come from same node. Why do we

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-03 Thread Christoph Lameter
On Tue, 4 Jun 2013, JoonSoo Kim wrote: > But, this modification adds lots of "#ifdef" and makes code less clean. > How about *not* removing struct page *partial in struct kmem_cache_cpu? > This introduces memory overhead and makes code bigger > for !CONFIG_SLUB_CPU_PARTIAL, but, this will help to

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-03 Thread Christoph Lameter
On Tue, 4 Jun 2013, JoonSoo Kim wrote: > And I re-read Steven initial problem report in RT kernel and find that > unfreeze_partial() do lock and unlock several times. This means that > each page in cpu partial list doesn't come from same node. Why do we > add other node's slab to this cpu partial

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-03 Thread JoonSoo Kim
Hello, Christoph. 2013/5/29 Christoph Lameter : > I just ran some quick tests and the following seems to work. > > Critical portions that need additional review (Joonsoo?) are the > modifications to get_partialinode() and __slab_free(). IMO, your code is fine to work. But, this modification adds

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-03 Thread JoonSoo Kim
Hello, Christoph. 2013/5/29 Christoph Lameter c...@linux.com: I just ran some quick tests and the following seems to work. Critical portions that need additional review (Joonsoo?) are the modifications to get_partialinode() and __slab_free(). IMO, your code is fine to work. But, this

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-03 Thread Christoph Lameter
On Tue, 4 Jun 2013, JoonSoo Kim wrote: And I re-read Steven initial problem report in RT kernel and find that unfreeze_partial() do lock and unlock several times. This means that each page in cpu partial list doesn't come from same node. Why do we add other node's slab to this cpu partial

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-06-03 Thread Christoph Lameter
On Tue, 4 Jun 2013, JoonSoo Kim wrote: But, this modification adds lots of #ifdef and makes code less clean. How about *not* removing struct page *partial in struct kmem_cache_cpu? This introduces memory overhead and makes code bigger for !CONFIG_SLUB_CPU_PARTIAL, but, this will help to make

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-05-28 Thread Christoph Lameter
I just ran some quick tests and the following seems to work. Critical portions that need additional review (Joonsoo?) are the modifications to get_partialinode() and __slab_free(). Subject: slub: Make cpu partial slab support configurable V2 cpu partial support can introduce level of

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-05-28 Thread Christoph Lameter
On Tue, 28 May 2013, Steven Rostedt wrote: > Any progress on this patch? Got a new vesion here but have not gotten too much testing done yet. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-05-28 Thread Steven Rostedt
On Thu, 2013-03-28 at 17:30 +, Christoph Lameter wrote: > This patch requires the earlier bug fix. > > > Subject: slub: Make cpu partial slab support configurable > > cpu partial support can introduce level of indeterminism that is not wanted > in certain context (like a realtime kernel).

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-05-28 Thread Steven Rostedt
On Thu, 2013-03-28 at 17:30 +, Christoph Lameter wrote: This patch requires the earlier bug fix. Subject: slub: Make cpu partial slab support configurable cpu partial support can introduce level of indeterminism that is not wanted in certain context (like a realtime kernel). Make it

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-05-28 Thread Christoph Lameter
On Tue, 28 May 2013, Steven Rostedt wrote: Any progress on this patch? Got a new vesion here but have not gotten too much testing done yet. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-05-28 Thread Christoph Lameter
I just ran some quick tests and the following seems to work. Critical portions that need additional review (Joonsoo?) are the modifications to get_partialinode() and __slab_free(). Subject: slub: Make cpu partial slab support configurable V2 cpu partial support can introduce level of

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-12 Thread Pekka Enberg
Hello, On 04/11/2013 07:42 PM, Christoph Lameter wrote: On Thu, 11 Apr 2013, Steven Rostedt wrote: I was wondering if you made any more forward progress with with patch yet. When it goes into mainline, I'd like to backport it to the -rt stable trees, and will probably make it enabled by

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-12 Thread Pekka Enberg
Hello, On 04/11/2013 07:42 PM, Christoph Lameter wrote: On Thu, 11 Apr 2013, Steven Rostedt wrote: I was wondering if you made any more forward progress with with patch yet. When it goes into mainline, I'd like to backport it to the -rt stable trees, and will probably make it enabled by

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-11 Thread Christoph Lameter
On Thu, 11 Apr 2013, Steven Rostedt wrote: > I was wondering if you made any more forward progress with with patch > yet. When it goes into mainline, I'd like to backport it to the -rt > stable trees, and will probably make it enabled by default when > PREEMPT_RT is enabled. Sorry I did not get

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-11 Thread Steven Rostedt
On Thu, 2013-03-28 at 17:30 +, Christoph Lameter wrote: > This patch requires the earlier bug fix. > > > Subject: slub: Make cpu partial slab support configurable > > cpu partial support can introduce level of indeterminism that is not wanted > in certain context (like a realtime kernel).

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-11 Thread Steven Rostedt
On Thu, 2013-03-28 at 17:30 +, Christoph Lameter wrote: This patch requires the earlier bug fix. Subject: slub: Make cpu partial slab support configurable cpu partial support can introduce level of indeterminism that is not wanted in certain context (like a realtime kernel). Make it

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-11 Thread Christoph Lameter
On Thu, 11 Apr 2013, Steven Rostedt wrote: I was wondering if you made any more forward progress with with patch yet. When it goes into mainline, I'd like to backport it to the -rt stable trees, and will probably make it enabled by default when PREEMPT_RT is enabled. Sorry I did not get

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Christoph Lameter
On Wed, 10 Apr 2013, Joonsoo Kim wrote: > > Christoph, Joonsoo, comments? > > Steven's comment is reasonable to me. > > If there is no objection from Christoph, > let's drop a patch in which I implement Christoph's idea. Fine with me. I do not like passing a reference just because we have to

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Steven Rostedt
On Wed, 2013-04-10 at 14:00 +, Christoph Lameter wrote: > On Wed, 10 Apr 2013, Joonsoo Kim wrote: > > > > Christoph, Joonsoo, comments? > > > > Steven's comment is reasonable to me. > > > > If there is no objection from Christoph, > > let's drop a patch in which I implement Christoph's idea.

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Joonsoo Kim
On Wed, Apr 10, 2013 at 09:31:10AM +0300, Pekka Enberg wrote: > On Mon, Apr 8, 2013 at 3:32 PM, Steven Rostedt wrote: > >> > Index: linux/mm/slub.c > >> > === > >> > --- linux.orig/mm/slub.c2013-03-28 12:14:26.958358688 -0500 >

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Pekka Enberg
On Mon, Apr 8, 2013 at 3:32 PM, Steven Rostedt wrote: >> > Index: linux/mm/slub.c >> > === >> > --- linux.orig/mm/slub.c2013-03-28 12:14:26.958358688 -0500 >> > +++ linux/mm/slub.c 2013-04-01 10:23:24.677584499 -0500 >> > @@

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Pekka Enberg
On Mon, Apr 8, 2013 at 3:32 PM, Steven Rostedt rost...@goodmis.org wrote: Index: linux/mm/slub.c === --- linux.orig/mm/slub.c2013-03-28 12:14:26.958358688 -0500 +++ linux/mm/slub.c 2013-04-01 10:23:24.677584499 -0500 @@

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Joonsoo Kim
On Wed, Apr 10, 2013 at 09:31:10AM +0300, Pekka Enberg wrote: On Mon, Apr 8, 2013 at 3:32 PM, Steven Rostedt rost...@goodmis.org wrote: Index: linux/mm/slub.c === --- linux.orig/mm/slub.c2013-03-28 12:14:26.958358688

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Steven Rostedt
On Wed, 2013-04-10 at 14:00 +, Christoph Lameter wrote: On Wed, 10 Apr 2013, Joonsoo Kim wrote: Christoph, Joonsoo, comments? Steven's comment is reasonable to me. If there is no objection from Christoph, let's drop a patch in which I implement Christoph's idea. Fine with

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-10 Thread Christoph Lameter
On Wed, 10 Apr 2013, Joonsoo Kim wrote: Christoph, Joonsoo, comments? Steven's comment is reasonable to me. If there is no objection from Christoph, let's drop a patch in which I implement Christoph's idea. Fine with me. I do not like passing a reference just because we have to return an

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-08 Thread Steven Rostedt
On Tue, 2013-04-02 at 09:42 +0900, Joonsoo Kim wrote: > > > > Signed-off-by: Christoph Lameter > > > > Index: linux/mm/slub.c > > === > > --- linux.orig/mm/slub.c2013-03-28 12:14:26.958358688 -0500 > > +++ linux/mm/slub.c

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-08 Thread Steven Rostedt
Sorry for the late reply, but I just got back from vacation. On Thu, 2013-03-28 at 17:29 +, Christoph Lameter wrote: > Two patches against slub to enable deconfiguring cpu_partial support. > > First one is a bug fix (Pekka please pick up this one or use Joonsoo's > earlier one). > > >

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-08 Thread Steven Rostedt
Sorry for the late reply, but I just got back from vacation. On Thu, 2013-03-28 at 17:29 +, Christoph Lameter wrote: Two patches against slub to enable deconfiguring cpu_partial support. First one is a bug fix (Pekka please pick up this one or use Joonsoo's earlier one). Subject:

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-08 Thread Steven Rostedt
On Tue, 2013-04-02 at 09:42 +0900, Joonsoo Kim wrote: Signed-off-by: Christoph Lameter c...@linux.com Index: linux/mm/slub.c === --- linux.orig/mm/slub.c2013-03-28 12:14:26.958358688 -0500 +++ linux/mm/slub.c

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-05 Thread Christoph Lameter
On Fri, 5 Apr 2013, Joonsoo Kim wrote: > Here goes a patch implementing Christoph's idea. > Instead of updating my previous patch, I re-write this patch on top of > your slab/next tree. Acked-by: Christoph Lameter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-05 Thread Christoph Lameter
On Fri, 5 Apr 2013, Joonsoo Kim wrote: Here goes a patch implementing Christoph's idea. Instead of updating my previous patch, I re-write this patch on top of your slab/next tree. Acked-by: Christoph Lameter c...@linux.com -- To unsubscribe from this list: send the line unsubscribe

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-04 Thread Joonsoo Kim
On Thu, Apr 04, 2013 at 01:53:25PM +, Christoph Lameter wrote: > On Thu, 4 Apr 2013, Joonsoo Kim wrote: > > > Pekka alreay applied it. > > Do we need update? > > Well I thought the passing of the count via lru.next would be something > worthwhile to pick up. > > -- > To unsubscribe from

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-04 Thread Christoph Lameter
On Thu, 4 Apr 2013, Joonsoo Kim wrote: > Pekka alreay applied it. > Do we need update? Well I thought the passing of the count via lru.next would be something worthwhile to pick up. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-04 Thread Christoph Lameter
On Thu, 4 Apr 2013, Joonsoo Kim wrote: Pekka alreay applied it. Do we need update? Well I thought the passing of the count via lru.next would be something worthwhile to pick up. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-04 Thread Joonsoo Kim
On Thu, Apr 04, 2013 at 01:53:25PM +, Christoph Lameter wrote: On Thu, 4 Apr 2013, Joonsoo Kim wrote: Pekka alreay applied it. Do we need update? Well I thought the passing of the count via lru.next would be something worthwhile to pick up. -- To unsubscribe from this list: send

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-03 Thread Joonsoo Kim
Hello, Christoph. On Tue, Apr 02, 2013 at 07:25:20PM +, Christoph Lameter wrote: > On Tue, 2 Apr 2013, Joonsoo Kim wrote: > > > We need one more fix for correctness. > > When available is assigned by put_cpu_partial, it doesn't count cpu slab's > > objects. > > Please reference my old

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-03 Thread Joonsoo Kim
Hello, Christoph. On Tue, Apr 02, 2013 at 07:25:20PM +, Christoph Lameter wrote: On Tue, 2 Apr 2013, Joonsoo Kim wrote: We need one more fix for correctness. When available is assigned by put_cpu_partial, it doesn't count cpu slab's objects. Please reference my old patch.

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-02 Thread Christoph Lameter
On Tue, 2 Apr 2013, Joonsoo Kim wrote: > We need one more fix for correctness. > When available is assigned by put_cpu_partial, it doesn't count cpu slab's > objects. > Please reference my old patch. > > https://lkml.org/lkml/2013/1/21/64 Could you update your patch and submit it again? -- To

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-02 Thread Pekka Enberg
On Tue, Apr 2, 2013 at 3:42 AM, Joonsoo Kim wrote: >> I have a vague memory that Joonsoo also ran into this issue awhile back. > > Yes. I sent a patch for this two month ago. :) It's applied now. Pekka -- To unsubscribe from this list: send the line "unsubscribe

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-02 Thread Pekka Enberg
On Tue, Apr 2, 2013 at 3:42 AM, Joonsoo Kim iamjoonsoo@lge.com wrote: I have a vague memory that Joonsoo also ran into this issue awhile back. Yes. I sent a patch for this two month ago. :) It's applied now. Pekka -- To unsubscribe from this list: send the line

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-02 Thread Christoph Lameter
On Tue, 2 Apr 2013, Joonsoo Kim wrote: We need one more fix for correctness. When available is assigned by put_cpu_partial, it doesn't count cpu slab's objects. Please reference my old patch. https://lkml.org/lkml/2013/1/21/64 Could you update your patch and submit it again? -- To

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Joonsoo Kim
Hello, Christoph. On Mon, Apr 01, 2013 at 03:32:43PM +, Christoph Lameter wrote: > On Thu, 28 Mar 2013, Paul Gortmaker wrote: > > > > Index: linux/init/Kconfig > > > === > > > --- linux.orig/init/Kconfig 2013-03-28

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Joonsoo Kim
Hello, Christoph. On Mon, Apr 01, 2013 at 03:33:23PM +, Christoph Lameter wrote: > Subject: slub: Fix object counts in acquire_slab V2 > > It seems that we were overallocating objects from the slab queues > since get_partial_node() assumed that page->inuse was undisturbed by >

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Paul Gortmaker
[Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.] On 01/04/2013 (Mon 12:06) Paul Gortmaker wrote: > On 13-04-01 11:32 AM, Christoph Lameter wrote: [...] > > Here is an updated patch. I will also send an updated fixup patch. > > I'll give t

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Paul Gortmaker
On 13-04-01 11:32 AM, Christoph Lameter wrote: [...] > @@ -4583,6 +4615,7 @@ static ssize_t min_partial_store(struct > } > SLAB_ATTR(min_partial); > > +#ifdef CONFIG_CPU_PARTIAL Above causes build failures when stats are on, because the name is wrong and hence is never defined, and >

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Paul Gortmaker
On 13-04-01 11:32 AM, Christoph Lameter wrote: > On Thu, 28 Mar 2013, Paul Gortmaker wrote: > >>> Index: linux/init/Kconfig >>> === >>> --- linux.orig/init/Kconfig 2013-03-28 12:14:26.958358688 -0500 >>> +++ linux/init/Kconfig

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Christoph Lameter
On Thu, 28 Mar 2013, Paul Gortmaker wrote: > > Index: linux/init/Kconfig > > === > > --- linux.orig/init/Kconfig 2013-03-28 12:14:26.958358688 -0500 > > +++ linux/init/Kconfig 2013-03-28 12:19:46.275866639 -0500 > > @@ -1514,6

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Christoph Lameter
Subject: slub: Fix object counts in acquire_slab V2 It seems that we were overallocating objects from the slab queues since get_partial_node() assumed that page->inuse was undisturbed by acquire_slab(). Save the # of objects in page->lru.next in acquire_slab() and pass it to get_partial_node()

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Christoph Lameter
Subject: slub: Fix object counts in acquire_slab V2 It seems that we were overallocating objects from the slab queues since get_partial_node() assumed that page-inuse was undisturbed by acquire_slab(). Save the # of objects in page-lru.next in acquire_slab() and pass it to get_partial_node() that

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Christoph Lameter
On Thu, 28 Mar 2013, Paul Gortmaker wrote: Index: linux/init/Kconfig === --- linux.orig/init/Kconfig 2013-03-28 12:14:26.958358688 -0500 +++ linux/init/Kconfig 2013-03-28 12:19:46.275866639 -0500 @@ -1514,6 +1514,14 @@

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Paul Gortmaker
On 13-04-01 11:32 AM, Christoph Lameter wrote: On Thu, 28 Mar 2013, Paul Gortmaker wrote: Index: linux/init/Kconfig === --- linux.orig/init/Kconfig 2013-03-28 12:14:26.958358688 -0500 +++ linux/init/Kconfig 2013-03-28

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Paul Gortmaker
On 13-04-01 11:32 AM, Christoph Lameter wrote: [...] @@ -4583,6 +4615,7 @@ static ssize_t min_partial_store(struct } SLAB_ATTR(min_partial); +#ifdef CONFIG_CPU_PARTIAL Above causes build failures when stats are on, because the name is wrong and hence is never defined, and static

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Paul Gortmaker
[Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.] On 01/04/2013 (Mon 12:06) Paul Gortmaker wrote: On 13-04-01 11:32 AM, Christoph Lameter wrote: [...] Here is an updated patch. I will also send an updated fixup patch. I'll give these some local testing

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Joonsoo Kim
Hello, Christoph. On Mon, Apr 01, 2013 at 03:33:23PM +, Christoph Lameter wrote: Subject: slub: Fix object counts in acquire_slab V2 It seems that we were overallocating objects from the slab queues since get_partial_node() assumed that page-inuse was undisturbed by acquire_slab(). Save

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-04-01 Thread Joonsoo Kim
Hello, Christoph. On Mon, Apr 01, 2013 at 03:32:43PM +, Christoph Lameter wrote: On Thu, 28 Mar 2013, Paul Gortmaker wrote: Index: linux/init/Kconfig === --- linux.orig/init/Kconfig 2013-03-28 12:14:26.958358688

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-28 Thread Paul Gortmaker
On Thu, Mar 28, 2013 at 1:30 PM, Christoph Lameter wrote: > This patch requires the earlier bug fix. > > > Subject: slub: Make cpu partial slab support configurable Hi Christoph, Minor nit: Applying: slub: Make cpu partial slab support configurable

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-28 Thread Christoph Lameter
This patch requires the earlier bug fix. Subject: slub: Make cpu partial slab support configurable cpu partial support can introduce level of indeterminism that is not wanted in certain context (like a realtime kernel). Make it configurable. Signed-off-by: Christoph Lameter Index:

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-28 Thread Christoph Lameter
Two patches against slub to enable deconfiguring cpu_partial support. First one is a bug fix (Pekka please pick up this one or use Joonsoo's earlier one). Subject: slub: Fix object counts in acquire_slab It seems that we were overallocating objects from the slab queues since get_partial_node()

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-28 Thread Christoph Lameter
Two patches against slub to enable deconfiguring cpu_partial support. First one is a bug fix (Pekka please pick up this one or use Joonsoo's earlier one). Subject: slub: Fix object counts in acquire_slab It seems that we were overallocating objects from the slab queues since get_partial_node()

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-28 Thread Christoph Lameter
This patch requires the earlier bug fix. Subject: slub: Make cpu partial slab support configurable cpu partial support can introduce level of indeterminism that is not wanted in certain context (like a realtime kernel). Make it configurable. Signed-off-by: Christoph Lameter c...@linux.com

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-28 Thread Paul Gortmaker
On Thu, Mar 28, 2013 at 1:30 PM, Christoph Lameter c...@linux.com wrote: This patch requires the earlier bug fix. Subject: slub: Make cpu partial slab support configurable Hi Christoph, Minor nit: Applying: slub: Make cpu partial slab support configurable

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-27 Thread Joonsoo Kim
On Tue, Mar 26, 2013 at 11:30:32PM -0400, Steven Rostedt wrote: > On Wed, 2013-03-27 at 11:59 +0900, Joonsoo Kim wrote: > > > How about using spin_try_lock() in unfreeze_partials() and > > using spin_lock_contented() in get_partial_node() to reduce latency? > > IMHO, this doesn't make code more

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-27 Thread Joonsoo Kim
On Tue, Mar 26, 2013 at 11:30:32PM -0400, Steven Rostedt wrote: On Wed, 2013-03-27 at 11:59 +0900, Joonsoo Kim wrote: How about using spin_try_lock() in unfreeze_partials() and using spin_lock_contented() in get_partial_node() to reduce latency? IMHO, this doesn't make code more

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-26 Thread Steven Rostedt
On Wed, 2013-03-27 at 11:59 +0900, Joonsoo Kim wrote: > How about using spin_try_lock() in unfreeze_partials() and > using spin_lock_contented() in get_partial_node() to reduce latency? > IMHO, this doesn't make code more deterministic, but can maintain > a benefit of cpu partial page with

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-26 Thread Joonsoo Kim
On Mon, Mar 25, 2013 at 02:32:35PM -0400, Steven Rostedt wrote: > On Mon, 2013-03-25 at 18:27 +, Christoph Lameter wrote: > > On Mon, 25 Mar 2013, Steven Rostedt wrote: > > > > > If this makes it more deterministic, and lower worse case latencies, > > > then it's definitely worth the price. >

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-26 Thread Steven Rostedt
On Mon, 2013-03-25 at 14:34 +, Christoph Lameter wrote: > You can eliminate the whole thing by setting > > /sys/kernel/slab/ > to zero. Giving you an update. I did the following: # ls /sys/kernel/slab/*/cpu_partial | while read f; do echo 0 > $f done Then ran our test suite for

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-26 Thread Steven Rostedt
On Mon, 2013-03-25 at 14:34 +, Christoph Lameter wrote: You can eliminate the whole thing by setting /sys/kernel/slab/whatever/cpu_partial to zero. Giving you an update. I did the following: # ls /sys/kernel/slab/*/cpu_partial | while read f; do echo 0 $f done Then ran

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-26 Thread Joonsoo Kim
On Mon, Mar 25, 2013 at 02:32:35PM -0400, Steven Rostedt wrote: On Mon, 2013-03-25 at 18:27 +, Christoph Lameter wrote: On Mon, 25 Mar 2013, Steven Rostedt wrote: If this makes it more deterministic, and lower worse case latencies, then it's definitely worth the price. Yes

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-26 Thread Steven Rostedt
On Wed, 2013-03-27 at 11:59 +0900, Joonsoo Kim wrote: How about using spin_try_lock() in unfreeze_partials() and using spin_lock_contented() in get_partial_node() to reduce latency? IMHO, this doesn't make code more deterministic, but can maintain a benefit of cpu partial page with tolerable

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Christoph Lameter
On Mon, 25 Mar 2013, Steven Rostedt wrote: > If this makes it more deterministic, and lower worse case latencies, > then it's definitely worth the price. Yes that would make it more deterministic. Maybe I should add an option to be able to compile the allocator without cpu partial page support?

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 18:27 +, Christoph Lameter wrote: > On Mon, 25 Mar 2013, Steven Rostedt wrote: > > > If this makes it more deterministic, and lower worse case latencies, > > then it's definitely worth the price. > > Yes that would make it more deterministic. Maybe I should add an

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 17:51 +, Christoph Lameter wrote: > On Mon, 25 Mar 2013, Steven Rostedt wrote: > > > BTW, what impact does setting all cpu_partial files to zero have on the > > system? > > Slight slowdown in kfree since the frees are less likely to be batched. If this makes it more

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Christoph Lameter
On Mon, 25 Mar 2013, Steven Rostedt wrote: > BTW, what impact does setting all cpu_partial files to zero have on the > system? Slight slowdown in kfree since the frees are less likely to be batched. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 11:57 -0400, Steven Rostedt wrote: > > > > You can eliminate the whole thing by setting > > > > /sys/kernel/slab/ > > > to zero. > > Thanks! I'll give this a try. > BTW, what impact does setting all cpu_partial files to zero have on the system? -- Steve -- To

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 14:34 +, Christoph Lameter wrote: > On Fri, 22 Mar 2013, Steven Rostedt wrote: > > > I uploaded it here: > > > > http://rostedt.homelinux.com/private/slab_partials > > > > See anything I should be worried about? > > Looks fine. Maybe its the spinlocks being contended

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Christoph Lameter
On Fri, 22 Mar 2013, Steven Rostedt wrote: > I uploaded it here: > > http://rostedt.homelinux.com/private/slab_partials > > See anything I should be worried about? Looks fine. Maybe its the spinlocks being contended that cause the long holdoff? How does RT deal with the spinlocks? Dont know too

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Christoph Lameter
On Fri, 22 Mar 2013, Steven Rostedt wrote: I uploaded it here: http://rostedt.homelinux.com/private/slab_partials See anything I should be worried about? Looks fine. Maybe its the spinlocks being contended that cause the long holdoff? How does RT deal with the spinlocks? Dont know too much

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 14:34 +, Christoph Lameter wrote: On Fri, 22 Mar 2013, Steven Rostedt wrote: I uploaded it here: http://rostedt.homelinux.com/private/slab_partials See anything I should be worried about? Looks fine. Maybe its the spinlocks being contended that cause the

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 11:57 -0400, Steven Rostedt wrote: You can eliminate the whole thing by setting /sys/kernel/slab/whatever/cpu_partial to zero. Thanks! I'll give this a try. BTW, what impact does setting all cpu_partial files to zero have on the system? -- Steve -- To

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Christoph Lameter
On Mon, 25 Mar 2013, Steven Rostedt wrote: BTW, what impact does setting all cpu_partial files to zero have on the system? Slight slowdown in kfree since the frees are less likely to be batched. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 17:51 +, Christoph Lameter wrote: On Mon, 25 Mar 2013, Steven Rostedt wrote: BTW, what impact does setting all cpu_partial files to zero have on the system? Slight slowdown in kfree since the frees are less likely to be batched. If this makes it more

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Steven Rostedt
On Mon, 2013-03-25 at 18:27 +, Christoph Lameter wrote: On Mon, 25 Mar 2013, Steven Rostedt wrote: If this makes it more deterministic, and lower worse case latencies, then it's definitely worth the price. Yes that would make it more deterministic. Maybe I should add an option to be

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-25 Thread Christoph Lameter
On Mon, 25 Mar 2013, Steven Rostedt wrote: If this makes it more deterministic, and lower worse case latencies, then it's definitely worth the price. Yes that would make it more deterministic. Maybe I should add an option to be able to compile the allocator without cpu partial page support?

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-22 Thread Steven Rostedt
On Fri, 2013-03-22 at 15:41 +, Christoph Lameter wrote: > The control is via the cpu_partial field in /sys/kernel/slab// > > There is also slabs_cpu_partial which gives a view as to how many objects > are cached in each per cpu structure. Do a cat > > /sys/kernel/*/slabs_cpu_partial to get

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-22 Thread Christoph Lameter
On Thu, 21 Mar 2013, Steven Rostedt wrote: > The 8115517a is just before put_cpu_partial() which calls > unfreeze_partials() with irqs disabled. So I started tracing again, this > time with: Hmmm... That is strange. unfreeze_partials should be rather fast. > It ran for 249.51

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-22 Thread Christoph Lameter
On Thu, 21 Mar 2013, Steven Rostedt wrote: The 8115517a is just before put_cpu_partial() which calls unfreeze_partials() with irqs disabled. So I started tracing again, this time with: Hmmm... That is strange. unfreeze_partials should be rather fast. It ran for 249.51

Re: [RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-22 Thread Steven Rostedt
On Fri, 2013-03-22 at 15:41 +, Christoph Lameter wrote: The control is via the cpu_partial field in /sys/kernel/slab/cache/ There is also slabs_cpu_partial which gives a view as to how many objects are cached in each per cpu structure. Do a cat /sys/kernel/*/slabs_cpu_partial to get a

[RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-21 Thread Steven Rostedt
Christoph, I've been trying to hunt down a latency on a 40 core box, where funny, the more tracing we enabled the less likely it would hit. It would take up to 12 hours to hit the latency, and if we had too much tracing on, it never would hit :-p Thus I would add little by little to find where

[RT LATENCY] 249 microsecond latency caused by slub's unfreeze_partials() code.

2013-03-21 Thread Steven Rostedt
Christoph, I've been trying to hunt down a latency on a 40 core box, where funny, the more tracing we enabled the less likely it would hit. It would take up to 12 hours to hit the latency, and if we had too much tracing on, it never would hit :-p Thus I would add little by little to find where