Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka J Enberg wrote:

Hi Christoph,

On Wed, 11 Jul 2007, Christoph Lameter wrote:

Of course you are the maintainer but you only authored a single patch 
which was the original submission in all the time that SLOB was in the 
tree. I keep having to clean up the allocator that has--according to 
Pekka--more memory requirements than SLUB. There is no point in keeping it 
around anymore it seems.



Well, it was a test setup with UML and busybox and didn't have all the 
SLOB optimizations Nick mentioned, so we shouldn't draw any definite 
conclusions from it. I couldn't get 2.6.22-rc6-mm1 to compile so I'll try 
again after Andrew pushes a new release out.


Furthermore, as much as I would like to see SLOB nuked too, we can't do 
that until Matt and Nick are satisfied with SLUB for small devices and 
what I can gather, they aren't.


Just to be clear: I do really like SLUB of course. And if that was able
to get as good or nearly as good (for appropriate values of nearly) memory
efficiency as SLOB in relevant situations, that would be fantastic and
SLOB could go away.

I don't really have a good knowledge of small memory devices being used,
other than apparently they can boot with 2MB (maybe less with nommu?). So
even a few K could be very significant for these.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

On Wed, 11 Jul 2007, Christoph Lameter wrote:
> But you did get 2.6.22 to compile it seems.
> 
> Here is the fix against 2.6.22-rc6-mm1 again.

I didn't get so far to compile with SLUB. There's some UML architecture 
problem and SLOB missing __kmalloc which UML also needs.

Pekka
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Wed, 11 Jul 2007, Pekka J Enberg wrote:

> Hi Christoph,
> 
> On Wed, 11 Jul 2007, Christoph Lameter wrote:
> > Of course you are the maintainer but you only authored a single patch 
> > which was the original submission in all the time that SLOB was in the 
> > tree. I keep having to clean up the allocator that has--according to 
> > Pekka--more memory requirements than SLUB. There is no point in keeping it 
> > around anymore it seems.
> 
> Well, it was a test setup with UML and busybox and didn't have all the 
> SLOB optimizations Nick mentioned, so we shouldn't draw any definite 
> conclusions from it. I couldn't get 2.6.22-rc6-mm1 to compile so I'll try 
> again after Andrew pushes a new release out.

But you did get 2.6.22 to compile it seems.

Here is the fix against 2.6.22-rc6-mm1 again.

---
 mm/slub.c |4 
 1 file changed, 4 insertions(+)

Index: linux-2.6.22-rc6-mm1/mm/slub.c
===
--- linux-2.6.22-rc6-mm1.orig/mm/slub.c 2007-07-06 13:28:57.0 -0700
+++ linux-2.6.22-rc6-mm1/mm/slub.c  2007-07-06 13:29:01.0 -0700
@@ -1868,7 +1868,9 @@ static void init_kmem_cache_node(struct 
atomic_long_set(>nr_slabs, 0);
spin_lock_init(>list_lock);
INIT_LIST_HEAD(>partial);
+#ifdef CONFIG_SLUB_DEBUG
INIT_LIST_HEAD(>full);
+#endif
 }
 
 #ifdef CONFIG_NUMA
@@ -1898,8 +1900,10 @@ static struct kmem_cache_node * __init e
page->freelist = get_freepointer(kmalloc_caches, n);
page->inuse++;
kmalloc_caches->node[node] = n;
+#ifdef CONFIG_SLUB_DEBUG
init_object(kmalloc_caches, n, 1);
init_tracking(kmalloc_caches, n);
+#endif
init_kmem_cache_node(n);
atomic_long_inc(>nr_slabs);
add_partial(n, page);
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Christoph,

On Wed, 11 Jul 2007, Christoph Lameter wrote:
> Of course you are the maintainer but you only authored a single patch 
> which was the original submission in all the time that SLOB was in the 
> tree. I keep having to clean up the allocator that has--according to 
> Pekka--more memory requirements than SLUB. There is no point in keeping it 
> around anymore it seems.

Well, it was a test setup with UML and busybox and didn't have all the 
SLOB optimizations Nick mentioned, so we shouldn't draw any definite 
conclusions from it. I couldn't get 2.6.22-rc6-mm1 to compile so I'll try 
again after Andrew pushes a new release out.

Furthermore, as much as I would like to see SLOB nuked too, we can't do 
that until Matt and Nick are satisfied with SLUB for small devices and 
what I can gather, they aren't.

Pekka
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

> >  git log --pretty=short mm/slob.c
> 
> A dozen trivial cleanups do not make you maintainer. Otherwise we'd
> all be sending our patches to Adrian rather than Linus.

Of course you are the maintainer but you only authored a single patch 
which was the original submission in all the time that SLOB was in the 
tree. I keep having to clean up the allocator that has--according to 
Pekka--more memory requirements than SLUB. There is no point in keeping it 
around anymore it seems.





-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

   git log --pretty=short mm/slob.c
 
 A dozen trivial cleanups do not make you maintainer. Otherwise we'd
 all be sending our patches to Adrian rather than Linus.

Of course you are the maintainer but you only authored a single patch 
which was the original submission in all the time that SLOB was in the 
tree. I keep having to clean up the allocator that has--according to 
Pekka--more memory requirements than SLUB. There is no point in keeping it 
around anymore it seems.





-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Christoph,

On Wed, 11 Jul 2007, Christoph Lameter wrote:
 Of course you are the maintainer but you only authored a single patch 
 which was the original submission in all the time that SLOB was in the 
 tree. I keep having to clean up the allocator that has--according to 
 Pekka--more memory requirements than SLUB. There is no point in keeping it 
 around anymore it seems.

Well, it was a test setup with UML and busybox and didn't have all the 
SLOB optimizations Nick mentioned, so we shouldn't draw any definite 
conclusions from it. I couldn't get 2.6.22-rc6-mm1 to compile so I'll try 
again after Andrew pushes a new release out.

Furthermore, as much as I would like to see SLOB nuked too, we can't do 
that until Matt and Nick are satisfied with SLUB for small devices and 
what I can gather, they aren't.

Pekka
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Wed, 11 Jul 2007, Pekka J Enberg wrote:

 Hi Christoph,
 
 On Wed, 11 Jul 2007, Christoph Lameter wrote:
  Of course you are the maintainer but you only authored a single patch 
  which was the original submission in all the time that SLOB was in the 
  tree. I keep having to clean up the allocator that has--according to 
  Pekka--more memory requirements than SLUB. There is no point in keeping it 
  around anymore it seems.
 
 Well, it was a test setup with UML and busybox and didn't have all the 
 SLOB optimizations Nick mentioned, so we shouldn't draw any definite 
 conclusions from it. I couldn't get 2.6.22-rc6-mm1 to compile so I'll try 
 again after Andrew pushes a new release out.

But you did get 2.6.22 to compile it seems.

Here is the fix against 2.6.22-rc6-mm1 again.

---
 mm/slub.c |4 
 1 file changed, 4 insertions(+)

Index: linux-2.6.22-rc6-mm1/mm/slub.c
===
--- linux-2.6.22-rc6-mm1.orig/mm/slub.c 2007-07-06 13:28:57.0 -0700
+++ linux-2.6.22-rc6-mm1/mm/slub.c  2007-07-06 13:29:01.0 -0700
@@ -1868,7 +1868,9 @@ static void init_kmem_cache_node(struct 
atomic_long_set(n-nr_slabs, 0);
spin_lock_init(n-list_lock);
INIT_LIST_HEAD(n-partial);
+#ifdef CONFIG_SLUB_DEBUG
INIT_LIST_HEAD(n-full);
+#endif
 }
 
 #ifdef CONFIG_NUMA
@@ -1898,8 +1900,10 @@ static struct kmem_cache_node * __init e
page-freelist = get_freepointer(kmalloc_caches, n);
page-inuse++;
kmalloc_caches-node[node] = n;
+#ifdef CONFIG_SLUB_DEBUG
init_object(kmalloc_caches, n, 1);
init_tracking(kmalloc_caches, n);
+#endif
init_kmem_cache_node(n);
atomic_long_inc(n-nr_slabs);
add_partial(n, page);
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

On Wed, 11 Jul 2007, Christoph Lameter wrote:
 But you did get 2.6.22 to compile it seems.
 
 Here is the fix against 2.6.22-rc6-mm1 again.

I didn't get so far to compile with SLUB. There's some UML architecture 
problem and SLOB missing __kmalloc which UML also needs.

Pekka
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka J Enberg wrote:

Hi Christoph,

On Wed, 11 Jul 2007, Christoph Lameter wrote:

Of course you are the maintainer but you only authored a single patch 
which was the original submission in all the time that SLOB was in the 
tree. I keep having to clean up the allocator that has--according to 
Pekka--more memory requirements than SLUB. There is no point in keeping it 
around anymore it seems.



Well, it was a test setup with UML and busybox and didn't have all the 
SLOB optimizations Nick mentioned, so we shouldn't draw any definite 
conclusions from it. I couldn't get 2.6.22-rc6-mm1 to compile so I'll try 
again after Andrew pushes a new release out.


Furthermore, as much as I would like to see SLOB nuked too, we can't do 
that until Matt and Nick are satisfied with SLUB for small devices and 
what I can gather, they aren't.


Just to be clear: I do really like SLUB of course. And if that was able
to get as good or nearly as good (for appropriate values of nearly) memory
efficiency as SLOB in relevant situations, that would be fantastic and
SLOB could go away.

I don't really have a good knowledge of small memory devices being used,
other than apparently they can boot with 2MB (maybe less with nommu?). So
even a few K could be very significant for these.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 06:37:42PM -0700, Christoph Lameter wrote:
> On Tue, 10 Jul 2007, Matt Mackall wrote:
> 
> > You're delusional.
> 
> Git log says otherwise:
> 
>  git log --pretty=short mm/slob.c

A dozen trivial cleanups do not make you maintainer. Otherwise we'd
all be sending our patches to Adrian rather than Linus.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

> You're delusional.


Git log says otherwise:

 git log --pretty=short mm/slob.c

Author: Christoph Lameter <[EMAIL PROTECTED]>
Remove SLAB_CTOR_CONSTRUCTOR
Author: Christoph Lameter <[EMAIL PROTECTED]>
Slab allocators: Drop support for destructors
Author: Nick Piggin <[EMAIL PROTECTED]>
slob: implement RCU freeing
Author: Akinobu Mita <[EMAIL PROTECTED]>
slob: fix page order calculation on not 4KB page
Author: Christoph Lameter <[EMAIL PROTECTED]>
slab allocators: Remove obsolete SLAB_MUST_HWCACHE_ALIGN
Author: Akinobu Mita <[EMAIL PROTECTED]>
slob: handle SLAB_PANIC flag
Author: Pekka Enberg <[EMAIL PROTECTED]>
slab: introduce krealloc
Author: Dimitri Gorokhovik <[EMAIL PROTECTED]>
[PATCH] MM: SLOB is broken by recent cleanup of slab.h
Author: Christoph Lameter <[EMAIL PROTECTED]>
[PATCH] More slab.h cleanups
Author: Christoph Lameter <[EMAIL PROTECTED]>
[PATCH] Cleanup slab headers / API to allow easy addition of new slab 
allocators
Author: Alexey Dobriyan <[EMAIL PROTECTED]>
[PATCH] Make kmem_cache_destroy() return void
Author: Christoph Lameter <[EMAIL PROTECTED]>
[PATCH] ZVC: Support NR_SLAB_RECLAIMABLE / NR_SLAB_UNRECLAIMABLE
Author: Christoph Lameter <[EMAIL PROTECTED]>
[PATCH] Extract the allocpercpu functions from the slab allocator
Author: Jörn Engel <[EMAIL PROTECTED]>
Remove obsolete #include 
Author: John Hawkes <[EMAIL PROTECTED]>
[PATCH] mm/slob.c: for_each_possible_cpu(), not NR_CPUS
Author: Pekka Enberg <[EMAIL PROTECTED]>
[PATCH] slab: introduce kmem_cache_zalloc allocator
Author: Ingo Molnar <[EMAIL PROTECTED]>
[PATCH] SLOB=y && SMP=y fix
Author: Matt Mackall <[EMAIL PROTECTED]>
[PATCH] slob: introduce the SLOB allocator

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 03:09:06PM -0700, Christoph Lameter wrote:
> On Tue, 10 Jul 2007, Nick Piggin wrote:
> 
> > But last time this discussion came up, IIRC you ended up handwaving
> > about all the ways in which SLOB was broken but didn't actually come
> > up with any real problems. Matt seemed willing to add those counters
> > or whatever it was if/when doing so solved a real problem. And remember
> > that SLOB doesn't have to have feature parity with SLUB, so long as it
> > implements the slab API such that the kernel *works*.
> 
> No it does not have to have feature parity. And yes we identified areas in 
> which SLOB may cause problems due to not implementing things (f.e. 
> suspend resume).

Please remind me what these things are. Suspend and hibernate work
fine here.

> The counters are still missing and thus core development 
> cannot rely on those being there.

None of the VM makes real use of the SLAB counters. Nor will they ever
make real use of them because the counters are not usefully defined.
In other words, reclaimable is a lie. If SLAB claims there's 1M of
reclaimable memory, the actual amount may more often than not be zero
because we already tried to shrink the SLAB.

As I've said before, if you come up with a real use for these numbers,
I will sacrifice the appropriate number of puppies to support it in
SLOB. But I'd rather not crush any puppies here prematurely.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

> Without the parameter, as the other way doesn't compile in -mm1.

here is the patch that went into mm after mm1 was released.

---
 mm/slub.c |4 
 1 file changed, 4 insertions(+)

Index: linux-2.6.22-rc6-mm1/mm/slub.c
===
--- linux-2.6.22-rc6-mm1.orig/mm/slub.c 2007-07-06 13:28:57.0 -0700
+++ linux-2.6.22-rc6-mm1/mm/slub.c  2007-07-06 13:29:01.0 -0700
@@ -1868,7 +1868,9 @@ static void init_kmem_cache_node(struct 
atomic_long_set(>nr_slabs, 0);
spin_lock_init(>list_lock);
INIT_LIST_HEAD(>partial);
+#ifdef CONFIG_SLUB_DEBUG
INIT_LIST_HEAD(>full);
+#endif
 }
 
 #ifdef CONFIG_NUMA
@@ -1898,8 +1900,10 @@ static struct kmem_cache_node * __init e
page->freelist = get_freepointer(kmalloc_caches, n);
page->inuse++;
kmalloc_caches->node[node] = n;
+#ifdef CONFIG_SLUB_DEBUG
init_object(kmalloc_caches, n, 1);
init_tracking(kmalloc_caches, n);
+#endif
init_kmem_cache_node(n);
atomic_long_inc(>nr_slabs);
add_partial(n, page);
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 03:12:38PM -0700, Christoph Lameter wrote:
> On Tue, 10 Jul 2007, Matt Mackall wrote:
> 
> > following as the best MemFree numbers after several boots each:
> > 
> > SLAB: 54796
> > SLOB: 55044
> > SLUB: 53944
> > SLUB: 54788 (debug turned off)
> 
> That was without "slub_debug" as a parameter or with !CONFIG_SLUB_DEBUG?

Without the parameter, as the other way doesn't compile in -mm1.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

> following as the best MemFree numbers after several boots each:
> 
> SLAB: 54796
> SLOB: 55044
> SLUB: 53944
> SLUB: 54788 (debug turned off)

That was without "slub_debug" as a parameter or with !CONFIG_SLUB_DEBUG?

Data size and code size will decrease if you compile with 
!CONFIG_SLUB_DEBUG. slub_debug on the command line governs if debug 
information is used.

> These numbers bounce around a lot more from boot to boot than I
> remember, so take these numbers with a grain of salt.
> 
> Disabling the debug code in the build gives this, by the way:
> 
> mm/slub.c: In function ÿÿinit_kmem_cache_nodeÿÿ:
> mm/slub.c:1873: error: ÿÿstruct kmem_cache_nodeÿÿ has no member named
> ÿÿfullÿÿ

A fix for that is in Andrew's tree.

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

> But last time this discussion came up, IIRC you ended up handwaving
> about all the ways in which SLOB was broken but didn't actually come
> up with any real problems. Matt seemed willing to add those counters
> or whatever it was if/when doing so solved a real problem. And remember
> that SLOB doesn't have to have feature parity with SLUB, so long as it
> implements the slab API such that the kernel *works*.

No it does not have to have feature parity. And yes we identified areas in 
which SLOB may cause problems due to not implementing things (f.e. 
suspend resume). The counters are still missing and thus core development 
cannot rely on those being there.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Matt,

On Tue, 10 Jul 2007, Matt Mackall wrote:
> Using 2.6.22-rc6-mm1 with a 64MB lguest and busybox, I'm seeing the
> following as the best MemFree numbers after several boots each:
> 
> SLAB: 54796
> SLOB: 55044
> SLUB: 53944
> SLUB: 54788 (debug turned off)
> 
> These numbers bounce around a lot more from boot to boot than I
> remember, so take these numbers with a grain of salt.

To rule out userland, 2.6.22 with 32 MB defconfig UML and busybox [1] on 
i386:

SLOB: 26708
SLUB: 27212 (no debug)

Unfortunately UML is broken in 2.6.22-rc6-mm1, so I don't know if SLOB 
patches help there.

  1. http://uml.nagafix.co.uk/BusyBox-1.5.0/BusyBox-1.5.0-x86-root_fs.bz2

Pekka
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 12:31:40PM +0300, Pekka Enberg wrote:
> Hi Nick,
> 
> Pekka J Enberg wrote:
> >> That's 92 KB advantage for SLUB with debugging enabled and 240 KB when
> >> debugging is disabled.
> 
> On 7/10/07, Nick Piggin <[EMAIL PROTECTED]> wrote:
> >Interesting. What kernel version are you using?
> 
> Linus' git head from yesterday so the results are likely to be
> sensitive to workload and mine doesn't represent real embedded use.

Using 2.6.22-rc6-mm1 with a 64MB lguest and busybox, I'm seeing the
following as the best MemFree numbers after several boots each:

SLAB: 54796
SLOB: 55044
SLUB: 53944
SLUB: 54788 (debug turned off)

These numbers bounce around a lot more from boot to boot than I
remember, so take these numbers with a grain of salt.

Disabling the debug code in the build gives this, by the way:

mm/slub.c: In function ‘init_kmem_cache_node’:
mm/slub.c:1873: error: ‘struct kmem_cache_node’ has no member named
‘full’

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka Enberg wrote:

Hi Nick,

Pekka J Enberg wrote:


> That's 92 KB advantage for SLUB with debugging enabled and 240 KB when
> debugging is disabled.



On 7/10/07, Nick Piggin <[EMAIL PROTECTED]> wrote:


Interesting. What kernel version are you using?



Linus' git head from yesterday so the results are likely to be
sensitive to workload and mine doesn't represent real embedded use.


Hi Pekka,

There is one thing that the SLOB patches in -mm do besides result in
slightly better packing and memory efficiency (which might be unlikely
to explain the difference you are seeing), and that is that they do
away with the delayed freeing of unused SLOB pages back to the page
allocator.

In git head, these pages are freed via a timer so they can take a
while to make their way back to the buddy allocator so they don't
register as free memory as such.

Anyway, I would be very interested to see any situation where the
SLOB in -mm uses more memory than SLUB, even on test configs like
yours.

Thanks,
Nick

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka Enberg]

Hi Nick,

Pekka J Enberg wrote:

> That's 92 KB advantage for SLUB with debugging enabled and 240 KB when
> debugging is disabled.


On 7/10/07, Nick Piggin <[EMAIL PROTECTED]> wrote:

Interesting. What kernel version are you using?


Linus' git head from yesterday so the results are likely to be
sensitive to workload and mine doesn't represent real embedded use.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Håvard Skinnemoen]

On 7/10/07, Nick Piggin <[EMAIL PROTECTED]> wrote:

Håvard Skinnemoen wrote:
> On 7/10/07, Matt Mackall <[EMAIL PROTECTED]> wrote:
>
>> The only remaining known bug is arguably a problem in nommu that SLOB
>> shouldn't be papering over.
>
>
> I've got another one for you: SLOB ignores ARCH_KMALLOC_MINALIGN so
> using SLOB in combination with DMA and non-coherent architectures
> causes data corruption.

Should be fixed in mm, I believe: slob-improved-alignment-handling.patch


Indeed. Thanks, I'll give it a try later today.

Håvard
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Håvard Skinnemoen wrote:

On 7/10/07, Matt Mackall <[EMAIL PROTECTED]> wrote:


The only remaining known bug is arguably a problem in nommu that SLOB
shouldn't be papering over.



I've got another one for you: SLOB ignores ARCH_KMALLOC_MINALIGN so
using SLOB in combination with DMA and non-coherent architectures
causes data corruption.


Should be fixed in mm, I believe: slob-improved-alignment-handling.patch

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Håvard Skinnemoen]

On 7/10/07, Matt Mackall <[EMAIL PROTECTED]> wrote:

The only remaining known bug is arguably a problem in nommu that SLOB
shouldn't be papering over.


I've got another one for you: SLOB ignores ARCH_KMALLOC_MINALIGN so
using SLOB in combination with DMA and non-coherent architectures
causes data corruption.

That said, there are currently very few architectures that define
ARCH_KMALLOC_MINALIGN, so I guess it might be something I'm doing
wrong on avr32. But I'd really like to know how other non-coherent
architectures handle DMA to buffers sharing cachelines with unrelated
data...

Håvard
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Mon, Jul 09, 2007 at 07:11:03PM -0700, Christoph Lameter wrote:
> On Tue, 10 Jul 2007, Nick Piggin wrote:
> 
> > It is reasonable to expect some help from maintainers, but I notice you
> > didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
> > if you tried working a bit closer with him you could get better results?
> 
> The maintainers last patch to SLOB was the initial submission of the 
> allocator. Then he acked subsequent patches. Most of the modifications to 
> SLOB are my work.

You're delusional.

http://www.kernel.org/hg/linux-2.6/annotate/tip/mm/slob.c

A grand total of 15 lines accounted to you, all of which are
completely trivial. Most of them are turning no-op macros into no-op
but non-inline functions. Not an improvement. Reverting that now on my
todo list, thanks for drawing my attention to it. The remainder is
just churn from your SLUB work.

While you're at it, note the lines from Dimitri, quickly fixing the
breakage you introduced.

Count many more lines from Nick, Pekka, and Akinobu making useful
changes. And note that all three of the real bugs fixed were fairly
hard to hit. It's unlikely that anyone would have ever hit the RCU
bug. The find_order bug only hit allocations of unlikely sizes like
8193 bytes. And SLAB_PANIC triggering is fairly unheard of, and simply
makes the kernel crash slightly sooner anyway.

The only remaining known bug is arguably a problem in nommu that SLOB
shouldn't be papering over.

That's pretty damn stable compared to the other allocators in the
kernel.

> Attempts to talk to the maintainer result in inventive explanations
> why SLOB does not have to conform to kernel standards.

Um.. would those be the same kernel standards you pulled out of your
ass to argue SLOB shouldn't exist?

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka J Enberg wrote:


Curious, /proc/meminfo immediately after boot shows:

SLUB (debugging enabled):

(none):~# cat /proc/meminfo 
MemTotal:30260 kB

MemFree: 22096 kB

SLUB (debugging disabled):

(none):~# cat /proc/meminfo 
MemTotal:30276 kB

MemFree: 22244 kB

SLOB:

(none):~# cat /proc/meminfo 
MemTotal:30280 kB

MemFree: 22004 kB

That's 92 KB advantage for SLUB with debugging enabled and 240 KB when 
debugging is disabled.


Interesting. What kernel version are you using?



Nick, Matt, care to retest SLUB and SLOB for your setups?


I don't think there has been a significant change in the area of
memory efficiency in either since I last tested, and Christoph and
I both produced the same result.

I can't say where SLOB is losing its memory, but there are a few
places that can still be improved, so I might get keen and take
another look at it once all the improvements to both allocators
gets upstream.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Christoph,

On Mon, 9 Jul 2007, Pekka Enberg wrote:
> > I assume with "slab external fragmentation" you mean allocating a
> > whole page for a slab when there are not enough objects to fill the
> > whole thing thus wasting memory? We could try to combat that by
> > packing multiple variable-sized slabs within a single page. Also,
> > adding some non-power-of-two kmalloc caches might help with internal
> > fragmentation.

On Mon, 9 Jul 2007, Christoph Lameter wrote:
> Ther are already non-power-of-two kmalloc caches for 96 and 192 bytes 
> sizes.

I know that, but for my setup at least, there seems to be a need for a 
non-power of two cache between 512 and 1024. What I am seeing is average 
allocation size for kmalloc-512 being around 270-280 which wastes total 
of 10 KB of memory due to internal fragmentation. Might be a buggy caller 
that can be fixed with its own cache too.

On Mon, 9 Jul 2007, Pekka Enberg wrote:
> > In any case, SLUB needs some serious tuning for smaller machines
> > before we can get rid of SLOB.

On Mon, 9 Jul 2007, Christoph Lameter wrote:
> Switch off CONFIG_SLUB_DEBUG to get memory savings.

Curious, /proc/meminfo immediately after boot shows:

SLUB (debugging enabled):

(none):~# cat /proc/meminfo 
MemTotal:30260 kB
MemFree: 22096 kB

SLUB (debugging disabled):

(none):~# cat /proc/meminfo 
MemTotal:30276 kB
MemFree: 22244 kB

SLOB:

(none):~# cat /proc/meminfo 
MemTotal:30280 kB
MemFree: 22004 kB

That's 92 KB advantage for SLUB with debugging enabled and 240 KB when 
debugging is disabled.

Nick, Matt, care to retest SLUB and SLOB for your setups?

Pekka
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Tue, 10 Jul 2007, Nick Piggin wrote:



It is reasonable to expect some help from maintainers, but I notice you
didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
if you tried working a bit closer with him you could get better results?



The maintainers last patch to SLOB was the initial submission of the 
allocator. Then he acked subsequent patches. Most of the modifications to 
SLOB are my work. Attempts to talk to the maintainer result in inventive 
explanations why SLOB does not have to conform to kernel standards. There 
is no reasonable expectation that this will change.


Well I really don't want to mediate, but even in the case of a
completely MIA maintainer, that isn't really a good idea to throw out
working and useful code.

But last time this discussion came up, IIRC you ended up handwaving
about all the ways in which SLOB was broken but didn't actually come
up with any real problems. Matt seemed willing to add those counters
or whatever it was if/when doing so solved a real problem. And remember
that SLOB doesn't have to have feature parity with SLUB, so long as it
implements the slab API such that the kernel *works*.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Matt Mackall wrote:

On Tue, Jul 10, 2007 at 11:58:44AM +1000, Nick Piggin wrote:



Just a fancy way of saying roughly that memory waste will increase as
the size of the system increases. But that aspect of it I think is
not really a problem for non-tiny systems anyway because the waste
tends not to be too bad (and possibly the number of active allocations
does not increase O(n) with the size of RAM either).



If active allocations doesn't increase O(n) with the size of RAM,
what's all that RAM for?

If your memory isn't getting used for large VMAs or large amounts of
page cache, that means it's getting used by task structs,
radix_tree_nodes, sockets, dentries, inodes, etc.


Yeah you could be right. Actually you most likey _are_ right for many
workloads.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 11:58:44AM +1000, Nick Piggin wrote:
> Christoph Lameter wrote:
> >On Tue, 10 Jul 2007, Nick Piggin wrote:
> >
> >
> >>>O(n) memory savings? What is that?
> >>
> >>Allocate n things and your memory waste is proportional to n (well that's
> >>O(n) waste, so I guess by savings I mean that SLOB's memory saving 
> >>compared
> >>to SLUB are proportional to n).
> >
> >
> >n is the size of the object?
> 
> n things -- n number of things (n calls to kmem_cache_alloc()).
> 
> Just a fancy way of saying roughly that memory waste will increase as
> the size of the system increases. But that aspect of it I think is
> not really a problem for non-tiny systems anyway because the waste
> tends not to be too bad (and possibly the number of active allocations
> does not increase O(n) with the size of RAM either).

If active allocations doesn't increase O(n) with the size of RAM,
what's all that RAM for?

If your memory isn't getting used for large VMAs or large amounts of
page cache, that means it's getting used by task structs,
radix_tree_nodes, sockets, dentries, inodes, etc.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 11:58:44AM +1000, Nick Piggin wrote:
 Christoph Lameter wrote:
 On Tue, 10 Jul 2007, Nick Piggin wrote:
 
 
 O(n) memory savings? What is that?
 
 Allocate n things and your memory waste is proportional to n (well that's
 O(n) waste, so I guess by savings I mean that SLOB's memory saving 
 compared
 to SLUB are proportional to n).
 
 
 n is the size of the object?
 
 n things -- n number of things (n calls to kmem_cache_alloc()).
 
 Just a fancy way of saying roughly that memory waste will increase as
 the size of the system increases. But that aspect of it I think is
 not really a problem for non-tiny systems anyway because the waste
 tends not to be too bad (and possibly the number of active allocations
 does not increase O(n) with the size of RAM either).

If active allocations doesn't increase O(n) with the size of RAM,
what's all that RAM for?

If your memory isn't getting used for large VMAs or large amounts of
page cache, that means it's getting used by task structs,
radix_tree_nodes, sockets, dentries, inodes, etc.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Matt Mackall wrote:

On Tue, Jul 10, 2007 at 11:58:44AM +1000, Nick Piggin wrote:



Just a fancy way of saying roughly that memory waste will increase as
the size of the system increases. But that aspect of it I think is
not really a problem for non-tiny systems anyway because the waste
tends not to be too bad (and possibly the number of active allocations
does not increase O(n) with the size of RAM either).



If active allocations doesn't increase O(n) with the size of RAM,
what's all that RAM for?

If your memory isn't getting used for large VMAs or large amounts of
page cache, that means it's getting used by task structs,
radix_tree_nodes, sockets, dentries, inodes, etc.


Yeah you could be right. Actually you most likey _are_ right for many
workloads.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Tue, 10 Jul 2007, Nick Piggin wrote:



It is reasonable to expect some help from maintainers, but I notice you
didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
if you tried working a bit closer with him you could get better results?



The maintainers last patch to SLOB was the initial submission of the 
allocator. Then he acked subsequent patches. Most of the modifications to 
SLOB are my work. Attempts to talk to the maintainer result in inventive 
explanations why SLOB does not have to conform to kernel standards. There 
is no reasonable expectation that this will change.


Well I really don't want to mediate, but even in the case of a
completely MIA maintainer, that isn't really a good idea to throw out
working and useful code.

But last time this discussion came up, IIRC you ended up handwaving
about all the ways in which SLOB was broken but didn't actually come
up with any real problems. Matt seemed willing to add those counters
or whatever it was if/when doing so solved a real problem. And remember
that SLOB doesn't have to have feature parity with SLUB, so long as it
implements the slab API such that the kernel *works*.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Christoph,

On Mon, 9 Jul 2007, Pekka Enberg wrote:
  I assume with slab external fragmentation you mean allocating a
  whole page for a slab when there are not enough objects to fill the
  whole thing thus wasting memory? We could try to combat that by
  packing multiple variable-sized slabs within a single page. Also,
  adding some non-power-of-two kmalloc caches might help with internal
  fragmentation.

On Mon, 9 Jul 2007, Christoph Lameter wrote:
 Ther are already non-power-of-two kmalloc caches for 96 and 192 bytes 
 sizes.

I know that, but for my setup at least, there seems to be a need for a 
non-power of two cache between 512 and 1024. What I am seeing is average 
allocation size for kmalloc-512 being around 270-280 which wastes total 
of 10 KB of memory due to internal fragmentation. Might be a buggy caller 
that can be fixed with its own cache too.

On Mon, 9 Jul 2007, Pekka Enberg wrote:
  In any case, SLUB needs some serious tuning for smaller machines
  before we can get rid of SLOB.

On Mon, 9 Jul 2007, Christoph Lameter wrote:
 Switch off CONFIG_SLUB_DEBUG to get memory savings.

Curious, /proc/meminfo immediately after boot shows:

SLUB (debugging enabled):

(none):~# cat /proc/meminfo 
MemTotal:30260 kB
MemFree: 22096 kB

SLUB (debugging disabled):

(none):~# cat /proc/meminfo 
MemTotal:30276 kB
MemFree: 22244 kB

SLOB:

(none):~# cat /proc/meminfo 
MemTotal:30280 kB
MemFree: 22004 kB

That's 92 KB advantage for SLUB with debugging enabled and 240 KB when 
debugging is disabled.

Nick, Matt, care to retest SLUB and SLOB for your setups?

Pekka
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka J Enberg wrote:


Curious, /proc/meminfo immediately after boot shows:

SLUB (debugging enabled):

(none):~# cat /proc/meminfo 
MemTotal:30260 kB

MemFree: 22096 kB

SLUB (debugging disabled):

(none):~# cat /proc/meminfo 
MemTotal:30276 kB

MemFree: 22244 kB

SLOB:

(none):~# cat /proc/meminfo 
MemTotal:30280 kB

MemFree: 22004 kB

That's 92 KB advantage for SLUB with debugging enabled and 240 KB when 
debugging is disabled.


Interesting. What kernel version are you using?



Nick, Matt, care to retest SLUB and SLOB for your setups?


I don't think there has been a significant change in the area of
memory efficiency in either since I last tested, and Christoph and
I both produced the same result.

I can't say where SLOB is losing its memory, but there are a few
places that can still be improved, so I might get keen and take
another look at it once all the improvements to both allocators
gets upstream.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Mon, Jul 09, 2007 at 07:11:03PM -0700, Christoph Lameter wrote:
 On Tue, 10 Jul 2007, Nick Piggin wrote:
 
  It is reasonable to expect some help from maintainers, but I notice you
  didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
  if you tried working a bit closer with him you could get better results?
 
 The maintainers last patch to SLOB was the initial submission of the 
 allocator. Then he acked subsequent patches. Most of the modifications to 
 SLOB are my work.

You're delusional.

http://www.kernel.org/hg/linux-2.6/annotate/tip/mm/slob.c

A grand total of 15 lines accounted to you, all of which are
completely trivial. Most of them are turning no-op macros into no-op
but non-inline functions. Not an improvement. Reverting that now on my
todo list, thanks for drawing my attention to it. The remainder is
just churn from your SLUB work.

While you're at it, note the lines from Dimitri, quickly fixing the
breakage you introduced.

Count many more lines from Nick, Pekka, and Akinobu making useful
changes. And note that all three of the real bugs fixed were fairly
hard to hit. It's unlikely that anyone would have ever hit the RCU
bug. The find_order bug only hit allocations of unlikely sizes like
8193 bytes. And SLAB_PANIC triggering is fairly unheard of, and simply
makes the kernel crash slightly sooner anyway.

The only remaining known bug is arguably a problem in nommu that SLOB
shouldn't be papering over.

That's pretty damn stable compared to the other allocators in the
kernel.

 Attempts to talk to the maintainer result in inventive explanations
 why SLOB does not have to conform to kernel standards.

Um.. would those be the same kernel standards you pulled out of your
ass to argue SLOB shouldn't exist?

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Håvard Skinnemoen]

On 7/10/07, Matt Mackall [EMAIL PROTECTED] wrote:

The only remaining known bug is arguably a problem in nommu that SLOB
shouldn't be papering over.


I've got another one for you: SLOB ignores ARCH_KMALLOC_MINALIGN so
using SLOB in combination with DMA and non-coherent architectures
causes data corruption.

That said, there are currently very few architectures that define
ARCH_KMALLOC_MINALIGN, so I guess it might be something I'm doing
wrong on avr32. But I'd really like to know how other non-coherent
architectures handle DMA to buffers sharing cachelines with unrelated
data...

Håvard
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Håvard Skinnemoen wrote:

On 7/10/07, Matt Mackall [EMAIL PROTECTED] wrote:


The only remaining known bug is arguably a problem in nommu that SLOB
shouldn't be papering over.



I've got another one for you: SLOB ignores ARCH_KMALLOC_MINALIGN so
using SLOB in combination with DMA and non-coherent architectures
causes data corruption.


Should be fixed in mm, I believe: slob-improved-alignment-handling.patch

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Håvard Skinnemoen]

On 7/10/07, Nick Piggin [EMAIL PROTECTED] wrote:

Håvard Skinnemoen wrote:
 On 7/10/07, Matt Mackall [EMAIL PROTECTED] wrote:

 The only remaining known bug is arguably a problem in nommu that SLOB
 shouldn't be papering over.


 I've got another one for you: SLOB ignores ARCH_KMALLOC_MINALIGN so
 using SLOB in combination with DMA and non-coherent architectures
 causes data corruption.

Should be fixed in mm, I believe: slob-improved-alignment-handling.patch


Indeed. Thanks, I'll give it a try later today.

Håvard
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka Enberg]

Hi Nick,

Pekka J Enberg wrote:

 That's 92 KB advantage for SLUB with debugging enabled and 240 KB when
 debugging is disabled.


On 7/10/07, Nick Piggin [EMAIL PROTECTED] wrote:

Interesting. What kernel version are you using?


Linus' git head from yesterday so the results are likely to be
sensitive to workload and mine doesn't represent real embedded use.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka Enberg wrote:

Hi Nick,

Pekka J Enberg wrote:


 That's 92 KB advantage for SLUB with debugging enabled and 240 KB when
 debugging is disabled.



On 7/10/07, Nick Piggin [EMAIL PROTECTED] wrote:


Interesting. What kernel version are you using?



Linus' git head from yesterday so the results are likely to be
sensitive to workload and mine doesn't represent real embedded use.


Hi Pekka,

There is one thing that the SLOB patches in -mm do besides result in
slightly better packing and memory efficiency (which might be unlikely
to explain the difference you are seeing), and that is that they do
away with the delayed freeing of unused SLOB pages back to the page
allocator.

In git head, these pages are freed via a timer so they can take a
while to make their way back to the buddy allocator so they don't
register as free memory as such.

Anyway, I would be very interested to see any situation where the
SLOB in -mm uses more memory than SLUB, even on test configs like
yours.

Thanks,
Nick

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 12:31:40PM +0300, Pekka Enberg wrote:
 Hi Nick,
 
 Pekka J Enberg wrote:
  That's 92 KB advantage for SLUB with debugging enabled and 240 KB when
  debugging is disabled.
 
 On 7/10/07, Nick Piggin [EMAIL PROTECTED] wrote:
 Interesting. What kernel version are you using?
 
 Linus' git head from yesterday so the results are likely to be
 sensitive to workload and mine doesn't represent real embedded use.

Using 2.6.22-rc6-mm1 with a 64MB lguest and busybox, I'm seeing the
following as the best MemFree numbers after several boots each:

SLAB: 54796
SLOB: 55044
SLUB: 53944
SLUB: 54788 (debug turned off)

These numbers bounce around a lot more from boot to boot than I
remember, so take these numbers with a grain of salt.

Disabling the debug code in the build gives this, by the way:

mm/slub.c: In function ‘init_kmem_cache_node’:
mm/slub.c:1873: error: ‘struct kmem_cache_node’ has no member named
‘full’

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Matt,

On Tue, 10 Jul 2007, Matt Mackall wrote:
 Using 2.6.22-rc6-mm1 with a 64MB lguest and busybox, I'm seeing the
 following as the best MemFree numbers after several boots each:
 
 SLAB: 54796
 SLOB: 55044
 SLUB: 53944
 SLUB: 54788 (debug turned off)
 
 These numbers bounce around a lot more from boot to boot than I
 remember, so take these numbers with a grain of salt.

To rule out userland, 2.6.22 with 32 MB defconfig UML and busybox [1] on 
i386:

SLOB: 26708
SLUB: 27212 (no debug)

Unfortunately UML is broken in 2.6.22-rc6-mm1, so I don't know if SLOB 
patches help there.

  1. http://uml.nagafix.co.uk/BusyBox-1.5.0/BusyBox-1.5.0-x86-root_fs.bz2

Pekka
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

 But last time this discussion came up, IIRC you ended up handwaving
 about all the ways in which SLOB was broken but didn't actually come
 up with any real problems. Matt seemed willing to add those counters
 or whatever it was if/when doing so solved a real problem. And remember
 that SLOB doesn't have to have feature parity with SLUB, so long as it
 implements the slab API such that the kernel *works*.

No it does not have to have feature parity. And yes we identified areas in 
which SLOB may cause problems due to not implementing things (f.e. 
suspend resume). The counters are still missing and thus core development 
cannot rely on those being there.

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

 following as the best MemFree numbers after several boots each:
 
 SLAB: 54796
 SLOB: 55044
 SLUB: 53944
 SLUB: 54788 (debug turned off)

That was without slub_debug as a parameter or with !CONFIG_SLUB_DEBUG?

Data size and code size will decrease if you compile with 
!CONFIG_SLUB_DEBUG. slub_debug on the command line governs if debug 
information is used.

 These numbers bounce around a lot more from boot to boot than I
 remember, so take these numbers with a grain of salt.
 
 Disabling the debug code in the build gives this, by the way:
 
 mm/slub.c: In function ÿÿinit_kmem_cache_nodeÿÿ:
 mm/slub.c:1873: error: ÿÿstruct kmem_cache_nodeÿÿ has no member named
 ÿÿfullÿÿ

A fix for that is in Andrew's tree.

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 03:12:38PM -0700, Christoph Lameter wrote:
 On Tue, 10 Jul 2007, Matt Mackall wrote:
 
  following as the best MemFree numbers after several boots each:
  
  SLAB: 54796
  SLOB: 55044
  SLUB: 53944
  SLUB: 54788 (debug turned off)
 
 That was without slub_debug as a parameter or with !CONFIG_SLUB_DEBUG?

Without the parameter, as the other way doesn't compile in -mm1.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

 Without the parameter, as the other way doesn't compile in -mm1.

here is the patch that went into mm after mm1 was released.

---
 mm/slub.c |4 
 1 file changed, 4 insertions(+)

Index: linux-2.6.22-rc6-mm1/mm/slub.c
===
--- linux-2.6.22-rc6-mm1.orig/mm/slub.c 2007-07-06 13:28:57.0 -0700
+++ linux-2.6.22-rc6-mm1/mm/slub.c  2007-07-06 13:29:01.0 -0700
@@ -1868,7 +1868,9 @@ static void init_kmem_cache_node(struct 
atomic_long_set(n-nr_slabs, 0);
spin_lock_init(n-list_lock);
INIT_LIST_HEAD(n-partial);
+#ifdef CONFIG_SLUB_DEBUG
INIT_LIST_HEAD(n-full);
+#endif
 }
 
 #ifdef CONFIG_NUMA
@@ -1898,8 +1900,10 @@ static struct kmem_cache_node * __init e
page-freelist = get_freepointer(kmalloc_caches, n);
page-inuse++;
kmalloc_caches-node[node] = n;
+#ifdef CONFIG_SLUB_DEBUG
init_object(kmalloc_caches, n, 1);
init_tracking(kmalloc_caches, n);
+#endif
init_kmem_cache_node(n);
atomic_long_inc(n-nr_slabs);
add_partial(n, page);
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 03:09:06PM -0700, Christoph Lameter wrote:
 On Tue, 10 Jul 2007, Nick Piggin wrote:
 
  But last time this discussion came up, IIRC you ended up handwaving
  about all the ways in which SLOB was broken but didn't actually come
  up with any real problems. Matt seemed willing to add those counters
  or whatever it was if/when doing so solved a real problem. And remember
  that SLOB doesn't have to have feature parity with SLUB, so long as it
  implements the slab API such that the kernel *works*.
 
 No it does not have to have feature parity. And yes we identified areas in 
 which SLOB may cause problems due to not implementing things (f.e. 
 suspend resume).

Please remind me what these things are. Suspend and hibernate work
fine here.

 The counters are still missing and thus core development 
 cannot rely on those being there.

None of the VM makes real use of the SLAB counters. Nor will they ever
make real use of them because the counters are not usefully defined.
In other words, reclaimable is a lie. If SLAB claims there's 1M of
reclaimable memory, the actual amount may more often than not be zero
because we already tried to shrink the SLAB.

As I've said before, if you come up with a real use for these numbers,
I will sacrifice the appropriate number of puppies to support it in
SLOB. But I'd rather not crush any puppies here prematurely.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Matt Mackall wrote:

 You're delusional.


Git log says otherwise:

 git log --pretty=short mm/slob.c

Author: Christoph Lameter [EMAIL PROTECTED]
Remove SLAB_CTOR_CONSTRUCTOR
Author: Christoph Lameter [EMAIL PROTECTED]
Slab allocators: Drop support for destructors
Author: Nick Piggin [EMAIL PROTECTED]
slob: implement RCU freeing
Author: Akinobu Mita [EMAIL PROTECTED]
slob: fix page order calculation on not 4KB page
Author: Christoph Lameter [EMAIL PROTECTED]
slab allocators: Remove obsolete SLAB_MUST_HWCACHE_ALIGN
Author: Akinobu Mita [EMAIL PROTECTED]
slob: handle SLAB_PANIC flag
Author: Pekka Enberg [EMAIL PROTECTED]
slab: introduce krealloc
Author: Dimitri Gorokhovik [EMAIL PROTECTED]
[PATCH] MM: SLOB is broken by recent cleanup of slab.h
Author: Christoph Lameter [EMAIL PROTECTED]
[PATCH] More slab.h cleanups
Author: Christoph Lameter [EMAIL PROTECTED]
[PATCH] Cleanup slab headers / API to allow easy addition of new slab 
allocators
Author: Alexey Dobriyan [EMAIL PROTECTED]
[PATCH] Make kmem_cache_destroy() return void
Author: Christoph Lameter [EMAIL PROTECTED]
[PATCH] ZVC: Support NR_SLAB_RECLAIMABLE / NR_SLAB_UNRECLAIMABLE
Author: Christoph Lameter [EMAIL PROTECTED]
[PATCH] Extract the allocpercpu functions from the slab allocator
Author: Jörn Engel [EMAIL PROTECTED]
Remove obsolete #include linux/config.h
Author: John Hawkes [EMAIL PROTECTED]
[PATCH] mm/slob.c: for_each_possible_cpu(), not NR_CPUS
Author: Pekka Enberg [EMAIL PROTECTED]
[PATCH] slab: introduce kmem_cache_zalloc allocator
Author: Ingo Molnar [EMAIL PROTECTED]
[PATCH] SLOB=y  SMP=y fix
Author: Matt Mackall [EMAIL PROTECTED]
[PATCH] slob: introduce the SLOB allocator

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Tue, Jul 10, 2007 at 06:37:42PM -0700, Christoph Lameter wrote:
 On Tue, 10 Jul 2007, Matt Mackall wrote:
 
  You're delusional.
 
 Git log says otherwise:
 
  git log --pretty=short mm/slob.c

A dozen trivial cleanups do not make you maintainer. Otherwise we'd
all be sending our patches to Adrian rather than Linus.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Mon, Jul 09, 2007 at 06:51:51PM -0700, Christoph Lameter wrote:
> On Tue, 10 Jul 2007, Nick Piggin wrote:
> 
> > > O(n) memory savings? What is that?
> > 
> > Allocate n things and your memory waste is proportional to n (well that's
> > O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
> > to SLUB are proportional to n).
> 
> n is the size of the object?
> 
> Its linear correlated to the object size. It does not grow 
> exponentially as object size grows. Waste in the kmalloc array in the 
> worst case is < objsize.

N is the number of objects.

So, for large N, the overhead of SLUB of allocations of size m, where
m % alignment == 0 (for simplicity):

 kmem_cache_alloc case:

  (PAGE_SIZE % m) / m

 kmalloc case:

  nextpowerof2(m) - m

And for SLOB (with Nick's improvements):

 kmem_cache_alloc case:

  0 (the remainder of the page is usable for other allocs)

 kmalloc case:

  2 bytes (or whatever the minimal arch alignment is)

SLUB wins by two bytes on kmallocs that happen to be a power of two,
ties on kmem_cache_allocs that happen to evenly divide PAGE_SIZE and
kmallocs 2 bytes smaller than a power of 2, and loses everywhere else.

It also loses whenever a particular type/size has both short-lived and
long-lived objects because of pinning. This effect is worse on small
machines as it's easier to pin all of memory.

The downsides of current SLOB are:

 - single lock
 - potentially long walks to find free blocks

The average case walk has been shortened quite a bit by Nick's
patches. The combinatorics here are enough to make Knuth hand-wave,
but we can in theory get close to O(1) performance when we don't have
much memory pressure. Needs some serious benchmarking. When we DO have
memory pressure, both SLOB and SLUB will degrade to forcing the kernel
to walk a bunch of pages.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

> It is reasonable to expect some help from maintainers, but I notice you
> didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
> if you tried working a bit closer with him you could get better results?

The maintainers last patch to SLOB was the initial submission of the 
allocator. Then he acked subsequent patches. Most of the modifications to 
SLOB are my work. Attempts to talk to the maintainer result in inventive 
explanations why SLOB does not have to conform to kernel standards. There 
is no reasonable expectation that this will change.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Tue, 10 Jul 2007, Nick Piggin wrote:



I don't see any problems with maintaining SLOB. It is simple enough
that I was able to write a userspace test harness for it and hack
away at it after reading the code the first time for half an hour or
so. It is nothing even slightly comparable to the problems of SLAB,
for example. And you don't have to maintain it at all anyway!



I have to maintain it because I have to keep the slab APIs consistent 
(recently I added GFP_ZERO support and had to provide shims for slab 
defreag). It is not in a good state as described in the patch and has a 
history of not being maintained properly. Everyone that modifies the 
behavior of the slab allocator has to do something to avoid breaking SLOB. 
Its certainly fun to hack on but is that a criterion for keeping it in the 
tree?


Pretty standard fare that when you add something or change APIs, most
of the burden is on you to not break the kernel. I'd love nothing better
than to remove all but about 3 filesystems :)

It is reasonable to expect some help from maintainers, but I notice you
didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
if you tried working a bit closer with him you could get better results?



I like removing code as much as the next person, but I don't
understand why you are so intent on removing SLOB and willing to
dismiss its advantages so quickly.



Quickly? We have considered this for months now.


Quickly -- as in you quickly sweep the savings of 100s of K under the
rug and just declare that it is insignificant :)

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Tue, 10 Jul 2007, Nick Piggin wrote:



O(n) memory savings? What is that?


Allocate n things and your memory waste is proportional to n (well that's
O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
to SLUB are proportional to n).



n is the size of the object?


n things -- n number of things (n calls to kmem_cache_alloc()).

Just a fancy way of saying roughly that memory waste will increase as
the size of the system increases. But that aspect of it I think is
not really a problem for non-tiny systems anyway because the waste
tends not to be too bad (and possibly the number of active allocations
does not increase O(n) with the size of RAM either).

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

> I don't see any problems with maintaining SLOB. It is simple enough
> that I was able to write a userspace test harness for it and hack
> away at it after reading the code the first time for half an hour or
> so. It is nothing even slightly comparable to the problems of SLAB,
> for example. And you don't have to maintain it at all anyway!

I have to maintain it because I have to keep the slab APIs consistent 
(recently I added GFP_ZERO support and had to provide shims for slab 
defreag). It is not in a good state as described in the patch and has a 
history of not being maintained properly. Everyone that modifies the 
behavior of the slab allocator has to do something to avoid breaking SLOB. 
Its certainly fun to hack on but is that a criterion for keeping it in the 
tree?

> I like removing code as much as the next person, but I don't
> understand why you are so intent on removing SLOB and willing to
> dismiss its advantages so quickly.

Quickly? We have considered this for months now.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

> > O(n) memory savings? What is that?
> 
> Allocate n things and your memory waste is proportional to n (well that's
> O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
> to SLUB are proportional to n).

n is the size of the object?

Its linear correlated to the object size. It does not grow 
exponentially as object size grows. Waste in the kmalloc array in the 
worst case is < objsize.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Mon, 9 Jul 2007, Andrew Morton wrote:


I think the advantage that SLOB generates here is pretty minimal and is 
easily offset by the problems of maintaining SLOB.


I don't get it. Have you got agreement from the small memory people
that the advantages of SLOB are pretty minimal, or did you just
decide that? If the latter, did you completely miss reading my email?
What happens to the people who jump through hoops to save 1 or 2 K?

I don't see any problems with maintaining SLOB. It is simple enough
that I was able to write a userspace test harness for it and hack
away at it after reading the code the first time for half an hour or
so. It is nothing even slightly comparable to the problems of SLAB,
for example. And you don't have to maintain it at all anyway!

I like removing code as much as the next person, but I don't
understand why you are so intent on removing SLOB and willing to
dismiss its advantages so quickly.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Mon, 9 Jul 2007, Nick Piggin wrote:



A reason for retaining slob would be that it has some O(n) memory saving
due to better packing, etc.  Indeed that was the reason for merging it in
the first place.  If slob no longer retains that advantage (wrt slub) then
we no longer need it.


SLOB contains several significant O(1) and also O(n) memory savings that
are so far impossible-by-design for SLUB. They are: slab external
fragmentation is significantly reduced; kmalloc internal fragmentation is
significantly reduced; order of magnitude smaller kmem_cache data type;
order of magnitude less code...



Well that is only true for kmalloc objects < PAGE_SIZE and to some extend 
offset by the need to keep per object data in SLUB. But yes the power of 
two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
operations of kmalloc slabs which in turns causes memory wastage because 
of rounding of the alloc to the next power of two. SLUB has less wastage
there than SLAB since it can fit power of two object tightly into a slab 
instead of having to place additional control information there like SLAB.


OK but we're talking about SLOB. And the number that matters is the amount
of memory used, which is higher with SLUB than with SLOB in our tests.



O(n) memory savings? What is that?


Allocate n things and your memory waste is proportional to n (well that's
O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
to SLUB are proportional to n).

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Mon, Jul 09, 2007 at 09:51:16AM -0700, Andrew Morton wrote:
> On Mon, 9 Jul 2007 09:06:46 -0700 (PDT) Christoph Lameter <[EMAIL PROTECTED]> 
> wrote:
> 
> > But yes the power of 
> > two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
> > operations of kmalloc slabs which in turns causes memory wastage because 
> > of rounding of the alloc to the next power of two.
> 
> I've frequently wondered why we don't just create more caches for kmalloc:
> make it denser than each-power-of-2-plus-a-few-others-in-between.
> 
> I assume the tradeoff here is better packing versus having a ridiculous
> number of caches.  Is there any other cost?

It magnifies the fragmentation problem.

SLAB (and SLUB) makes the optimistic assumption that objects of the
same type/size have similar lifetimes. But for some objects, it's not
uncommon to do many temporary allocations but have some objects with
indefinite lifespans. dcache is a very frequently encountered example,
but there's no reason we couldn't see it with sockets and many other
object types. 

Every new arena introduces further opportunity for this sort of
fragmentation. If we had, say, separate pools for 48 byte and 64 byte
objects, an unfortunate usage pattern for 48-byte kmallocs could DoS
requests for 64 byte objects that would work just fine if they both
came out of the same pool. If we have 10 pools with long-lived
objects, we're much worse off than if we had 1 or 2.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Sun, Jul 08, 2007 at 11:02:24AM -0700, Andrew Morton wrote:
> Guys, look at this the other way.  Suppose we only had slub, and someone
> came along and said "here's a whole new allocator which saves 4.5k of
> text", would we merge it on that basis?  Hell no, it's not worth it.  What
> we might do is to get motivated to see if we can make slub less porky under
> appropriate config settings.

Well I think we would obviously throw out SLAB and SLUB if they
weren't somewhat faster than SLOB. They're much more problematic and
one of the big features that Christoph's pushing is a fix for a
problem that SLOB simply doesn't have: huge numbers of SLAB/SLUB pages
being held down by small numbers of objects. 

> Let's not get sentimental about these things: in general, if there's any
> reasonable way in which we can rid ourselves of any code at all, we should
> do so, no?

I keep suggesting a Voyager Replacement Fund, but James isn't
interested.

But seriously, I don't think it should be at all surprising that the
allocator that's most appropriate for machines with < 32MB of RAM is
different than the one for machines with > 1TB of RAM.

The maintenance overhead of SLOB is fairly minimal. The biggest
outstanding SLOB problem is nommu's rather broken memory size
reporting.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

First, WTF wasn't I cc:ed on this? Are you actually trying to me make
me fuming mad?

On Sat, Jul 07, 2007 at 08:50:01PM -0700, Christoph Lameter wrote:
> Maintenance of slab allocators becomes a problem as new features for
> allocators are developed. The SLOB allocator in particular has been lagging
> behind in many ways in the past:
> 
> - Had no support for SLAB_DESTROY_BY_RCU for years (but no one noticed)

We've been over this 50 times. The target users were never affected.
And it's fixed. So why the HELL are you mentioning this again? 
 
> - Still has no support for slab reclaim counters. This may currently not
>   be necessary if one would restrict the supported configurations for
>   functionality relying on these. But even that has not been done.

We've been over this 50 times. Last time around, I inspected all the
code paths and demonstrated that despite your handwaving, IT DIDN'T
MATTER.

> The only current advantage over SLUB in terms of memory savings is through
> SLOBs kmalloc layout that is not power of two based like SLAB and SLUB which
> allows to eliminate some memory waste.
> 
> Through that SLOB has still a slight memory advantage over SLUB of ~350k in
> for a standard server configuration. It is likely that the savings are is
> smaller for real embedded configurations that have less functionality.

Sometimes I do not think there is a cluebat large enough for you. 350K
is FREAKING HUGE on a cell phone. That's most of a kernel!

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Andrew Morton]

On Mon, 9 Jul 2007 10:26:08 -0700 (PDT)
Christoph Lameter <[EMAIL PROTECTED]> wrote:

> > I assume the tradeoff here is better packing versus having a ridiculous
> > number of caches.  Is there any other cost?
> > Because even having 1024 caches wouldn't consume a terrible amount of
> > memory and I bet it would result in aggregate savings.
> 
> I have tried any number of approaches without too much success. Even one 
> slab cache for every 8 bytes. This creates additional admin overhead 
> through more control structure (that is pretty minimal but nevertheless 
> exists)
> 
> The main issue is that kmallocs of different size must use different 
> pages. If one allocates one 64 byte item and one 256 byte item and both 64 
> byte and 256 byte are empty then SLAB/SLUB will have to allocate 2 pages. 
> SLUB can fit them into one. This is basically only relevant early after 
> boot. The advantage goes away as the system starts to work and as more 
> objects are allocated in the slabs but the power-of-two slab will always
> have to extend its size in page size chunks which leads to some overhead 
> that SLOB can avoid by placing entities of multiple size in one slab. 
> The tradeoff in SLOB is that is cannot be an O(1) allocator because it 
> has to manage these variable sized objects by traversing the lists.
> 
> I think the advantage that SLOB generates here is pretty minimal and is 
> easily offset by the problems of maintaining SLOB.

Sure.  But I wasn't proposing this as a way to make slub cover slob's advantage.
I was wondering what effect it would have on a more typical medium to large 
sized
system.

Not much, really: if any particular subsystem is using a "lot" of slab memory 
then
it should create its own cache rather than using kmalloc anyway, so forget it ;)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Mon, 9 Jul 2007, Andrew Morton wrote:

> On Mon, 9 Jul 2007 09:06:46 -0700 (PDT) Christoph Lameter <[EMAIL PROTECTED]> 
> wrote:
> 
> > But yes the power of 
> > two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
> > operations of kmalloc slabs which in turns causes memory wastage because 
> > of rounding of the alloc to the next power of two.
> 
> I've frequently wondered why we don't just create more caches for kmalloc:
> make it denser than each-power-of-2-plus-a-few-others-in-between.
> 
> I assume the tradeoff here is better packing versus having a ridiculous
> number of caches.  Is there any other cost?
> Because even having 1024 caches wouldn't consume a terrible amount of
> memory and I bet it would result in aggregate savings.

I have tried any number of approaches without too much success. Even one 
slab cache for every 8 bytes. This creates additional admin overhead 
through more control structure (that is pretty minimal but nevertheless 
exists)

The main issue is that kmallocs of different size must use different 
pages. If one allocates one 64 byte item and one 256 byte item and both 64 
byte and 256 byte are empty then SLAB/SLUB will have to allocate 2 pages. 
SLUB can fit them into one. This is basically only relevant early after 
boot. The advantage goes away as the system starts to work and as more 
objects are allocated in the slabs but the power-of-two slab will always
have to extend its size in page size chunks which leads to some overhead 
that SLOB can avoid by placing entities of multiple size in one slab. 
The tradeoff in SLOB is that is cannot be an O(1) allocator because it 
has to manage these variable sized objects by traversing the lists.

I think the advantage that SLOB generates here is pretty minimal and is 
easily offset by the problems of maintaining SLOB.
 
> Of course, a scheme which creates kmalloc caches on-demand would be better,
> but that would kill our compile-time cache selection, I suspect.

SLUB creates kmalloc caches on-demand for DMA caches already. But then we 
are not allowing compile time cache selection for DMA caches.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Andrew Morton]

On Mon, 9 Jul 2007 09:06:46 -0700 (PDT) Christoph Lameter <[EMAIL PROTECTED]> 
wrote:

> But yes the power of 
> two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
> operations of kmalloc slabs which in turns causes memory wastage because 
> of rounding of the alloc to the next power of two.

I've frequently wondered why we don't just create more caches for kmalloc:
make it denser than each-power-of-2-plus-a-few-others-in-between.

I assume the tradeoff here is better packing versus having a ridiculous
number of caches.  Is there any other cost?

Because even having 1024 caches wouldn't consume a terrible amount of
memory and I bet it would result in aggregate savings.

Of course, a scheme which creates kmalloc caches on-demand would be better,
but that would kill our compile-time cache selection, I suspect.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Mon, 9 Jul 2007, Pekka Enberg wrote:

> I assume with "slab external fragmentation" you mean allocating a
> whole page for a slab when there are not enough objects to fill the
> whole thing thus wasting memory? We could try to combat that by
> packing multiple variable-sized slabs within a single page. Also,
> adding some non-power-of-two kmalloc caches might help with internal
> fragmentation.

Ther are already non-power-of-two kmalloc caches for 96 and 192 bytes 
sizes.
> 
> In any case, SLUB needs some serious tuning for smaller machines
> before we can get rid of SLOB.

Switch off CONFIG_SLUB_DEBUG to get memory savings.
 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Mon, 9 Jul 2007, Nick Piggin wrote:

> > A reason for retaining slob would be that it has some O(n) memory saving
> > due to better packing, etc.  Indeed that was the reason for merging it in
> > the first place.  If slob no longer retains that advantage (wrt slub) then
> > we no longer need it.
> 
> SLOB contains several significant O(1) and also O(n) memory savings that
> are so far impossible-by-design for SLUB. They are: slab external
> fragmentation is significantly reduced; kmalloc internal fragmentation is
> significantly reduced; order of magnitude smaller kmem_cache data type;
> order of magnitude less code...

Well that is only true for kmalloc objects < PAGE_SIZE and to some extend 
offset by the need to keep per object data in SLUB. But yes the power of 
two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
operations of kmalloc slabs which in turns causes memory wastage because 
of rounding of the alloc to the next power of two. SLUB has less wastage
there than SLAB since it can fit power of two object tightly into a slab 
instead of having to place additional control information there like SLAB.

O(n) memory savings? What is that?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Sun, 8 Jul 2007, Ingo Molnar wrote:

> actually, one real advantage of the SLOB is that it is a minimal, really 
> simple allocator. Its text and data size is so small as well.
> 
> here's the size comparison:
> 
>textdata bss dec hex filename
>   10788 837  16   116412d79 mm/slab.o
>62054207 124   105362928 mm/slub.o
>1640  44   41688 698 mm/slob.o
> 
> slab/slub have roughly the same footprint, but slob is 10% of that size. 
> Would be a waste to throw this away.

The last of my tests showed that SLOB is at about 50% of the size of 
SLUB. You need to compile SLUB in embedded mode with !CONFIG_SLUB_DEBUG to 
get a reduce code size.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Nick,

Pekka Enberg wrote:
> > adding some non-power-of-two kmalloc caches might help with internal
> > fragmentation.

On Mon, 9 Jul 2007, Nick Piggin wrote:
> That too, although of course it will work against the external
> fragmentation problem. This is more of an O(n) problem and may not
> be responsible for as much waste as the first issue on small memory
> machines (I haven't done detailed profiling so I don't know).

I would have thought so too, but my crude hack to approximate internal 
fragmentation says otherwise. On the same Debian on UML virtual machine I 
see up to 190 KB of wasted space due to internal fragmentation (average 
allocation size being considerably smaller than object size for cache) 
with the biggest offender being kmalloc-512.

But, what we really need is some real workloads on small machines using 
something resembling my scripts to figure out the memory profile for SLUB.

Pekka

#!/bin/bash

total_wasted=0

for i in $(find /sys/slab -type d -mindepth 1 -maxdepth 1 | sort)
do
  slabs=$(cat $i/slabs)
  objs_per_slab=$(cat $i/objs_per_slab)
  let "max_objs=$objs_per_slab*$slabs"

  object_size=$(cat $i/object_size)
  average_alloc_size=$(cat $i/average_alloc_size)

  if [ "0" -ne "$average_alloc_size" ]; then
let "wasted=($object_size-$average_alloc_size)*max_objs"
echo "$i: object_size=$object_size, 
average_alloc_size=$average_alloc_size, $wasted bytes wasted"
let "total_wasted=$total_wasted+$wasted"
  fi
done
echo "Total internal fragmentation: $total_wasted bytes"

---
 include/linux/slub_def.h |1 +
 mm/slub.c|   20 ++--
 2 files changed, 19 insertions(+), 2 deletions(-)

Index: 2.6/include/linux/slub_def.h
===
--- 2.6.orig/include/linux/slub_def.h   2007-07-09 16:09:24.0 +0300
+++ 2.6/include/linux/slub_def.h2007-07-09 16:18:42.0 +0300
@@ -29,6 +29,7 @@ struct kmem_cache {
int objsize;/* The size of an object without meta data */
int offset; /* Free pointer offset. */
int order;
+   int average_alloc_size;
 
/*
 * Avoid an extra cache line for UP, SMP and for the node local to
Index: 2.6/mm/slub.c
===
--- 2.6.orig/mm/slub.c  2007-07-09 16:09:24.0 +0300
+++ 2.6/mm/slub.c   2007-07-09 16:35:53.0 +0300
@@ -2238,12 +2238,22 @@ BUG_ON(index < 0);
return _caches[index];
 }
 
+static void update_avg(struct kmem_cache *s, size_t size)
+{
+   if (s->average_alloc_size)
+   s->average_alloc_size = (s->average_alloc_size + size) / 2;
+   else
+   s->average_alloc_size = size;
+}
+
 void *__kmalloc(size_t size, gfp_t flags)
 {
struct kmem_cache *s = get_slab(size, flags);
 
-   if (s)
+   if (s) {
+   update_avg(s, size);
return slab_alloc(s, flags, -1, __builtin_return_address(0));
+   }
return ZERO_SIZE_PTR;
 }
 EXPORT_SYMBOL(__kmalloc);
@@ -2253,8 +2263,11 @@ void *__kmalloc_node(size_t size, gfp_t 
 {
struct kmem_cache *s = get_slab(size, flags);
 
-   if (s)
+   if (s) {
+   update_avg(s, size);
return slab_alloc(s, flags, node, __builtin_return_address(0));
+   }
+
return ZERO_SIZE_PTR;
 }
 EXPORT_SYMBOL(__kmalloc_node);
@@ -2677,6 +2690,8 @@ void *__kmalloc_track_caller(size_t size
if (!s)
return ZERO_SIZE_PTR;
 
+   update_avg(s, size);
+
return slab_alloc(s, gfpflags, -1, caller);
 }
 
@@ -2688,6 +2703,8 @@ void *__kmalloc_node_track_caller(size_t
if (!s)
return ZERO_SIZE_PTR;
 
+   update_avg(s, size);
+
return slab_alloc(s, gfpflags, node, caller);
 }
 
@@ -3268,6 +3285,12 @@ static ssize_t objects_show(struct kmem_
 }
 SLAB_ATTR_RO(objects);
 
+static ssize_t average_alloc_size_show(struct kmem_cache *s, char *buf)
+{
+   return sprintf(buf, "%d\n", s->average_alloc_size);
+}
+SLAB_ATTR_RO(average_alloc_size);
+
 static ssize_t sanity_checks_show(struct kmem_cache *s, char *buf)
 {
return sprintf(buf, "%d\n", !!(s->flags & SLAB_DEBUG_FREE));
@@ -3466,6 +3489,7 @@ static struct attribute * slab_attrs[] =
_attr.attr,
_attr.attr,
_attr.attr,
+   _alloc_size_attr.attr,
_attr.attr,
_slabs_attr.attr,
_attr.attr,
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka Enberg]

Hi Nick,

Pekka Enberg wrote:

> I assume with "slab external fragmentation" you mean allocating a
> whole page for a slab when there are not enough objects to fill the
> whole thing thus wasting memory?


On 7/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:

Yep. Without really analysing it, I guess SLOB's savings here are
O(1) over SLUB and will relatively diminish as the machine size gets
larger, however with the number of slabs even a small kernel creates,
this is likely to be significant on small memory systems.


Running the included script on my little Debian on UML with 32 MB of
RAM shows anywhere from 20 KB up to 100 KB of wasted space on light
load. What's interesting is that the wasted amount seems to stabilize
around 70 KB and never goes below that.

  Pekka

#!/bin/bash

total_wasted=0

for i in $(find /sys/slab -type d -mindepth 1 -maxdepth 1 | sort)
do
 nr_objs=$(cat $i/objects)
 slabs=$(cat $i/slabs)
 objs_per_slab=$(cat $i/objs_per_slab)
 let "max_objs=$objs_per_slab*$slabs"

 object_size=$(cat $i/object_size)
 let "wasted=($max_objs-$nr_objs)*$object_size"

 if [ "$wasted" -ne "0" ]; then
   echo "$i: max_objs=$max_objs, nr_objs=$nr_objs, $wasted bytes wasted"
 fi
 let "total_wasted=$total_wasted+$wasted"
done

echo "Total wasted: $total_wasted"
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matthieu CASTET]

Ingo Molnar  elte.hu> writes:


> A year ago the -rt kernel defaulted to the SLOB for a few releases, and 
> barring some initial scalability issues (which were solved in -rt) it 
> worked pretty well on generic PCs, so i dont buy the 'it doesnt work' 
> argument either.
> 
Last time I tried it, I cannot mount (or mount very slowly) jffs2 flash with 
slob.
Changing slob to slab fix the issue.


Matthieu

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka Enberg wrote:

Hi Nick,

On 7/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:


SLOB contains several significant O(1) and also O(n) memory savings that
are so far impossible-by-design for SLUB. They are: slab external
fragmentation is significantly reduced; kmalloc internal fragmentation is
significantly reduced; order of magnitude smaller kmem_cache data type;
order of magnitude less code...



I assume with "slab external fragmentation" you mean allocating a
whole page for a slab when there are not enough objects to fill the
whole thing thus wasting memory?


Yep. Without really analysing it, I guess SLOB's savings here are
O(1) over SLUB and will relatively diminish as the machine size gets
larger, however with the number of slabs even a small kernel creates,
this is likely to be significant on small memory systems.



We could try to combat that by
packing multiple variable-sized slabs within a single page. Also,


Yeah, that could help.



adding some non-power-of-two kmalloc caches might help with internal
fragmentation.


That too, although of course it will work against the external
fragmentation problem. This is more of an O(n) problem and may not
be responsible for as much waste as the first issue on small memory
machines (I haven't done detailed profiling so I don't know).



In any case, SLUB needs some serious tuning for smaller machines
before we can get rid of SLOB.


I would always be happy to see the default allocator become more
space efficient... I think it would be most productive to get
detailed profiles of exactly where the memory is being wasted on
small mem= boots. Although I don't think matching SLOB would be
an easy task at all.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka Enberg]

Hi Nick,

On 7/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:

SLOB contains several significant O(1) and also O(n) memory savings that
are so far impossible-by-design for SLUB. They are: slab external
fragmentation is significantly reduced; kmalloc internal fragmentation is
significantly reduced; order of magnitude smaller kmem_cache data type;
order of magnitude less code...


I assume with "slab external fragmentation" you mean allocating a
whole page for a slab when there are not enough objects to fill the
whole thing thus wasting memory? We could try to combat that by
packing multiple variable-sized slabs within a single page. Also,
adding some non-power-of-two kmalloc caches might help with internal
fragmentation.

In any case, SLUB needs some serious tuning for smaller machines
before we can get rid of SLOB.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka Enberg]

Hi Nick,

On 7/9/07, Nick Piggin [EMAIL PROTECTED] wrote:

SLOB contains several significant O(1) and also O(n) memory savings that
are so far impossible-by-design for SLUB. They are: slab external
fragmentation is significantly reduced; kmalloc internal fragmentation is
significantly reduced; order of magnitude smaller kmem_cache data type;
order of magnitude less code...


I assume with slab external fragmentation you mean allocating a
whole page for a slab when there are not enough objects to fill the
whole thing thus wasting memory? We could try to combat that by
packing multiple variable-sized slabs within a single page. Also,
adding some non-power-of-two kmalloc caches might help with internal
fragmentation.

In any case, SLUB needs some serious tuning for smaller machines
before we can get rid of SLOB.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Pekka Enberg wrote:

Hi Nick,

On 7/9/07, Nick Piggin [EMAIL PROTECTED] wrote:


SLOB contains several significant O(1) and also O(n) memory savings that
are so far impossible-by-design for SLUB. They are: slab external
fragmentation is significantly reduced; kmalloc internal fragmentation is
significantly reduced; order of magnitude smaller kmem_cache data type;
order of magnitude less code...



I assume with slab external fragmentation you mean allocating a
whole page for a slab when there are not enough objects to fill the
whole thing thus wasting memory?


Yep. Without really analysing it, I guess SLOB's savings here are
O(1) over SLUB and will relatively diminish as the machine size gets
larger, however with the number of slabs even a small kernel creates,
this is likely to be significant on small memory systems.



We could try to combat that by
packing multiple variable-sized slabs within a single page. Also,


Yeah, that could help.



adding some non-power-of-two kmalloc caches might help with internal
fragmentation.


That too, although of course it will work against the external
fragmentation problem. This is more of an O(n) problem and may not
be responsible for as much waste as the first issue on small memory
machines (I haven't done detailed profiling so I don't know).



In any case, SLUB needs some serious tuning for smaller machines
before we can get rid of SLOB.


I would always be happy to see the default allocator become more
space efficient... I think it would be most productive to get
detailed profiles of exactly where the memory is being wasted on
small mem= boots. Although I don't think matching SLOB would be
an easy task at all.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matthieu CASTET]

Ingo Molnar mingo at elte.hu writes:


 A year ago the -rt kernel defaulted to the SLOB for a few releases, and 
 barring some initial scalability issues (which were solved in -rt) it 
 worked pretty well on generic PCs, so i dont buy the 'it doesnt work' 
 argument either.
 
Last time I tried it, I cannot mount (or mount very slowly) jffs2 flash with 
slob.
Changing slob to slab fix the issue.


Matthieu

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka Enberg]

Hi Nick,

Pekka Enberg wrote:

 I assume with slab external fragmentation you mean allocating a
 whole page for a slab when there are not enough objects to fill the
 whole thing thus wasting memory?


On 7/9/07, Nick Piggin [EMAIL PROTECTED] wrote:

Yep. Without really analysing it, I guess SLOB's savings here are
O(1) over SLUB and will relatively diminish as the machine size gets
larger, however with the number of slabs even a small kernel creates,
this is likely to be significant on small memory systems.


Running the included script on my little Debian on UML with 32 MB of
RAM shows anywhere from 20 KB up to 100 KB of wasted space on light
load. What's interesting is that the wasted amount seems to stabilize
around 70 KB and never goes below that.

  Pekka

#!/bin/bash

total_wasted=0

for i in $(find /sys/slab -type d -mindepth 1 -maxdepth 1 | sort)
do
 nr_objs=$(cat $i/objects)
 slabs=$(cat $i/slabs)
 objs_per_slab=$(cat $i/objs_per_slab)
 let max_objs=$objs_per_slab*$slabs

 object_size=$(cat $i/object_size)
 let wasted=($max_objs-$nr_objs)*$object_size

 if [ $wasted -ne 0 ]; then
   echo $i: max_objs=$max_objs, nr_objs=$nr_objs, $wasted bytes wasted
 fi
 let total_wasted=$total_wasted+$wasted
done

echo Total wasted: $total_wasted
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Pekka J Enberg]

Hi Nick,

Pekka Enberg wrote:
  adding some non-power-of-two kmalloc caches might help with internal
  fragmentation.

On Mon, 9 Jul 2007, Nick Piggin wrote:
 That too, although of course it will work against the external
 fragmentation problem. This is more of an O(n) problem and may not
 be responsible for as much waste as the first issue on small memory
 machines (I haven't done detailed profiling so I don't know).

I would have thought so too, but my crude hack to approximate internal 
fragmentation says otherwise. On the same Debian on UML virtual machine I 
see up to 190 KB of wasted space due to internal fragmentation (average 
allocation size being considerably smaller than object size for cache) 
with the biggest offender being kmalloc-512.

But, what we really need is some real workloads on small machines using 
something resembling my scripts to figure out the memory profile for SLUB.

Pekka

#!/bin/bash

total_wasted=0

for i in $(find /sys/slab -type d -mindepth 1 -maxdepth 1 | sort)
do
  slabs=$(cat $i/slabs)
  objs_per_slab=$(cat $i/objs_per_slab)
  let max_objs=$objs_per_slab*$slabs

  object_size=$(cat $i/object_size)
  average_alloc_size=$(cat $i/average_alloc_size)

  if [ 0 -ne $average_alloc_size ]; then
let wasted=($object_size-$average_alloc_size)*max_objs
echo $i: object_size=$object_size, 
average_alloc_size=$average_alloc_size, $wasted bytes wasted
let total_wasted=$total_wasted+$wasted
  fi
done
echo Total internal fragmentation: $total_wasted bytes

---
 include/linux/slub_def.h |1 +
 mm/slub.c|   20 ++--
 2 files changed, 19 insertions(+), 2 deletions(-)

Index: 2.6/include/linux/slub_def.h
===
--- 2.6.orig/include/linux/slub_def.h   2007-07-09 16:09:24.0 +0300
+++ 2.6/include/linux/slub_def.h2007-07-09 16:18:42.0 +0300
@@ -29,6 +29,7 @@ struct kmem_cache {
int objsize;/* The size of an object without meta data */
int offset; /* Free pointer offset. */
int order;
+   int average_alloc_size;
 
/*
 * Avoid an extra cache line for UP, SMP and for the node local to
Index: 2.6/mm/slub.c
===
--- 2.6.orig/mm/slub.c  2007-07-09 16:09:24.0 +0300
+++ 2.6/mm/slub.c   2007-07-09 16:35:53.0 +0300
@@ -2238,12 +2238,22 @@ BUG_ON(index  0);
return kmalloc_caches[index];
 }
 
+static void update_avg(struct kmem_cache *s, size_t size)
+{
+   if (s-average_alloc_size)
+   s-average_alloc_size = (s-average_alloc_size + size) / 2;
+   else
+   s-average_alloc_size = size;
+}
+
 void *__kmalloc(size_t size, gfp_t flags)
 {
struct kmem_cache *s = get_slab(size, flags);
 
-   if (s)
+   if (s) {
+   update_avg(s, size);
return slab_alloc(s, flags, -1, __builtin_return_address(0));
+   }
return ZERO_SIZE_PTR;
 }
 EXPORT_SYMBOL(__kmalloc);
@@ -2253,8 +2263,11 @@ void *__kmalloc_node(size_t size, gfp_t 
 {
struct kmem_cache *s = get_slab(size, flags);
 
-   if (s)
+   if (s) {
+   update_avg(s, size);
return slab_alloc(s, flags, node, __builtin_return_address(0));
+   }
+
return ZERO_SIZE_PTR;
 }
 EXPORT_SYMBOL(__kmalloc_node);
@@ -2677,6 +2690,8 @@ void *__kmalloc_track_caller(size_t size
if (!s)
return ZERO_SIZE_PTR;
 
+   update_avg(s, size);
+
return slab_alloc(s, gfpflags, -1, caller);
 }
 
@@ -2688,6 +2703,8 @@ void *__kmalloc_node_track_caller(size_t
if (!s)
return ZERO_SIZE_PTR;
 
+   update_avg(s, size);
+
return slab_alloc(s, gfpflags, node, caller);
 }
 
@@ -3268,6 +3285,12 @@ static ssize_t objects_show(struct kmem_
 }
 SLAB_ATTR_RO(objects);
 
+static ssize_t average_alloc_size_show(struct kmem_cache *s, char *buf)
+{
+   return sprintf(buf, %d\n, s-average_alloc_size);
+}
+SLAB_ATTR_RO(average_alloc_size);
+
 static ssize_t sanity_checks_show(struct kmem_cache *s, char *buf)
 {
return sprintf(buf, %d\n, !!(s-flags  SLAB_DEBUG_FREE));
@@ -3466,6 +3489,7 @@ static struct attribute * slab_attrs[] =
order_attr.attr,
objects_attr.attr,
slabs_attr.attr,
+   average_alloc_size_attr.attr,
partial_attr.attr,
cpu_slabs_attr.attr,
ctor_attr.attr,
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Sun, 8 Jul 2007, Ingo Molnar wrote:

 actually, one real advantage of the SLOB is that it is a minimal, really 
 simple allocator. Its text and data size is so small as well.
 
 here's the size comparison:
 
textdata bss dec hex filename
   10788 837  16   116412d79 mm/slab.o
62054207 124   105362928 mm/slub.o
1640  44   41688 698 mm/slob.o
 
 slab/slub have roughly the same footprint, but slob is 10% of that size. 
 Would be a waste to throw this away.

The last of my tests showed that SLOB is at about 50% of the size of 
SLUB. You need to compile SLUB in embedded mode with !CONFIG_SLUB_DEBUG to 
get a reduce code size.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Mon, 9 Jul 2007, Nick Piggin wrote:

  A reason for retaining slob would be that it has some O(n) memory saving
  due to better packing, etc.  Indeed that was the reason for merging it in
  the first place.  If slob no longer retains that advantage (wrt slub) then
  we no longer need it.
 
 SLOB contains several significant O(1) and also O(n) memory savings that
 are so far impossible-by-design for SLUB. They are: slab external
 fragmentation is significantly reduced; kmalloc internal fragmentation is
 significantly reduced; order of magnitude smaller kmem_cache data type;
 order of magnitude less code...

Well that is only true for kmalloc objects  PAGE_SIZE and to some extend 
offset by the need to keep per object data in SLUB. But yes the power of 
two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
operations of kmalloc slabs which in turns causes memory wastage because 
of rounding of the alloc to the next power of two. SLUB has less wastage
there than SLAB since it can fit power of two object tightly into a slab 
instead of having to place additional control information there like SLAB.

O(n) memory savings? What is that?
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Mon, 9 Jul 2007, Pekka Enberg wrote:

 I assume with slab external fragmentation you mean allocating a
 whole page for a slab when there are not enough objects to fill the
 whole thing thus wasting memory? We could try to combat that by
 packing multiple variable-sized slabs within a single page. Also,
 adding some non-power-of-two kmalloc caches might help with internal
 fragmentation.

Ther are already non-power-of-two kmalloc caches for 96 and 192 bytes 
sizes.
 
 In any case, SLUB needs some serious tuning for smaller machines
 before we can get rid of SLOB.

Switch off CONFIG_SLUB_DEBUG to get memory savings.
 
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Andrew Morton]

On Mon, 9 Jul 2007 09:06:46 -0700 (PDT) Christoph Lameter [EMAIL PROTECTED] 
wrote:

 But yes the power of 
 two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
 operations of kmalloc slabs which in turns causes memory wastage because 
 of rounding of the alloc to the next power of two.

I've frequently wondered why we don't just create more caches for kmalloc:
make it denser than each-power-of-2-plus-a-few-others-in-between.

I assume the tradeoff here is better packing versus having a ridiculous
number of caches.  Is there any other cost?

Because even having 1024 caches wouldn't consume a terrible amount of
memory and I bet it would result in aggregate savings.

Of course, a scheme which creates kmalloc caches on-demand would be better,
but that would kill our compile-time cache selection, I suspect.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Mon, 9 Jul 2007, Andrew Morton wrote:

 On Mon, 9 Jul 2007 09:06:46 -0700 (PDT) Christoph Lameter [EMAIL PROTECTED] 
 wrote:
 
  But yes the power of 
  two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
  operations of kmalloc slabs which in turns causes memory wastage because 
  of rounding of the alloc to the next power of two.
 
 I've frequently wondered why we don't just create more caches for kmalloc:
 make it denser than each-power-of-2-plus-a-few-others-in-between.
 
 I assume the tradeoff here is better packing versus having a ridiculous
 number of caches.  Is there any other cost?
 Because even having 1024 caches wouldn't consume a terrible amount of
 memory and I bet it would result in aggregate savings.

I have tried any number of approaches without too much success. Even one 
slab cache for every 8 bytes. This creates additional admin overhead 
through more control structure (that is pretty minimal but nevertheless 
exists)

The main issue is that kmallocs of different size must use different 
pages. If one allocates one 64 byte item and one 256 byte item and both 64 
byte and 256 byte are empty then SLAB/SLUB will have to allocate 2 pages. 
SLUB can fit them into one. This is basically only relevant early after 
boot. The advantage goes away as the system starts to work and as more 
objects are allocated in the slabs but the power-of-two slab will always
have to extend its size in page size chunks which leads to some overhead 
that SLOB can avoid by placing entities of multiple size in one slab. 
The tradeoff in SLOB is that is cannot be an O(1) allocator because it 
has to manage these variable sized objects by traversing the lists.

I think the advantage that SLOB generates here is pretty minimal and is 
easily offset by the problems of maintaining SLOB.
 
 Of course, a scheme which creates kmalloc caches on-demand would be better,
 but that would kill our compile-time cache selection, I suspect.

SLUB creates kmalloc caches on-demand for DMA caches already. But then we 
are not allowing compile time cache selection for DMA caches.

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Andrew Morton]

On Mon, 9 Jul 2007 10:26:08 -0700 (PDT)
Christoph Lameter [EMAIL PROTECTED] wrote:

  I assume the tradeoff here is better packing versus having a ridiculous
  number of caches.  Is there any other cost?
  Because even having 1024 caches wouldn't consume a terrible amount of
  memory and I bet it would result in aggregate savings.
 
 I have tried any number of approaches without too much success. Even one 
 slab cache for every 8 bytes. This creates additional admin overhead 
 through more control structure (that is pretty minimal but nevertheless 
 exists)
 
 The main issue is that kmallocs of different size must use different 
 pages. If one allocates one 64 byte item and one 256 byte item and both 64 
 byte and 256 byte are empty then SLAB/SLUB will have to allocate 2 pages. 
 SLUB can fit them into one. This is basically only relevant early after 
 boot. The advantage goes away as the system starts to work and as more 
 objects are allocated in the slabs but the power-of-two slab will always
 have to extend its size in page size chunks which leads to some overhead 
 that SLOB can avoid by placing entities of multiple size in one slab. 
 The tradeoff in SLOB is that is cannot be an O(1) allocator because it 
 has to manage these variable sized objects by traversing the lists.
 
 I think the advantage that SLOB generates here is pretty minimal and is 
 easily offset by the problems of maintaining SLOB.

Sure.  But I wasn't proposing this as a way to make slub cover slob's advantage.
I was wondering what effect it would have on a more typical medium to large 
sized
system.

Not much, really: if any particular subsystem is using a lot of slab memory 
then
it should create its own cache rather than using kmalloc anyway, so forget it ;)
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

First, WTF wasn't I cc:ed on this? Are you actually trying to me make
me fuming mad?

On Sat, Jul 07, 2007 at 08:50:01PM -0700, Christoph Lameter wrote:
 Maintenance of slab allocators becomes a problem as new features for
 allocators are developed. The SLOB allocator in particular has been lagging
 behind in many ways in the past:
 
 - Had no support for SLAB_DESTROY_BY_RCU for years (but no one noticed)

We've been over this 50 times. The target users were never affected.
And it's fixed. So why the HELL are you mentioning this again? 
 
 - Still has no support for slab reclaim counters. This may currently not
   be necessary if one would restrict the supported configurations for
   functionality relying on these. But even that has not been done.

We've been over this 50 times. Last time around, I inspected all the
code paths and demonstrated that despite your handwaving, IT DIDN'T
MATTER.

 The only current advantage over SLUB in terms of memory savings is through
 SLOBs kmalloc layout that is not power of two based like SLAB and SLUB which
 allows to eliminate some memory waste.
 
 Through that SLOB has still a slight memory advantage over SLUB of ~350k in
 for a standard server configuration. It is likely that the savings are is
 smaller for real embedded configurations that have less functionality.

Sometimes I do not think there is a cluebat large enough for you. 350K
is FREAKING HUGE on a cell phone. That's most of a kernel!

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Sun, Jul 08, 2007 at 11:02:24AM -0700, Andrew Morton wrote:
 Guys, look at this the other way.  Suppose we only had slub, and someone
 came along and said here's a whole new allocator which saves 4.5k of
 text, would we merge it on that basis?  Hell no, it's not worth it.  What
 we might do is to get motivated to see if we can make slub less porky under
 appropriate config settings.

Well I think we would obviously throw out SLAB and SLUB if they
weren't somewhat faster than SLOB. They're much more problematic and
one of the big features that Christoph's pushing is a fix for a
problem that SLOB simply doesn't have: huge numbers of SLAB/SLUB pages
being held down by small numbers of objects. 

 Let's not get sentimental about these things: in general, if there's any
 reasonable way in which we can rid ourselves of any code at all, we should
 do so, no?

I keep suggesting a Voyager Replacement Fund, but James isn't
interested.

But seriously, I don't think it should be at all surprising that the
allocator that's most appropriate for machines with  32MB of RAM is
different than the one for machines with  1TB of RAM.

The maintenance overhead of SLOB is fairly minimal. The biggest
outstanding SLOB problem is nommu's rather broken memory size
reporting.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Mon, Jul 09, 2007 at 09:51:16AM -0700, Andrew Morton wrote:
 On Mon, 9 Jul 2007 09:06:46 -0700 (PDT) Christoph Lameter [EMAIL PROTECTED] 
 wrote:
 
  But yes the power of 
  two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
  operations of kmalloc slabs which in turns causes memory wastage because 
  of rounding of the alloc to the next power of two.
 
 I've frequently wondered why we don't just create more caches for kmalloc:
 make it denser than each-power-of-2-plus-a-few-others-in-between.
 
 I assume the tradeoff here is better packing versus having a ridiculous
 number of caches.  Is there any other cost?

It magnifies the fragmentation problem.

SLAB (and SLUB) makes the optimistic assumption that objects of the
same type/size have similar lifetimes. But for some objects, it's not
uncommon to do many temporary allocations but have some objects with
indefinite lifespans. dcache is a very frequently encountered example,
but there's no reason we couldn't see it with sockets and many other
object types. 

Every new arena introduces further opportunity for this sort of
fragmentation. If we had, say, separate pools for 48 byte and 64 byte
objects, an unfortunate usage pattern for 48-byte kmallocs could DoS
requests for 64 byte objects that would work just fine if they both
came out of the same pool. If we have 10 pools with long-lived
objects, we're much worse off than if we had 1 or 2.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Mon, 9 Jul 2007, Nick Piggin wrote:



A reason for retaining slob would be that it has some O(n) memory saving
due to better packing, etc.  Indeed that was the reason for merging it in
the first place.  If slob no longer retains that advantage (wrt slub) then
we no longer need it.


SLOB contains several significant O(1) and also O(n) memory savings that
are so far impossible-by-design for SLUB. They are: slab external
fragmentation is significantly reduced; kmalloc internal fragmentation is
significantly reduced; order of magnitude smaller kmem_cache data type;
order of magnitude less code...



Well that is only true for kmalloc objects  PAGE_SIZE and to some extend 
offset by the need to keep per object data in SLUB. But yes the power of 
two caches are a necessary design feature of SLAB/SLUB that allows O(1) 
operations of kmalloc slabs which in turns causes memory wastage because 
of rounding of the alloc to the next power of two. SLUB has less wastage
there than SLAB since it can fit power of two object tightly into a slab 
instead of having to place additional control information there like SLAB.


OK but we're talking about SLOB. And the number that matters is the amount
of memory used, which is higher with SLUB than with SLOB in our tests.



O(n) memory savings? What is that?


Allocate n things and your memory waste is proportional to n (well that's
O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
to SLUB are proportional to n).

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Mon, 9 Jul 2007, Andrew Morton wrote:


I think the advantage that SLOB generates here is pretty minimal and is 
easily offset by the problems of maintaining SLOB.


I don't get it. Have you got agreement from the small memory people
that the advantages of SLOB are pretty minimal, or did you just
decide that? If the latter, did you completely miss reading my email?
What happens to the people who jump through hoops to save 1 or 2 K?

I don't see any problems with maintaining SLOB. It is simple enough
that I was able to write a userspace test harness for it and hack
away at it after reading the code the first time for half an hour or
so. It is nothing even slightly comparable to the problems of SLAB,
for example. And you don't have to maintain it at all anyway!

I like removing code as much as the next person, but I don't
understand why you are so intent on removing SLOB and willing to
dismiss its advantages so quickly.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

  O(n) memory savings? What is that?
 
 Allocate n things and your memory waste is proportional to n (well that's
 O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
 to SLUB are proportional to n).

n is the size of the object?

Its linear correlated to the object size. It does not grow 
exponentially as object size grows. Waste in the kmalloc array in the 
worst case is  objsize.

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

 I don't see any problems with maintaining SLOB. It is simple enough
 that I was able to write a userspace test harness for it and hack
 away at it after reading the code the first time for half an hour or
 so. It is nothing even slightly comparable to the problems of SLAB,
 for example. And you don't have to maintain it at all anyway!

I have to maintain it because I have to keep the slab APIs consistent 
(recently I added GFP_ZERO support and had to provide shims for slab 
defreag). It is not in a good state as described in the patch and has a 
history of not being maintained properly. Everyone that modifies the 
behavior of the slab allocator has to do something to avoid breaking SLOB. 
Its certainly fun to hack on but is that a criterion for keeping it in the 
tree?

 I like removing code as much as the next person, but I don't
 understand why you are so intent on removing SLOB and willing to
 dismiss its advantages so quickly.

Quickly? We have considered this for months now.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Tue, 10 Jul 2007, Nick Piggin wrote:



O(n) memory savings? What is that?


Allocate n things and your memory waste is proportional to n (well that's
O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
to SLUB are proportional to n).



n is the size of the object?


n things -- n number of things (n calls to kmem_cache_alloc()).

Just a fancy way of saying roughly that memory waste will increase as
the size of the system increases. But that aspect of it I think is
not really a problem for non-tiny systems anyway because the waste
tends not to be too bad (and possibly the number of active allocations
does not increase O(n) with the size of RAM either).

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Christoph Lameter wrote:

On Tue, 10 Jul 2007, Nick Piggin wrote:



I don't see any problems with maintaining SLOB. It is simple enough
that I was able to write a userspace test harness for it and hack
away at it after reading the code the first time for half an hour or
so. It is nothing even slightly comparable to the problems of SLAB,
for example. And you don't have to maintain it at all anyway!



I have to maintain it because I have to keep the slab APIs consistent 
(recently I added GFP_ZERO support and had to provide shims for slab 
defreag). It is not in a good state as described in the patch and has a 
history of not being maintained properly. Everyone that modifies the 
behavior of the slab allocator has to do something to avoid breaking SLOB. 
Its certainly fun to hack on but is that a criterion for keeping it in the 
tree?


Pretty standard fare that when you add something or change APIs, most
of the burden is on you to not break the kernel. I'd love nothing better
than to remove all but about 3 filesystems :)

It is reasonable to expect some help from maintainers, but I notice you
didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
if you tried working a bit closer with him you could get better results?



I like removing code as much as the next person, but I don't
understand why you are so intent on removing SLOB and willing to
dismiss its advantages so quickly.



Quickly? We have considered this for months now.


Quickly -- as in you quickly sweep the savings of 100s of K under the
rug and just declare that it is insignificant :)

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Christoph Lameter]

On Tue, 10 Jul 2007, Nick Piggin wrote:

 It is reasonable to expect some help from maintainers, but I notice you
 didn't even CC the SLOB maintainer in the patch to remove SLOB! So maybe
 if you tried working a bit closer with him you could get better results?

The maintainers last patch to SLOB was the initial submission of the 
allocator. Then he acked subsequent patches. Most of the modifications to 
SLOB are my work. Attempts to talk to the maintainer result in inventive 
explanations why SLOB does not have to conform to kernel standards. There 
is no reasonable expectation that this will change.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Matt Mackall]

On Mon, Jul 09, 2007 at 06:51:51PM -0700, Christoph Lameter wrote:
 On Tue, 10 Jul 2007, Nick Piggin wrote:
 
   O(n) memory savings? What is that?
  
  Allocate n things and your memory waste is proportional to n (well that's
  O(n) waste, so I guess by savings I mean that SLOB's memory saving compared
  to SLUB are proportional to n).
 
 n is the size of the object?
 
 Its linear correlated to the object size. It does not grow 
 exponentially as object size grows. Waste in the kmalloc array in the 
 worst case is  objsize.

N is the number of objects.

So, for large N, the overhead of SLUB of allocations of size m, where
m % alignment == 0 (for simplicity):

 kmem_cache_alloc case:

  (PAGE_SIZE % m) / m

 kmalloc case:

  nextpowerof2(m) - m

And for SLOB (with Nick's improvements):

 kmem_cache_alloc case:

  0 (the remainder of the page is usable for other allocs)

 kmalloc case:

  2 bytes (or whatever the minimal arch alignment is)

SLUB wins by two bytes on kmallocs that happen to be a power of two,
ties on kmem_cache_allocs that happen to evenly divide PAGE_SIZE and
kmallocs 2 bytes smaller than a power of 2, and loses everywhere else.

It also loses whenever a particular type/size has both short-lived and
long-lived objects because of pinning. This effect is worse on small
machines as it's easier to pin all of memory.

The downsides of current SLOB are:

 - single lock
 - potentially long walks to find free blocks

The average case walk has been shortened quite a bit by Nick's
patches. The combinatorics here are enough to make Knuth hand-wave,
but we can in theory get close to O(1) performance when we don't have
much memory pressure. Needs some serious benchmarking. When we DO have
memory pressure, both SLOB and SLUB will degrade to forcing the kernel
to walk a bunch of pages.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Andrew Morton wrote:

On Sun, 8 Jul 2007 09:51:19 +0200 Ingo Molnar <[EMAIL PROTECTED]> wrote:


A year ago the -rt kernel defaulted to the SLOB for a few releases, and 
barring some initial scalability issues (which were solved in -rt) it 
worked pretty well on generic PCs, so i dont buy the 'it doesnt work' 
argument either.





I don't think a saving of a few k of text would justify slob's retention.


Probably not.



A reason for retaining slob would be that it has some O(n) memory saving
due to better packing, etc.  Indeed that was the reason for merging it in
the first place.  If slob no longer retains that advantage (wrt slub) then
we no longer need it.


SLOB contains several significant O(1) and also O(n) memory savings that
are so far impossible-by-design for SLUB. They are: slab external
fragmentation is significantly reduced; kmalloc internal fragmentation is
significantly reduced; order of magnitude smaller kmem_cache data type;
order of magnitude less code...

Actually with an unscientific test boot of a semi-stripped down kernel and
mem=16MB, SLOB (the version in -mm) uses 400K less than SLUB (or a full 50%
more RAM free after booting into bash as the init).

Now it's not for me to say that this is significant enough to make SLOB
worth keeping, or what sort of results it yields in the field, so I cc'ed
Denis who is the busybox maintainer, and Erik who is ulibc maintainer in
case they have anything to add.



Guys, look at this the other way.  Suppose we only had slub, and someone
came along and said "here's a whole new allocator which saves 4.5k of
text", would we merge it on that basis?  Hell no, it's not worth it.  What
we might do is to get motivated to see if we can make slub less porky under
appropriate config settings.



In light of Denis's recent statement I saw "In busybox project people
can kill for 1.7K", there might be a mass killing of kernel developers
in Cambridge this year if SLOB gets removed ;)

Joking aside, the last time this came up, I thought we concluded that
removal of SLOB would be a severe regression for a significant userbase.



Let's not get sentimental about these things: in general, if there's any
reasonable way in which we can rid ourselves of any code at all, we should
do so, no?


Definitely. And this is exactly what we said last time as well. If the
small memory embedded guys are happy for SLOB to go, then I'm happy too.
So, the relevant question is -- are most/all current SLOB users are now
happy to switch over to SLUB, in light of the recent advances to both
allocators?

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Andrew Morton]

On Sun, 8 Jul 2007 09:51:19 +0200 Ingo Molnar <[EMAIL PROTECTED]> wrote:

> 
> (added Matt to the Cc: list)
> 
> * Christoph Lameter <[EMAIL PROTECTED]> wrote:
> 
> > Maintenance of slab allocators becomes a problem as new features for 
> > allocators are developed. The SLOB allocator in particular has been 
> > lagging behind in many ways in the past:
> > 
> > - Had no support for SLAB_DESTROY_BY_RCU for years (but no one 
> >   noticed)
> > 
> > - Still has no support for slab reclaim counters. This may currently 
> >   not be necessary if one would restrict the supported configurations 
> >   for functionality relying on these. But even that has not been done.
> > 
> > The only current advantage over SLUB in terms of memory savings is 
> > through SLOBs kmalloc layout that is not power of two based like SLAB 
> > and SLUB which allows to eliminate some memory waste.
> > 
> > Through that SLOB has still a slight memory advantage over SLUB of 
> > ~350k in for a standard server configuration. It is likely that the 
> > savings are is smaller for real embedded configurations that have less 
> > functionality.
> 
> actually, one real advantage of the SLOB is that it is a minimal, really 
> simple allocator. Its text and data size is so small as well.
> 
> here's the size comparison:
> 
>textdata bss dec hex filename
>   10788 837  16   116412d79 mm/slab.o
>62054207 124   105362928 mm/slub.o
>1640  44   41688 698 mm/slob.o
> 
> slab/slub have roughly the same footprint, but slob is 10% of that size. 
> Would be a waste to throw this away.
> 
> A year ago the -rt kernel defaulted to the SLOB for a few releases, and 
> barring some initial scalability issues (which were solved in -rt) it 
> worked pretty well on generic PCs, so i dont buy the 'it doesnt work' 
> argument either.
> 

I don't think a saving of a few k of text would justify slob's retention.

A reason for retaining slob would be that it has some O(n) memory saving
due to better packing, etc.  Indeed that was the reason for merging it in
the first place.  If slob no longer retains that advantage (wrt slub) then
we no longer need it.


Guys, look at this the other way.  Suppose we only had slub, and someone
came along and said "here's a whole new allocator which saves 4.5k of
text", would we merge it on that basis?  Hell no, it's not worth it.  What
we might do is to get motivated to see if we can make slub less porky under
appropriate config settings.


Let's not get sentimental about these things: in general, if there's any
reasonable way in which we can rid ourselves of any code at all, we should
do so, no?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Ingo Molnar]

* Nick Piggin <[EMAIL PROTECTED]> wrote:

> >cool. I was referring to something else: people were running -rt on 
> >their beefy desktop boxes with several gigs of RAM they complained 
> >about the slowdown that is caused by SLOB's linear list walking.
> 
> That is what I meant by scalable to large memories. It is not perfect, 
> but it is much better now. I noticed huge slowdowns too when test 
> booting the slob RCU patch on my 4GB desktop, so I did a few things to 
> improve freelist walking as well (the patches are in -mm, prefixed 
> with slob-).

ah, good - i only looked at the upstream mm/slob.c git-log. I like those 
4 slob-* patches in -mm: in particular slob-remove-bigblock-tracking.patch
and slob-rework-freelist-handling.patch are really elegant. Simplicity of 
slob does not seem to suffer either.

Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Ingo Molnar wrote:

* Nick Piggin <[EMAIL PROTECTED]> wrote:


I said exactly the same thing last time this came up. I would love to 
remove code if its functionality can be adequately replaced by 
existing code, but I think your reasons for removing SLOB aren't that 
good, and just handwaving away the significant memory savings doesn't 
work.



yeah. Also, the decision here is pretty easy: the behavior of the 
allocator is not really visible to applications. So this isnt like 
having a parallel IO scheduler or a parallel process scheduler (which 
cause problems to us by fragmenting the application space) - so the 
long-term cost to us kernel maintainers should be relatively low.


Yep.


A year ago the -rt kernel defaulted to the SLOB for a few releases, 
and barring some initial scalability issues (which were solved in 
-rt) it worked pretty well on generic PCs, so i dont buy the 'it 
doesnt work' argument either.


It's actually recently been made to work on SMP, it is much more 
scalable to large memories, and some initial NUMA work is happening 
that some embedded guys are interested in, all without increasing 
static footprint too much, and it has actually decreased dynamic 
footprint too.



cool. I was referring to something else: people were running -rt on 
their beefy desktop boxes with several gigs of RAM they complained about 
the slowdown that is caused by SLOB's linear list walking.


That is what I meant by scalable to large memories. It is not perfect,
but it is much better now. I noticed huge slowdowns too when test booting
the slob RCU patch on my 4GB desktop, so I did a few things to improve
freelist walking as well (the patches are in -mm, prefixed with slob-).

Afterwards, performance seems to be fairly good (obviously not as good
as SLAB or SLUB on such a configuration, but definitely usable and the
desktop was not noticably slower).


Steve Rostedt did two nice changes to fix those scalability problems. 
I've attached Steve's two patches. With these in place SLOB was very 
usable for large systems as well, with no measurable overhead. 
(obviously the lack of per-cpu caching can still be measured, but it's a 
lot less of a problem in practice than the linear list walking was.)


Thanks for sending those. One is actually obsolete because we removed
bigblock list completely, however I had not seen the other one. Such an
approach could be used, OTOH, having all allocations come from the same
pool does have its advantages in terms of memory usage.

I don't think it has been quite decided on the next step to take with
SLOB, however I have an idea that if we had per-cpu freelists (where
other lists could be used as a fallback), then that would go a long way
to improving the SMP scalability, CPU cache hotness, and long list
walking issues all at once.

However I like the fact that there is no need for a big rush to improve
it, and so patches and ideas can be brewed up slowly :)

Thanks,
Nick

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Ingo Molnar]

* Nick Piggin <[EMAIL PROTECTED]> wrote:

> I said exactly the same thing last time this came up. I would love to 
> remove code if its functionality can be adequately replaced by 
> existing code, but I think your reasons for removing SLOB aren't that 
> good, and just handwaving away the significant memory savings doesn't 
> work.

yeah. Also, the decision here is pretty easy: the behavior of the 
allocator is not really visible to applications. So this isnt like 
having a parallel IO scheduler or a parallel process scheduler (which 
cause problems to us by fragmenting the application space) - so the 
long-term cost to us kernel maintainers should be relatively low.

> > A year ago the -rt kernel defaulted to the SLOB for a few releases, 
> > and barring some initial scalability issues (which were solved in 
> > -rt) it worked pretty well on generic PCs, so i dont buy the 'it 
> > doesnt work' argument either.
> 
> It's actually recently been made to work on SMP, it is much more 
> scalable to large memories, and some initial NUMA work is happening 
> that some embedded guys are interested in, all without increasing 
> static footprint too much, and it has actually decreased dynamic 
> footprint too.

cool. I was referring to something else: people were running -rt on 
their beefy desktop boxes with several gigs of RAM they complained about 
the slowdown that is caused by SLOB's linear list walking.

Steve Rostedt did two nice changes to fix those scalability problems. 
I've attached Steve's two patches. With these in place SLOB was very 
usable for large systems as well, with no measurable overhead. 
(obviously the lack of per-cpu caching can still be measured, but it's a 
lot less of a problem in practice than the linear list walking was.)

Ingo
This patch uses the mem_map pages to find the bigblock descriptor for
large allocations.

-- Steve

Signed-off-by: Steven Rostedt <[EMAIL PROTECTED]>

 mm/slob.c |   74 ++
 1 file changed, 41 insertions(+), 33 deletions(-)

Index: linux/mm/slob.c
===
--- linux.orig/mm/slob.c
+++ linux/mm/slob.c
@@ -49,15 +49,42 @@ typedef struct slob_block slob_t;
 struct bigblock {
int order;
void *pages;
-   struct bigblock *next;
 };
 typedef struct bigblock bigblock_t;
 
 static slob_t arena = { .next = , .units = 1 };
 static slob_t *slobfree = 
-static bigblock_t *bigblocks;
 static DEFINE_SPINLOCK(slob_lock);
-static DEFINE_SPINLOCK(block_lock);
+
+#define __get_slob_block(b) ((unsigned long)(b) & ~(PAGE_SIZE-1))
+
+static inline struct page *get_slob_page(const void *mem)
+{
+   void *virt = (void*)__get_slob_block(mem);
+
+   return virt_to_page(virt);
+}
+
+static inline void zero_slob_block(const void *b)
+{
+   struct page *page;
+   page = get_slob_page(b);
+   memset(>lru, 0, sizeof(page->lru));
+}
+
+static inline void *get_slob_block(const void *b)
+{
+   struct page *page;
+   page = get_slob_page(b);
+   return page->lru.next;
+}
+
+static inline void set_slob_block(const void *b, void *data)
+{
+   struct page *page;
+   page = get_slob_page(b);
+   page->lru.next = data;
+}
 
 static void slob_free(void *b, int size);
 static void slob_timer_cbk(void);
@@ -109,6 +136,7 @@ static void *slob_alloc(size_t size, gfp
if (!cur)
return 0;
 
+   zero_slob_block(cur);
slob_free(cur, PAGE_SIZE);
spin_lock_irqsave(_lock, flags);
cur = slobfree;
@@ -163,7 +191,6 @@ void *__kmalloc(size_t size, gfp_t gfp)
 {
slob_t *m;
bigblock_t *bb;
-   unsigned long flags;
 
if (size < PAGE_SIZE - SLOB_UNIT) {
m = slob_alloc(size + SLOB_UNIT, gfp, 0);
@@ -178,10 +205,7 @@ void *__kmalloc(size_t size, gfp_t gfp)
bb->pages = (void *)__get_free_pages(gfp, bb->order);
 
if (bb->pages) {
-   spin_lock_irqsave(_lock, flags);
-   bb->next = bigblocks;
-   bigblocks = bb;
-   spin_unlock_irqrestore(_lock, flags);
+   set_slob_block(bb->pages, bb);
return bb->pages;
}
 
@@ -192,25 +216,16 @@ EXPORT_SYMBOL(__kmalloc);
 
 void kfree(const void *block)
 {
-   bigblock_t *bb, **last = 
-   unsigned long flags;
+   bigblock_t *bb;
 
if (!block)
return;
 
-   if (!((unsigned long)block & (PAGE_SIZE-1))) {
-   /* might be on the big block list */
-   spin_lock_irqsave(_lock, flags);
-   for (bb = bigblocks; bb; last = >next, bb = bb->next) {
-   if (bb->pages == block) {
-   *last = bb->next;
-   spin_unlock_irqrestore(_lock, flags);
-   

Re: [patch 09/10] Remove the SLOB allocator for 2.6.23

[Nick Piggin]

Ingo Molnar wrote:

(added Matt to the Cc: list)

* Christoph Lameter <[EMAIL PROTECTED]> wrote:


Maintenance of slab allocators becomes a problem as new features for 
allocators are developed. The SLOB allocator in particular has been 
lagging behind in many ways in the past:


- Had no support for SLAB_DESTROY_BY_RCU for years (but no one 
 noticed)


It likely was not frequently used on SMP, I guess.


- Still has no support for slab reclaim counters. This may currently 
 not be necessary if one would restrict the supported configurations 
 for functionality relying on these. But even that has not been done.


SLOB has so far run fine without any of these, hasn't it?


The only current advantage over SLUB in terms of memory savings is 
through SLOBs kmalloc layout that is not power of two based like SLAB 
and SLUB which allows to eliminate some memory waste.


Wrong. All "slabs" allocate out of the same pool of memory in SLOB, so
you also wind up with less waste via _external_ fragmentation, which is
espeically important on small memory machines (the kmalloc layout issue
is a problem of internal fragmentation). SLOB is also smaller and simpler
code as Ingo pointed out.


Through that SLOB has still a slight memory advantage over SLUB of 
~350k in for a standard server configuration. It is likely that the 
savings are is smaller for real embedded configurations that have less 
functionality.


When I last tested, I got similar savings with a pretty stripped down
kernel and a small mem= available RAM. Ie. to the point where those 350K
saved were a very significant chunk of remaining free memory after init
comes up.

I said exactly the same thing last time this came up. I would love to
remove code if its functionality can be adequately replaced by existing
code, but I think your reasons for removing SLOB aren't that good, and
just handwaving away the significant memory savings doesn't work.

People run 2.6 kernels with several MB of RAM, don't they? So losing
several hundred K is as bad to them as a patch that causes an Altix to
waste several hundred GB is to you.


A year ago the -rt kernel defaulted to the SLOB for a few releases, and 
barring some initial scalability issues (which were solved in -rt) it 
worked pretty well on generic PCs, so i dont buy the 'it doesnt work' 
argument either.


It's actually recently been made to work on SMP, it is much more scalable
to large memories, and some initial NUMA work is happening that some
embedded guys are interested in, all without increasing static footprint
too much, and it has actually decreased dynamic footprint too.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/