Re: large lockdep bss (was: Re: [patch 01/28] cpu alloc: The allocator)

2007-11-16 Thread Andi Kleen
> How about moving this bit: > > +#ifndef ARCH_HAS_EARLY_ALLOC > +#define LARGEVAR(x,y) { static typeof(*x) __ ## x[y]; x = __ ## x; } > +#else > +#define LARGEVAR(x,y) x = arch_early_alloc(sizeof(*x) * y) > +#endif > > out of the lockdep code and into the generic early alloc code? Will do.

large lockdep bss (was: Re: [patch 01/28] cpu alloc: The allocator)

2007-11-16 Thread Peter Zijlstra
On Wed, 2007-11-14 at 03:28 +0100, Andi Kleen wrote: > On Tue, Nov 13, 2007 at 05:52:08PM -0800, David Miller wrote: > > Yes, I've run into similar problems with lockdep as well. > > I had to build an ultra minimalized kernel to get it to > > boot on my Niagara boxes. > > > > I think I even

large lockdep bss (was: Re: [patch 01/28] cpu alloc: The allocator)

2007-11-16 Thread Peter Zijlstra
On Wed, 2007-11-14 at 03:28 +0100, Andi Kleen wrote: On Tue, Nov 13, 2007 at 05:52:08PM -0800, David Miller wrote: Yes, I've run into similar problems with lockdep as well. I had to build an ultra minimalized kernel to get it to boot on my Niagara boxes. I think I even looked at the

Re: large lockdep bss (was: Re: [patch 01/28] cpu alloc: The allocator)

2007-11-16 Thread Andi Kleen
How about moving this bit: +#ifndef ARCH_HAS_EARLY_ALLOC +#define LARGEVAR(x,y) { static typeof(*x) __ ## x[y]; x = __ ## x; } +#else +#define LARGEVAR(x,y) x = arch_early_alloc(sizeof(*x) * y) +#endif out of the lockdep code and into the generic early alloc code? Will do. -Andi - To

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: > > There is no assembly code required. I overdid it in the patch that I sent > > you trying to make sparc64 use large mappings like x86_64 NUMA. You really > > do not need that. Look at the IA64 and i386 configurations. There is no C > > code required.

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Thu, 15 Nov 2007 19:10:15 -0800 (PST) > On Thu, 15 Nov 2007, David Miller wrote: > > > > I have not looked at sparc32 sorry. If you simply set up a couple of > > > configuration values in arch/sparc32/Kconfig then everything will be fine. > > >

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: > > I have not looked at sparc32 sorry. If you simply set up a couple of > > configuration values in arch/sparc32/Kconfig then everything will be fine. > > There is assembler code to write, which as I stated several > times nobody is going to work on or

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Thu, 15 Nov 2007 18:55:21 -0800 (PST) > On Thu, 15 Nov 2007, David Miller wrote: > > > Sparc32 is still broken, as just one of several possible examples. > > I have not looked at sparc32 sorry. If you simply set up a couple of > configuration

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: > If you're talking about the VMEMMAP thing, that patch didn't remove > the problem, it simply added optimizations for sparc64 so that you could > sweep the problem under the rug. The virtual mapping of the cpu areas is used by the patch I posted for

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Thu, 15 Nov 2007 18:19:37 -0800 (PST) > On Thu, 15 Nov 2007, David Miller wrote: > > > Well, the first version of the patch set, the one I tested, did > > require a lot of BSS space. And that's the one they are writing > > about. > > I am

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: > Well, the first version of the patch set, the one I tested, did > require a lot of BSS space. And that's the one they are writing > about. I am running the same version that you also ran. The problem is that you did not configure the stuff properly

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Thu, 15 Nov 2007 10:49:37 -0800 (PST) > Well there is an LWN article now that also claims that the cpu_alloc > patchset requires a large bss space. Sigh. See > > http://lwn.net/Articles/257828/ > > Not true! 44 bytes is reasonable. Well, the

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
Well there is an LWN article now that also claims that the cpu_alloc patchset requires a large bss space. Sigh. See http://lwn.net/Articles/257828/ Not true! 44 bytes is reasonable. [EMAIL PROTECTED]:~/linux-2.6$ size mm/cpu_alloc.o textdata bss dec hex filename 5625

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
Well there is an LWN article now that also claims that the cpu_alloc patchset requires a large bss space. Sigh. See http://lwn.net/Articles/257828/ Not true! 44 bytes is reasonable. [EMAIL PROTECTED]:~/linux-2.6$ size mm/cpu_alloc.o textdata bss dec hex filename 5625

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 10:49:37 -0800 (PST) Well there is an LWN article now that also claims that the cpu_alloc patchset requires a large bss space. Sigh. See http://lwn.net/Articles/257828/ Not true! 44 bytes is reasonable. Well, the first

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 18:19:37 -0800 (PST) On Thu, 15 Nov 2007, David Miller wrote: Well, the first version of the patch set, the one I tested, did require a lot of BSS space. And that's the one they are writing about. I am running the same

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: If you're talking about the VMEMMAP thing, that patch didn't remove the problem, it simply added optimizations for sparc64 so that you could sweep the problem under the rug. The virtual mapping of the cpu areas is used by the patch I posted for i386,

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 18:55:21 -0800 (PST) On Thu, 15 Nov 2007, David Miller wrote: Sparc32 is still broken, as just one of several possible examples. I have not looked at sparc32 sorry. If you simply set up a couple of configuration values in

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: Well, the first version of the patch set, the one I tested, did require a lot of BSS space. And that's the one they are writing about. I am running the same version that you also ran. The problem is that you did not configure the stuff properly for

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: There is no assembly code required. I overdid it in the patch that I sent you trying to make sparc64 use large mappings like x86_64 NUMA. You really do not need that. Look at the IA64 and i386 configurations. There is no C code required. The

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 19:10:15 -0800 (PST) On Thu, 15 Nov 2007, David Miller wrote: I have not looked at sparc32 sorry. If you simply set up a couple of configuration values in arch/sparc32/Kconfig then everything will be fine. There is

Re: [patch 01/28] cpu alloc: The allocator

2007-11-15 Thread Christoph Lameter
On Thu, 15 Nov 2007, David Miller wrote: I have not looked at sparc32 sorry. If you simply set up a couple of configuration values in arch/sparc32/Kconfig then everything will be fine. There is assembler code to write, which as I stated several times nobody is going to work on or test.

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 13 Nov 2007 20:26:33 -0800 (PST) > The only problem that I see so far is a communication problem. Thus > we need more documentation. Fair enough, I'll look more closely at the next rev of your patch set. - To unsubscribe from this list: send

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > From: Christoph Lameter <[EMAIL PROTECTED]> > Date: Tue, 13 Nov 2007 20:15:55 -0800 (PST) > > > Guess I need also to add an arch configuration guide to V2 as well so that > > the other arches can do similar tricks and emphasize that the static > >

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
BTW [EMAIL PROTECTED] does not exist, please remove it from the CC: in the future :-) Thanks. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 13 Nov 2007 20:15:55 -0800 (PST) > Guess I need also to add an arch configuration guide to V2 as well so that > the other arches can do similar tricks and emphasize that the static > default that requires bss is only suitable for small

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
Hmmm.. I have v2 in preparation here that puts the pda and the per cpu data into the cpu_alloc area. Thus gs: can be used to access all per cpu data. Any ideas how to abstract out the pda operations? Wasnt local_t supposed to be able to do atomic ops on cpu data? Is there an segment register

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Wed, 14 Nov 2007, Andi Kleen wrote: > On Tue, Nov 13, 2007 at 05:52:08PM -0800, David Miller wrote: > > Yes, I've run into similar problems with lockdep as well. > > I had to build an ultra minimalized kernel to get it to > > boot on my Niagara boxes. > > > > I think I even looked at the same

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Andi Kleen <[EMAIL PROTECTED]> Date: Wed, 14 Nov 2007 03:28:32 +0100 > ftp://firstfloor.org/pub/ak/x86_64/quilt/patches/early-reserve > ftp://firstfloor.org/pub/ak/x86_64/quilt/patches/early-alloc > ftp://firstfloor.org/pub/ak/x86_64/quilt/patches/lockdep-early-alloc > > I didn't plan to

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 05:52:08PM -0800, David Miller wrote: > Yes, I've run into similar problems with lockdep as well. > I had to build an ultra minimalized kernel to get it to > boot on my Niagara boxes. > > I think I even looked at the same lockdep code, and I'd > appreciate it if you'd

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > The idea about doling out vmalloc space seemed the most promising. Well that is basically the same as the virtual mode. Just ditch the fallback mode? vmalloc directly does not guarantee a fixed address. - To unsubscribe from this list: send the line

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > From: Christoph Lameter <[EMAIL PROTECTED]> > Date: Tue, 13 Nov 2007 17:57:15 -0800 (PST) > > > H. cpu_alloc really does not need zeroed data. Just an address > > fixed by the compiler where stuff can be put. Can the loader do that > > somehow? > >

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 13 Nov 2007 17:57:15 -0800 (PST) > H. cpu_alloc really does not need zeroed data. Just an address > fixed by the compiler where stuff can be put. Can the loader do that > somehow? Yes, and I think IA64 uses such a scheme for it's 64KB

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 13 Nov 2007 17:50:24 -0800 (PST) > On Tue, 13 Nov 2007, David Miller wrote: > > > BTW, I'm going to stop testing your patches on sparc64 for > > a while until you start to make me feel like you understand > > that ignoring the BSS bloat

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > Yes, I've run into similar problems with lockdep as well. > I had to build an ultra minimalized kernel to get it to > boot on my Niagara boxes. H. cpu_alloc really does not need zeroed data. Just an address fixed by the compiler where stuff can be

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Andi Kleen <[EMAIL PROTECTED]> Date: Wed, 14 Nov 2007 02:06:28 +0100 > David Miller <[EMAIL PROTECTED]> writes: > > > > The problem is that for the non-virtualized case this patch bloats up > > the BSS section to be more than 8MB in size. Sparc64 kernel images > > cannot be more than 8MB

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > I'm surprised this is your reaction instead of "oh damn, sorry > I bloated up the kernel image size by 8mb, I'll find a way to > fix that." Well I found a way to fix that and its in the patch... - To unsubscribe from this list: send the line

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > BTW, I'm going to stop testing your patches on sparc64 for > a while until you start to make me feel like you understand > that ignoring the BSS bloat issue is bad. Well this is just the fallback. How can I avoid this and still keep a constant? Add a

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
BTW, I'm going to stop testing your patches on sparc64 for a while until you start to make me feel like you understand that ignoring the BSS bloat issue is bad. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 13 Nov 2007 14:20:02 -0800 (PST) > On Tue, 13 Nov 2007, David Miller wrote: > > > Yes, I could add virtualized area support to sparc64, but we cannot > > impose this on every platform. > > Other platforms do not have the 8MB restriction nor

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 13 Nov 2007 14:00:45 -0800 (PST) > On Tue, 13 Nov 2007, Eric Dumazet wrote: > > > Once you add in mm/vmalloc.c all needed helpers, no need to use BSS Megablob > > anymore ? > > Well I think all of this can be avoided by simply copying the

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 13 Nov 2007 13:40:19 -0800 (PST) > On Tue, 13 Nov 2007, David Miller wrote: > > > One thing you could do is simply use a vmalloc allocation in the > > non-virtualized case. > > Yuck. Meaning to add more crappy code. The bss limitations to

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Andi Kleen
David Miller <[EMAIL PROTECTED]> writes: > > The problem is that for the non-virtualized case this patch bloats up > the BSS section to be more than 8MB in size. Sparc64 kernel images > cannot be more than 8MB in size total due to various boot loader and > firmware limitations. I recently ran

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > Yes, I could add virtualized area support to sparc64, but we cannot > impose this on every platform. Other platforms do not have the 8MB restriction nor do they have so many processors. Here is the draft of a virtual cpu area implementation for

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, Eric Dumazet wrote: > This is because NR_CPUS is defaulted to 32 on i386 (with a limit of 256), so > reserving 256*256KB = 64 MB of virtual space might be too much. (this is half > the typical vmalloc area) It defaults to 8 except if you use a NUMA system. config NR_CPUS

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, Eric Dumazet wrote: > Once you add in mm/vmalloc.c all needed helpers, no need to use BSS Megablob > anymore ? Well I think all of this can be avoided by simply copying the existing vmemmap helper functions and providing a virtual address for sparc64. - To unsubscribe from

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Eric Dumazet
Christoph Lameter a écrit : On Tue, 13 Nov 2007, David Miller wrote: One thing you could do is simply use a vmalloc allocation in the non-virtualized case. Yuck. Meaning to add more crappy code. The bss limitations to 8M is a bit strange though. Do other platforms have the same issues?

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: > One thing you could do is simply use a vmalloc allocation in the > non-virtualized case. Yuck. Meaning to add more crappy code. The bss limitations to 8M is a bit strange though. Do other platforms have the same issues? - To unsubscribe from this

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Tue, 06 Nov 2007 11:51:45 -0800 > The core portion of the cpu allocator. > > The per cpu allocator allows dynamic allocation of memory on all > processor simultaneously. A bitmap is used to track used areas. > The allocator implements tight

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 06 Nov 2007 11:51:45 -0800 The core portion of the cpu allocator. The per cpu allocator allows dynamic allocation of memory on all processor simultaneously. A bitmap is used to track used areas. The allocator implements tight packing to

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: One thing you could do is simply use a vmalloc allocation in the non-virtualized case. Yuck. Meaning to add more crappy code. The bss limitations to 8M is a bit strange though. Do other platforms have the same issues? - To unsubscribe from this list:

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Eric Dumazet
Christoph Lameter a écrit : On Tue, 13 Nov 2007, David Miller wrote: One thing you could do is simply use a vmalloc allocation in the non-virtualized case. Yuck. Meaning to add more crappy code. The bss limitations to 8M is a bit strange though. Do other platforms have the same issues?

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, Eric Dumazet wrote: Once you add in mm/vmalloc.c all needed helpers, no need to use BSS Megablob anymore ? Well I think all of this can be avoided by simply copying the existing vmemmap helper functions and providing a virtual address for sparc64. - To unsubscribe from

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, Eric Dumazet wrote: This is because NR_CPUS is defaulted to 32 on i386 (with a limit of 256), so reserving 256*256KB = 64 MB of virtual space might be too much. (this is half the typical vmalloc area) It defaults to 8 except if you use a NUMA system. config NR_CPUS

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: Yes, I could add virtualized area support to sparc64, but we cannot impose this on every platform. Other platforms do not have the 8MB restriction nor do they have so many processors. Here is the draft of a virtual cpu area implementation for sparc64.

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Andi Kleen
David Miller [EMAIL PROTECTED] writes: The problem is that for the non-virtualized case this patch bloats up the BSS section to be more than 8MB in size. Sparc64 kernel images cannot be more than 8MB in size total due to various boot loader and firmware limitations. I recently ran into a

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 13:40:19 -0800 (PST) On Tue, 13 Nov 2007, David Miller wrote: One thing you could do is simply use a vmalloc allocation in the non-virtualized case. Yuck. Meaning to add more crappy code. The bss limitations to 8M is a bit

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 14:00:45 -0800 (PST) On Tue, 13 Nov 2007, Eric Dumazet wrote: Once you add in mm/vmalloc.c all needed helpers, no need to use BSS Megablob anymore ? Well I think all of this can be avoided by simply copying the existing

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 14:20:02 -0800 (PST) On Tue, 13 Nov 2007, David Miller wrote: Yes, I could add virtualized area support to sparc64, but we cannot impose this on every platform. Other platforms do not have the 8MB restriction nor do they

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
BTW, I'm going to stop testing your patches on sparc64 for a while until you start to make me feel like you understand that ignoring the BSS bloat issue is bad. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: I'm surprised this is your reaction instead of oh damn, sorry I bloated up the kernel image size by 8mb, I'll find a way to fix that. Well I found a way to fix that and its in the patch... - To unsubscribe from this list: send the line unsubscribe

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: BTW, I'm going to stop testing your patches on sparc64 for a while until you start to make me feel like you understand that ignoring the BSS bloat issue is bad. Well this is just the fallback. How can I avoid this and still keep a constant? Add a new

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Andi Kleen [EMAIL PROTECTED] Date: Wed, 14 Nov 2007 02:06:28 +0100 David Miller [EMAIL PROTECTED] writes: The problem is that for the non-virtualized case this patch bloats up the BSS section to be more than 8MB in size. Sparc64 kernel images cannot be more than 8MB in size total

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: Yes, I've run into similar problems with lockdep as well. I had to build an ultra minimalized kernel to get it to boot on my Niagara boxes. H. cpu_alloc really does not need zeroed data. Just an address fixed by the compiler where stuff can be

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 17:50:24 -0800 (PST) On Tue, 13 Nov 2007, David Miller wrote: BTW, I'm going to stop testing your patches on sparc64 for a while until you start to make me feel like you understand that ignoring the BSS bloat issue is bad.

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 17:57:15 -0800 (PST) H. cpu_alloc really does not need zeroed data. Just an address fixed by the compiler where stuff can be put. Can the loader do that somehow? Yes, and I think IA64 uses such a scheme for it's 64KB fixed

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 17:57:15 -0800 (PST) H. cpu_alloc really does not need zeroed data. Just an address fixed by the compiler where stuff can be put. Can the loader do that somehow? Yes, and I

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: The idea about doling out vmalloc space seemed the most promising. Well that is basically the same as the virtual mode. Just ditch the fallback mode? vmalloc directly does not guarantee a fixed address. - To unsubscribe from this list: send the line

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 05:52:08PM -0800, David Miller wrote: Yes, I've run into similar problems with lockdep as well. I had to build an ultra minimalized kernel to get it to boot on my Niagara boxes. I think I even looked at the same lockdep code, and I'd appreciate it if you'd submit

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Andi Kleen [EMAIL PROTECTED] Date: Wed, 14 Nov 2007 03:28:32 +0100 ftp://firstfloor.org/pub/ak/x86_64/quilt/patches/early-reserve ftp://firstfloor.org/pub/ak/x86_64/quilt/patches/early-alloc ftp://firstfloor.org/pub/ak/x86_64/quilt/patches/lockdep-early-alloc I didn't plan to submit

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Wed, 14 Nov 2007, Andi Kleen wrote: On Tue, Nov 13, 2007 at 05:52:08PM -0800, David Miller wrote: Yes, I've run into similar problems with lockdep as well. I had to build an ultra minimalized kernel to get it to boot on my Niagara boxes. I think I even looked at the same lockdep

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
Hmmm.. I have v2 in preparation here that puts the pda and the per cpu data into the cpu_alloc area. Thus gs: can be used to access all per cpu data. Any ideas how to abstract out the pda operations? Wasnt local_t supposed to be able to do atomic ops on cpu data? Is there an segment register

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
BTW [EMAIL PROTECTED] does not exist, please remove it from the CC: in the future :-) Thanks. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread Christoph Lameter
On Tue, 13 Nov 2007, David Miller wrote: From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 20:15:55 -0800 (PST) Guess I need also to add an arch configuration guide to V2 as well so that the other arches can do similar tricks and emphasize that the static default that

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 20:15:55 -0800 (PST) Guess I need also to add an arch configuration guide to V2 as well so that the other arches can do similar tricks and emphasize that the static default that requires bss is only suitable for small systems.

Re: [patch 01/28] cpu alloc: The allocator

2007-11-13 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Tue, 13 Nov 2007 20:26:33 -0800 (PST) The only problem that I see so far is a communication problem. Thus we need more documentation. Fair enough, I'll look more closely at the next rev of your patch set. - To unsubscribe from this list: send the

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread David Miller
From: Peter Zijlstra <[EMAIL PROTECTED]> Date: Thu, 08 Nov 2007 21:19:08 +0100 > > On Thu, 2007-11-08 at 10:31 -0800, Christoph Lameter wrote: > > On Thu, 8 Nov 2007, Peter Zijlstra wrote: > > > > > I don't like those shouting macros. > > > > The convention for macros is to use upper case. >

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread David Miller
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Thu, 8 Nov 2007 12:24:22 -0800 (PST) > On Thu, 8 Nov 2007, Peter Zijlstra wrote: > > > > The convention for macros is to use upper case. > > > > We have plent macros that look like regular functions. And as a primary > > interface to this

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Christoph Lameter
On Thu, 8 Nov 2007, Peter Zijlstra wrote: > > The convention for macros is to use upper case. > > We have plent macros that look like regular functions. And as a primary > interface to this functionality these shouting things look really out of > place. One point of the patchset is to clean up

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Thu, 2007-11-08 at 10:50 -0800, Christoph Lameter wrote: > On Thu, 8 Nov 2007, Christoph Lameter wrote: > > > > Also, looking at the lock usage, this thing is not IRQ safe, so it > > > should not be called from hardirq context. Please document this. > > Well I went the other way and made it

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Thu, 2007-11-08 at 10:31 -0800, Christoph Lameter wrote: > On Thu, 8 Nov 2007, Peter Zijlstra wrote: > > > > +#define CPU_OFFSET(__cpu) \ > > > + ((unsigned long)(__cpu) << (CONFIG_CPU_AREA_ORDER + PAGE_SHIFT)) > > > + > > > +#define CPU_PTR(__p, __cpu) ((__typeof__(__p))((void *)(__p) + \ >

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Christoph Lameter
On Thu, 8 Nov 2007, Christoph Lameter wrote: > > Also, looking at the lock usage, this thing is not IRQ safe, so it > > should not be called from hardirq context. Please document this. Well I went the other way and made it work like the slab allocators. cpu_alloc: Make it irq safe Use the

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Christoph Lameter
On Thu, 8 Nov 2007, Peter Zijlstra wrote: > On Tue, 2007-11-06 at 11:51 -0800, Christoph Lameter wrote: > > > +/* > > + * Lock to protect the bitmap and the meta data for the cpu allocator. > > + */ > > +static DEFINE_SPINLOCK(cpu_alloc_map_lock); > > I thought you got nightmares from global

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Thu, 2007-11-08 at 13:34 +0100, Peter Zijlstra wrote: > On Tue, 2007-11-06 at 11:51 -0800, Christoph Lameter wrote: > > +#ifndef _LINUX_CPU_ALLOC_H_ > > +#define _LINUX_CPU_ALLOC_H_ > > + > > +#define CPU_OFFSET(__cpu) \ > > + ((unsigned long)(__cpu) << (CONFIG_CPU_AREA_ORDER + PAGE_SHIFT))

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Tue, 2007-11-06 at 11:51 -0800, Christoph Lameter wrote: > +/* > + * Lock to protect the bitmap and the meta data for the cpu allocator. > + */ > +static DEFINE_SPINLOCK(cpu_alloc_map_lock); I thought you got nightmares from global locks :-) > +/* > + * Allocate an object of a certain size >

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Tue, 2007-11-06 at 11:51 -0800, Christoph Lameter wrote: +/* + * Lock to protect the bitmap and the meta data for the cpu allocator. + */ +static DEFINE_SPINLOCK(cpu_alloc_map_lock); I thought you got nightmares from global locks :-) +/* + * Allocate an object of a certain size + * +

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Thu, 2007-11-08 at 13:34 +0100, Peter Zijlstra wrote: On Tue, 2007-11-06 at 11:51 -0800, Christoph Lameter wrote: +#ifndef _LINUX_CPU_ALLOC_H_ +#define _LINUX_CPU_ALLOC_H_ + +#define CPU_OFFSET(__cpu) \ + ((unsigned long)(__cpu) (CONFIG_CPU_AREA_ORDER + PAGE_SHIFT)) +

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Christoph Lameter
On Thu, 8 Nov 2007, Peter Zijlstra wrote: On Tue, 2007-11-06 at 11:51 -0800, Christoph Lameter wrote: +/* + * Lock to protect the bitmap and the meta data for the cpu allocator. + */ +static DEFINE_SPINLOCK(cpu_alloc_map_lock); I thought you got nightmares from global locks :-) Yes

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Christoph Lameter
On Thu, 8 Nov 2007, Christoph Lameter wrote: Also, looking at the lock usage, this thing is not IRQ safe, so it should not be called from hardirq context. Please document this. Well I went the other way and made it work like the slab allocators. cpu_alloc: Make it irq safe Use the same

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread David Miller
From: Peter Zijlstra [EMAIL PROTECTED] Date: Thu, 08 Nov 2007 21:19:08 +0100 On Thu, 2007-11-08 at 10:31 -0800, Christoph Lameter wrote: On Thu, 8 Nov 2007, Peter Zijlstra wrote: I don't like those shouting macros. The convention for macros is to use upper case. We have plent

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread David Miller
From: Christoph Lameter [EMAIL PROTECTED] Date: Thu, 8 Nov 2007 12:24:22 -0800 (PST) On Thu, 8 Nov 2007, Peter Zijlstra wrote: The convention for macros is to use upper case. We have plent macros that look like regular functions. And as a primary interface to this functionality these

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Thu, 2007-11-08 at 10:50 -0800, Christoph Lameter wrote: On Thu, 8 Nov 2007, Christoph Lameter wrote: Also, looking at the lock usage, this thing is not IRQ safe, so it should not be called from hardirq context. Please document this. Well I went the other way and made it work like

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Peter Zijlstra
On Thu, 2007-11-08 at 10:31 -0800, Christoph Lameter wrote: On Thu, 8 Nov 2007, Peter Zijlstra wrote: +#define CPU_OFFSET(__cpu) \ + ((unsigned long)(__cpu) (CONFIG_CPU_AREA_ORDER + PAGE_SHIFT)) + +#define CPU_PTR(__p, __cpu) ((__typeof__(__p))((void *)(__p) + \ +

Re: [patch 01/28] cpu alloc: The allocator

2007-11-08 Thread Christoph Lameter
On Thu, 8 Nov 2007, Peter Zijlstra wrote: The convention for macros is to use upper case. We have plent macros that look like regular functions. And as a primary interface to this functionality these shouting things look really out of place. One point of the patchset is to clean up the

[patch 01/28] cpu alloc: The allocator

2007-11-06 Thread Christoph Lameter
The core portion of the cpu allocator. The per cpu allocator allows dynamic allocation of memory on all processor simultaneously. A bitmap is used to track used areas. The allocator implements tight packing to reduce the cache footprint and increase speed since cacheline contention is typically

[patch 01/28] cpu alloc: The allocator

2007-11-06 Thread Christoph Lameter
The core portion of the cpu allocator. The per cpu allocator allows dynamic allocation of memory on all processor simultaneously. A bitmap is used to track used areas. The allocator implements tight packing to reduce the cache footprint and increase speed since cacheline contention is typically