Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-28 Thread Adrian Bunk
On Thu, Aug 28, 2008 at 09:32:13AM +0900, Paul Mundt wrote: On Wed, Aug 27, 2008 at 08:35:44PM +0300, Adrian Bunk wrote: On Thu, Aug 28, 2008 at 01:00:52AM +0900, Paul Mundt wrote: On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote: In addition to that, debugging the runaway

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Paul Mackerras
Linus Torvalds writes: 4kB used to be the _only_ choice. And no, there weren't even irq stacks. So that 4kB was not just the whole kernel call-chain, it was also all the irq nesting above it. I think your memory is failing you. In 2.4 and earlier, the kernel stack was 8kB minus the size of

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Bernd Petrovitsch
On Tue, 2008-08-26 at 18:54 -0400, Parag Warudkar wrote: On Tue, Aug 26, 2008 at 5:04 PM, Linus Torvalds [EMAIL PROTECTED] wrote: And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more! This is something I never

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Bernd Petrovitsch
On Tue, 2008-08-26 at 22:16 -0400, Parag Warudkar wrote: [...] Well, sure - but the industry as a whole seems to have gone the other The industry as a whole doesn't exist on that low level. You can't compare the laptop and/or desktop computer market (where one may buy today hardware that runs

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Alan Cox
What about deep call chains? The problem with the uptake of 4K stacks seems to be that is not reliably provable that it will work under all circumstances. On x86-32 with 8K stacks your IRQ paths share them so that is even harder to prove (not that you can prove any of them) and the bugs are

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Bernd Petrovitsch
On Tue, 2008-08-26 at 20:58 -0400, Parag Warudkar wrote: [...] The savings part -financial ones- are not always realizable with the way memory is priced/sized/fitted. Savings in few Mb of Kernel stack are not necessarily going to allow getting rid of a single memory chip of 64M or so. No, but

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 05:28:37PM -0700, Linus Torvalds wrote: On Wed, 27 Aug 2008, Adrian Bunk wrote: When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32? XFS may never have been usable, but the rest, sure. And you seem to be making

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 06:49:19PM -0700, Linus Torvalds wrote: ... But part of it is definitely gcc. Some versions of gcc used to be absolutely _horrid_ when it came to stack usage, especially with some flags, and especially with the crazy inlining that module-at-a-time caused. ... That

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Parag Warudkar
On Wed, Aug 27, 2008 at 5:00 AM, Bernd Petrovitsch [EMAIL PROTECTED] wrote: They probably gave the idea pretty soon because you need to rework/improve large parts of the kernel + drivers (and that has two major problems - it consumes a lot of man power for no new features and everything must

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Alan Cox
By your logic though, XFS on x86 should work fine with 4K stacks - many will attest that it does not and blows up due to stack issues. I have first hand experiences of things blowing up with deep call chains when using 4K stacks where 8K worked just fine on same workload. So there is

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Linus Torvalds
On Wed, 27 Aug 2008, Paul Mackerras wrote: I think your memory is failing you. In 2.4 and earlier, the kernel stack was 8kB minus the size of the task_struct, which sat at the start of the 8kB. Yup, you're right. Linus -- To unsubscribe from this list: send the

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Jamie Lokier
Bernd Petrovitsch wrote: If you develop an embedded system (which is partly system integration of existing apps) to be installed in the field, you don't have that many conceivable work loads compared to a desktop/server system. And you have a fixed list of drivers and applications. Hah! Not

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Paul Mundt
On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote: On Tue, Aug 26, 2008 at 05:28:37PM -0700, Linus Torvalds wrote: On Wed, 27 Aug 2008, Adrian Bunk wrote: When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32? XFS may never have

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Jamie Lokier
Linus Torvalds wrote: Most LOCs of the kernel are not written by people like you or Al Viro or David Miller, and the average kernel developer is unlikely to do it as good as gcc. Sure. But we do have tools. We do have checkstack.pl, it's just that it hasn't been an issue in a long

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Bernd Petrovitsch
On Wed, 2008-08-27 at 16:48 +0100, Jamie Lokier wrote: Bernd Petrovitsch wrote: If you develop an embedded system (which is partly system integration of existing apps) to be installed in the field, you don't have that many conceivable work loads compared to a desktop/server system. And you

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Adrian Bunk
On Thu, Aug 28, 2008 at 01:00:52AM +0900, Paul Mundt wrote: On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote: On Tue, Aug 26, 2008 at 05:28:37PM -0700, Linus Torvalds wrote: On Wed, 27 Aug 2008, Adrian Bunk wrote: When did we get callpaths like like nfs+xfs+md+scsi

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Jamie Lokier
Bernd Petrovitsch wrote: 32MB no-MMU ARM boards which people run new things and attach new devices to rather often - without making new hardware. Volume's too low per individual application to get new hardware designed and made. Yes, you may have several products on the same hardware

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Bernd Petrovitsch
On Mit, 2008-08-27 at 18:51 +0100, Jamie Lokier wrote: Bernd Petrovitsch wrote: [...] It is, but the idea that small embedded systems go through a 'all components are known, drivers are known, test and if it passes it's shippable' does not always apply. Not always but often enough. And yes,

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Greg Ungerer
Jamie Lokier wrote: Bernd Petrovitsch wrote: 32MB no-MMU ARM boards which people run new things and attach new devices to rather often - without making new hardware. Volume's too low per individual application to get new hardware designed and made. Yes, you may have several products on the

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Greg Ungerer
Jamie Lokier wrote: Bernd Petrovitsch wrote: If you develop an embedded system (which is partly system integration of existing apps) to be installed in the field, you don't have that many conceivable work loads compared to a desktop/server system. And you have a fixed list of drivers and

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Paul Mundt
On Wed, Aug 27, 2008 at 08:35:44PM +0300, Adrian Bunk wrote: On Thu, Aug 28, 2008 at 01:00:52AM +0900, Paul Mundt wrote: On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote: In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread David Miller
From: Paul Mundt [EMAIL PROTECTED] Date: Thu, 28 Aug 2008 09:32:13 +0900 On Wed, Aug 27, 2008 at 08:35:44PM +0300, Adrian Bunk wrote: CONFIG_DEBUG_STACKOVERFLOW should give you the same information, and if wanted with an arbitrary limit. In some cases, yes. In the

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Paul Mundt
On Wed, Aug 27, 2008 at 05:46:05PM -0700, David Miller wrote: From: Paul Mundt [EMAIL PROTECTED] Date: Thu, 28 Aug 2008 09:32:13 +0900 On Wed, Aug 27, 2008 at 08:35:44PM +0300, Adrian Bunk wrote: CONFIG_DEBUG_STACKOVERFLOW should give you the same information, and if wanted with an

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-27 Thread Greg Ungerer
Paul Mundt wrote: On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote: On Tue, Aug 26, 2008 at 05:28:37PM -0700, Linus Torvalds wrote: On Wed, 27 Aug 2008, Adrian Bunk wrote: When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32? XFS may

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 10:35:05AM -0700, Linus Torvalds wrote: On Tue, 26 Aug 2008, Rusty Russell wrote: Your workaround is very random, and that scares me. I think a huge number of CPUs needs a real solution (an actual cpumask allocator, then do something clever if we come

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 11:40:10AM -0700, Linus Torvalds wrote: On Tue, 26 Aug 2008, Adrian Bunk wrote: A debugging option (for better traces) to disallow gcc some inlining might make sense (and might even make sense for distributions to enable in their kernels), but when you go to

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 11:47:01AM -0700, Linus Torvalds wrote: On Tue, 26 Aug 2008, Adrian Bunk wrote: I added -fno-inline-functions-called-once -fno-early-inlining to KBUILD_CFLAGS, and (with gcc 4.3) that increased the size of my kernel image by 2%. Btw, did you check with

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Linus Torvalds
On Tue, 26 Aug 2008, Adrian Bunk wrote: If you think we have too many stacksize problems I'd suggest to consider removing the choice of 4k stacks on i386, sh and m68knommu instead of using -fno-inline-functions-called-once: Don't be silly. That makes the problem _worse_. We're much

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread David VomLehn
Parag Warudkar wrote: On Tue, Aug 26, 2008 at 5:04 PM, Linus Torvalds [EMAIL PROTECTED] wrote: And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more! This is something I never understood - embedded devices are not going

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 02:04:57PM -0700, Linus Torvalds wrote: On Tue, 26 Aug 2008, Adrian Bunk wrote: If you think we have too many stacksize problems I'd suggest to consider removing the choice of 4k stacks on i386, sh and m68knommu instead of using

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 04:00:33PM -0700, David VomLehn wrote: Parag Warudkar wrote: On Tue, Aug 26, 2008 at 5:04 PM, Linus Torvalds [EMAIL PROTECTED] wrote: And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more! This

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Linus Torvalds
On Tue, 26 Aug 2008, Parag Warudkar wrote: This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes) IMHO is not worth a saving now a days even in embedded world given falling memory prices. Or do I misunderstand? Well,

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Linus Torvalds
On Wed, 27 Aug 2008, Adrian Bunk wrote: We're much better off with a 1% code-size reduction than forcing big stacks on people. The 4kB stack option is also a good way of saying if it works with this, then 8kB is certainly safe. You implicitely assume both would solve the same

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Adrian Bunk
On Tue, Aug 26, 2008 at 04:51:52PM -0700, Linus Torvalds wrote: On Wed, 27 Aug 2008, Adrian Bunk wrote: We're much better off with a 1% code-size reduction than forcing big stacks on people. The 4kB stack option is also a good way of saying if it works with this, then 8kB

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Greg Ungerer
Linus Torvalds wrote: On Tue, 26 Aug 2008, Parag Warudkar wrote: This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes) IMHO is not worth a saving now a days even in embedded world given falling memory prices. Or do I

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Parag Warudkar
On Tue, Aug 26, 2008 at 7:47 PM, Linus Torvalds [EMAIL PROTECTED] wrote: If that matters, then so should the difference of 3-8 processes' kernel stack usage when you have a 4k/8k stack choice. The savings part -financial ones- are not always realizable with the way memory is

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Parag Warudkar
On Tue, Aug 26, 2008 at 8:53 PM, Greg Ungerer [EMAIL PROTECTED] wrote: I have some simple devices (network access/routers) with 8MB of RAM, at power up not really being configured to do anything running 25 processes. (Heck there is over 10 kernel processes running!). Configure some interfaces

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Linus Torvalds
On Tue, 26 Aug 2008, Parag Warudkar wrote: And although you said in your later reply that Linux x86 with 4K stacks should be more than usable - my experiences running a untainted desktop/file server with 4K stack have been always disastrous XFS or not. It _might_ work for some well defined

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Parag Warudkar
On Tue, Aug 26, 2008 at 9:31 PM, Greg Ungerer [EMAIL PROTECTED] wrote: And the pressure will still be on in _real_ products to reduce the RAM footprint as much as possible. There are exceptions but generally less is cheaper. Simple economics really. Well, sure - but the industry as a whole

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Parag Warudkar
On Tue, Aug 26, 2008 at 9:49 PM, Linus Torvalds [EMAIL PROTECTED] wrote: On Tue, 26 Aug 2008, Parag Warudkar wrote: And although you said in your later reply that Linux x86 with 4K stacks should be more than usable - my experiences running a untainted desktop/file server with 4K stack have

Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected

2008-08-26 Thread Linus Torvalds
On Tue, 26 Aug 2008, Parag Warudkar wrote: What about deep call chains? The problem with the uptake of 4K stacks seems to be that is not reliably provable that it will work under all circumstances. Umm. Neither is 8k stacks. Nobody proved anything. But yes, some subsystems have insanely