Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Srihari Vijayaraghavan
--- Christoph Lameter <[EMAIL PROTECTED]> wrote: > I finally figured out the second issue. Took some time to get that figure > out. Sorry. But now all the bug reports make sense. [...] Impressive Christoph. Indeed, this fixes my problem on latest -git (its hg equivalent :-)). Well done.

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Christoph Lameter
I finally figured out the second issue. Took some time to get that figure out. Sorry. But now all the bug reports make sense. Here is the fix SLUB: Fix NUMA / SYSFS bootstrap issue The kmem_cache_node cache is very special because it is needed for NUMA bootstrap. Under certain conditions

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Jens Axboe
On Fri, May 25 2007, Christoph Lameter wrote: > On Fri, 25 May 2007, Jens Axboe wrote: > > > The .config I sent is the one that the last boot of the machine was > > based on. The kernel is 2.6.22-rc2 with the patch moving the #ifdef from > > you, and it works for me with or without

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Christoph Lameter
On Fri, 25 May 2007, Jens Axboe wrote: > The .config I sent is the one that the last boot of the machine was > based on. The kernel is 2.6.22-rc2 with the patch moving the #ifdef from > you, and it works for me with or without CONFIG_SLUB_DEBUG set. I was not talking about your config but the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Jens Axboe
On Thu, May 24 2007, Christoph Lameter wrote: > On Thu, 24 May 2007, Jens Axboe wrote: > > > On Wed, May 23 2007, Christoph Lameter wrote: > > > On Wed, 23 May 2007, Jens Axboe wrote: > > > > > > > That works for me with the patch, .config attached. > > > > > > H... That means the .config

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Jens Axboe
On Thu, May 24 2007, Christoph Lameter wrote: On Thu, 24 May 2007, Jens Axboe wrote: On Wed, May 23 2007, Christoph Lameter wrote: On Wed, 23 May 2007, Jens Axboe wrote: That works for me with the patch, .config attached. H... That means the .config sent initially here

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Christoph Lameter
On Fri, 25 May 2007, Jens Axboe wrote: The .config I sent is the one that the last boot of the machine was based on. The kernel is 2.6.22-rc2 with the patch moving the #ifdef from you, and it works for me with or without CONFIG_SLUB_DEBUG set. I was not talking about your config but the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Jens Axboe
On Fri, May 25 2007, Christoph Lameter wrote: On Fri, 25 May 2007, Jens Axboe wrote: The .config I sent is the one that the last boot of the machine was based on. The kernel is 2.6.22-rc2 with the patch moving the #ifdef from you, and it works for me with or without CONFIG_SLUB_DEBUG set.

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Christoph Lameter
I finally figured out the second issue. Took some time to get that figure out. Sorry. But now all the bug reports make sense. Here is the fix SLUB: Fix NUMA / SYSFS bootstrap issue The kmem_cache_node cache is very special because it is needed for NUMA bootstrap. Under certain conditions

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-25 Thread Srihari Vijayaraghavan
--- Christoph Lameter [EMAIL PROTECTED] wrote: I finally figured out the second issue. Took some time to get that figure out. Sorry. But now all the bug reports make sense. [...] Impressive Christoph. Indeed, this fixes my problem on latest -git (its hg equivalent :-)). Well done. (Tested

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Srihari Vijayaraghavan
--- Christoph Lameter <[EMAIL PROTECTED]> wrote: > slabinfo -v [...] > Any corrupted objects will be reported to the syslog. (Only only could have made that sentence better.) Sorry I didn't know slabinfo -v produces nothing on stdout, stderr, rather all findings reported to syslog. Right? You

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Christoph Lameter
On Fri, 25 May 2007, Srihari Vijayaraghavan wrote: > I know slabinfo_without_v might not be good enough for you. For completeness, > I attach it here anyway. Right. "slabinfo" output is useless. You need to specify -v for it to have any effect. slabinfo -v can detect object corruptions

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Srihari Vijayaraghavan
--- Christoph Lameter <[EMAIL PROTECTED]> wrote: [...] > You need to be root to do this. Sorry. Of course, I tried it as root also. (Kernel hg commit a4c9979b8d42 is 2.6.22-rc2 + all commits available as of a few minutes ago) [EMAIL PROTECTED] ~]# whoami root [EMAIL PROTECTED] ~]# uname -a

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Christoph Lameter
On Thu, 24 May 2007, Srihari Vijayaraghavan wrote: > slabinfo -v produces this error message: > Cannot write to Acpi-Namespace/validate You need to be root to do this. Sorry. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED]

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Christoph Lameter
On Thu, 24 May 2007, Jens Axboe wrote: > On Wed, May 23 2007, Christoph Lameter wrote: > > On Wed, 23 May 2007, Jens Axboe wrote: > > > > > That works for me with the patch, .config attached. > > > > H... That means the .config sent initially here was bogus. > > ? > > Considering we're

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Srihari Vijayaraghavan
--- Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Thu, 24 May 2007, Srihari Vijayaraghavan wrote: [...] > Could you boot with slub_debug and then run Done. > slabinfo -v > > to validate all slabs? If there is anything wrong with an object then it > should show in the syslog. slabinfo

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Jens Axboe
On Wed, May 23 2007, Christoph Lameter wrote: > On Wed, 23 May 2007, Jens Axboe wrote: > > > That works for me with the patch, .config attached. > > H... That means the .config sent initially here was bogus. ? Considering we're trying to help you fix bugs in your code, you could be

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Jens Axboe
On Wed, May 23 2007, Christoph Lameter wrote: On Wed, 23 May 2007, Jens Axboe wrote: That works for me with the patch, .config attached. H... That means the .config sent initially here was bogus. ? Considering we're trying to help you fix bugs in your code, you could be considerably

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Srihari Vijayaraghavan
--- Christoph Lameter [EMAIL PROTECTED] wrote: On Thu, 24 May 2007, Srihari Vijayaraghavan wrote: [...] Could you boot with slub_debug and then run Done. slabinfo -v to validate all slabs? If there is anything wrong with an object then it should show in the syslog. slabinfo -v

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Christoph Lameter
On Thu, 24 May 2007, Jens Axboe wrote: On Wed, May 23 2007, Christoph Lameter wrote: On Wed, 23 May 2007, Jens Axboe wrote: That works for me with the patch, .config attached. H... That means the .config sent initially here was bogus. ? Considering we're trying to help you

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Christoph Lameter
On Thu, 24 May 2007, Srihari Vijayaraghavan wrote: slabinfo -v produces this error message: Cannot write to Acpi-Namespace/validate You need to be root to do this. Sorry. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED]

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Srihari Vijayaraghavan
--- Christoph Lameter [EMAIL PROTECTED] wrote: [...] You need to be root to do this. Sorry. Of course, I tried it as root also. (Kernel hg commit a4c9979b8d42 is 2.6.22-rc2 + all commits available as of a few minutes ago) [EMAIL PROTECTED] ~]# whoami root [EMAIL PROTECTED] ~]# uname -a Linux

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Christoph Lameter
On Fri, 25 May 2007, Srihari Vijayaraghavan wrote: I know slabinfo_without_v might not be good enough for you. For completeness, I attach it here anyway. Right. slabinfo output is useless. You need to specify -v for it to have any effect. slabinfo -v can detect object corruptions without

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-24 Thread Srihari Vijayaraghavan
--- Christoph Lameter [EMAIL PROTECTED] wrote: slabinfo -v [...] Any corrupted objects will be reported to the syslog. (Only only could have made that sentence better.) Sorry I didn't know slabinfo -v produces nothing on stdout, stderr, rather all findings reported to syslog. Right? You don't

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Thu, 24 May 2007, Srihari Vijayaraghavan wrote: > I'm stunned. Honestly, I have no possible explanations for this behaviour. Do > you? I need more time to work out (until otherwise you might know a reason). Hmmm... Bad. We have conflicting reports and no clear way to trigger the bug. This

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Srihari Vijayaraghavan
--- Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: > > > I'm personally very happy that slub works stably without slub debug > options, > > because that's what I'd run in a production env. Thanks to your patch, > slub is > > quite stable without

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: > I'm personally very happy that slub works stably without slub debug options, > because that's what I'd run in a production env. Thanks to your patch, slub is > quite stable without the slub debug for me :-)). But it'd to nice to have a >

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: > > and then try to boot without slub_debug. > > I guess you mean with CONFIG_SLUB_CONFIG=y? If so, I built another kernel with > CONFIG_SLUB_DEBUG=y (plus all of the above) & tested it. It panics by default, > but with slub_nomerge it works

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Wed, 23 May 2007, Jens Axboe wrote: > That works for me with the patch, .config attached. H... That means the .config sent initially here was bogus. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Srihari Vijayaraghavan
--- Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: [...] > Yup. compile with > > CONFIG_NUMA > > CONFIG_LOCKDEP > > CONFIG_DEBUG_LOCK_ALLOCS (All the tests in this email was conducted on top of your patch) Yup done that. The resulting kernel

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Jens Axboe
On Tue, May 22 2007, Christoph Lameter wrote: > On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: > > > (If you want me to test it with other slub or kernel debug options please > > let > > me know. It just takes a lot of time to eliminate the variables, if there > > are > > problems.) > >

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Jens Axboe
On Tue, May 22 2007, Christoph Lameter wrote: On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: (If you want me to test it with other slub or kernel debug options please let me know. It just takes a lot of time to eliminate the variables, if there are problems.) Yup. compile

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Srihari Vijayaraghavan
--- Christoph Lameter [EMAIL PROTECTED] wrote: On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: [...] Yup. compile with CONFIG_NUMA CONFIG_LOCKDEP CONFIG_DEBUG_LOCK_ALLOCS (All the tests in this email was conducted on top of your patch) Yup done that. The resulting kernel (without

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Wed, 23 May 2007, Jens Axboe wrote: That works for me with the patch, .config attached. H... That means the .config sent initially here was bogus. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: and then try to boot without slub_debug. I guess you mean with CONFIG_SLUB_CONFIG=y? If so, I built another kernel with CONFIG_SLUB_DEBUG=y (plus all of the above) tested it. It panics by default, but with slub_nomerge it works just fine

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: I'm personally very happy that slub works stably without slub debug options, because that's what I'd run in a production env. Thanks to your patch, slub is quite stable without the slub debug for me :-)). But it'd to nice to have a working

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Srihari Vijayaraghavan
--- Christoph Lameter [EMAIL PROTECTED] wrote: On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: I'm personally very happy that slub works stably without slub debug options, because that's what I'd run in a production env. Thanks to your patch, slub is quite stable without the slub

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-23 Thread Christoph Lameter
On Thu, 24 May 2007, Srihari Vijayaraghavan wrote: I'm stunned. Honestly, I have no possible explanations for this behaviour. Do you? I need more time to work out (until otherwise you might know a reason). Hmmm... Bad. We have conflicting reports and no clear way to trigger the bug. This may

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Christoph Lameter
On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: > (If you want me to test it with other slub or kernel debug options please let > me know. It just takes a lot of time to eliminate the variables, if there are > problems.) Yup. compile with CONFIG_NUMA CONFIG_LOCKDEP CONFIG_DEBUG_LOCK_ALLOCS

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
--- Hugh Dickins <[EMAIL PROTECTED]> wrote: > On Tue, 22 May 2007, Srihari Vijayaraghavan wrote: [...] > > Compiled slub with SMP & CONFIG_PROVE_LOCKING. No luck. It still hangs > solid [...] > You've made no mention of trying the patch I sent yesterday, or better, > the patch Christoph

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Jens Axboe
On Tue, May 22 2007, Christoph Lameter wrote: > On Tue, 22 May 2007, Jens Axboe wrote: > > > > You've made no mention of trying the patch I sent yesterday, or better, > > > the patch Christoph replied with to replace it. Please clarify whether > > > you're getting the above after applying one of

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Christoph Lameter
On Tue, 22 May 2007, Jens Axboe wrote: > > You've made no mention of trying the patch I sent yesterday, or better, > > the patch Christoph replied with to replace it. Please clarify whether > > you're getting the above after applying one of those patches - thanks. > > Christophs patch works for

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Jens Axboe
On Tue, May 22 2007, Hugh Dickins wrote: > On Tue, 22 May 2007, Srihari Vijayaraghavan wrote: > > --- Ingo Molnar <[EMAIL PROTECTED]> wrote: > > > * Srihari Vijayaraghavan <[EMAIL PROTECTED]> wrote: > > > > Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's > > > > quite stable

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Hugh Dickins
On Tue, 22 May 2007, Srihari Vijayaraghavan wrote: > --- Ingo Molnar <[EMAIL PROTECTED]> wrote: > > * Srihari Vijayaraghavan <[EMAIL PROTECTED]> wrote: > > > Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's > > > quite stable actually (having completed a dozen kernel compile > >

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
--- Ingo Molnar <[EMAIL PROTECTED]> wrote: [...] > yes - PROVE_LOCKING reactivates spinlocks even on UP. At least this > suggests that you'd have gotten the hang even with maxcpus=1 - i.e. the > spinlock corruption is not caused by some genuine SMP race. You're right on the mark there: even

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Ingo Molnar
* Srihari Vijayaraghavan <[EMAIL PROTECTED]> wrote: > Compiled slub with SMP & CONFIG_PROVE_LOCKING. No luck. It still hangs > solid after the second spinlock lockup call trace. hm. This suggests that the spinlock got corrupted - otherwise lockdep would have complained about the lockup before

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
--- Ingo Molnar <[EMAIL PROTECTED]> wrote: > * Srihari Vijayaraghavan <[EMAIL PROTECTED]> wrote: > > Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's > > quite stable actually (having completed a dozen kernel compile > > sessions so far). [...] > could you enable

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Ingo Molnar
* Srihari Vijayaraghavan <[EMAIL PROTECTED]> wrote: > Just another data point: with the flick of CONFIG_SMP, I'm in control > of the hangs/crashes ;-). > > Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's > quite stable actually (having completed a dozen kernel compile >

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
Just another data point: with the flick of CONFIG_SMP, I'm in control of the hangs/crashes ;-). Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's quite stable actually (having completed a dozen kernel compile sessions so far). I suspected this after seeing spinlock issues on both

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Jens Axboe
On Mon, May 21 2007, Christoph Lameter wrote: > Ok booting SLUB without "slub_debug" and having > > CONFIG_DEBUG_LOCK_ALLOC > CONFIG_LOCKDEP > > will trigger the problem. > > So I guess the issue is that lockdep does a slab alloc while we get the > slab lock during slab alloc? If I have

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Christoph Lameter
On Tue, 22 May 2007, Jens Axboe wrote: You've made no mention of trying the patch I sent yesterday, or better, the patch Christoph replied with to replace it. Please clarify whether you're getting the above after applying one of those patches - thanks. Christophs patch works for me! So

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Jens Axboe
On Tue, May 22 2007, Christoph Lameter wrote: On Tue, 22 May 2007, Jens Axboe wrote: You've made no mention of trying the patch I sent yesterday, or better, the patch Christoph replied with to replace it. Please clarify whether you're getting the above after applying one of those

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
--- Hugh Dickins [EMAIL PROTECTED] wrote: On Tue, 22 May 2007, Srihari Vijayaraghavan wrote: [...] Compiled slub with SMP CONFIG_PROVE_LOCKING. No luck. It still hangs solid [...] You've made no mention of trying the patch I sent yesterday, or better, the patch Christoph replied with to

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Christoph Lameter
On Wed, 23 May 2007, Srihari Vijayaraghavan wrote: (If you want me to test it with other slub or kernel debug options please let me know. It just takes a lot of time to eliminate the variables, if there are problems.) Yup. compile with CONFIG_NUMA CONFIG_LOCKDEP CONFIG_DEBUG_LOCK_ALLOCS

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Jens Axboe
On Mon, May 21 2007, Christoph Lameter wrote: Ok booting SLUB without slub_debug and having CONFIG_DEBUG_LOCK_ALLOC CONFIG_LOCKDEP will trigger the problem. So I guess the issue is that lockdep does a slab alloc while we get the slab lock during slab alloc? If I have

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
Just another data point: with the flick of CONFIG_SMP, I'm in control of the hangs/crashes ;-). Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's quite stable actually (having completed a dozen kernel compile sessions so far). I suspected this after seeing spinlock issues on both

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Ingo Molnar
* Srihari Vijayaraghavan [EMAIL PROTECTED] wrote: Just another data point: with the flick of CONFIG_SMP, I'm in control of the hangs/crashes ;-). Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's quite stable actually (having completed a dozen kernel compile sessions so

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
--- Ingo Molnar [EMAIL PROTECTED] wrote: * Srihari Vijayaraghavan [EMAIL PROTECTED] wrote: Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's quite stable actually (having completed a dozen kernel compile sessions so far). [...] could you enable CONFIG_PROVE_LOCKING -

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Ingo Molnar
* Srihari Vijayaraghavan [EMAIL PROTECTED] wrote: Compiled slub with SMP CONFIG_PROVE_LOCKING. No luck. It still hangs solid after the second spinlock lockup call trace. hm. This suggests that the spinlock got corrupted - otherwise lockdep would have complained about the lockup before the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Srihari Vijayaraghavan
--- Ingo Molnar [EMAIL PROTECTED] wrote: [...] yes - PROVE_LOCKING reactivates spinlocks even on UP. At least this suggests that you'd have gotten the hang even with maxcpus=1 - i.e. the spinlock corruption is not caused by some genuine SMP race. You're right on the mark there: even with

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Hugh Dickins
On Tue, 22 May 2007, Srihari Vijayaraghavan wrote: --- Ingo Molnar [EMAIL PROTECTED] wrote: * Srihari Vijayaraghavan [EMAIL PROTECTED] wrote: Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's quite stable actually (having completed a dozen kernel compile sessions so

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-22 Thread Jens Axboe
On Tue, May 22 2007, Hugh Dickins wrote: On Tue, 22 May 2007, Srihari Vijayaraghavan wrote: --- Ingo Molnar [EMAIL PROTECTED] wrote: * Srihari Vijayaraghavan [EMAIL PROTECTED] wrote: Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's quite stable actually (having

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Christoph Lameter wrote: > So I guess the issue is that lockdep does a slab alloc while we get the > slab lock during slab alloc? Lockdep is not available on IA64 where I would be able to figure it out using a simulator. x86_64 early printk support seems to be broken? No

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
Ok booting SLUB without "slub_debug" and having CONFIG_DEBUG_LOCK_ALLOC CONFIG_LOCKDEP will trigger the problem. So I guess the issue is that lockdep does a slab alloc while we get the slab lock during slab alloc? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Hugh Dickins wrote: > > the problem. And the earlier case that you reported was a version of SLUB > > that did not have the capability to switch off SLUB_DEBUG. > > Are you sure? If CONFIG_EMBEDDED=y then -rc1 and -rc2 give you the > choice to turn it off. Right. That

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: > On Mon, 21 May 2007, Jens Axboe wrote: > > > I can test whatever you want tomorrow morning, it was 100% repeatable > > here. So which one, your patch or Hughs? > > This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If > you did

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Jens Axboe
On Mon, May 21 2007, Christoph Lameter wrote: > On Mon, 21 May 2007, Jens Axboe wrote: > > > I can test whatever you want tomorrow morning, it was 100% repeatable > > here. So which one, your patch or Hughs? > > This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If > you did

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Jens Axboe
On Mon, May 21 2007, Hugh Dickins wrote: > On Mon, 21 May 2007, Jens Axboe wrote: > > > > I can test whatever you want tomorrow morning, it was 100% repeatable > > here. So which one, your patch or Hughs? > > Great, thanks: Christoph's please - I'm sure he'll agree! Alright, will do! -- Jens

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Jens Axboe wrote: > I can test whatever you want tomorrow morning, it was 100% repeatable > here. So which one, your patch or Hughs? This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If you did not run without CONFIG_SLUB_DEBUG then you were not affected

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: > On Mon, 21 May 2007, Hugh Dickins wrote: > > On Mon, 21 May 2007, Christoph Lameter wrote: > > > On Mon, 21 May 2007, Hugh Dickins wrote: > > > > SLUB Debug: Fix object size calculation > > > > > > The object size calculation is wrong if

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Jens Axboe wrote: > > I can test whatever you want tomorrow morning, it was 100% repeatable > here. So which one, your patch or Hughs? Great, thanks: Christoph's please - I'm sure he'll agree! Hugh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Jens Axboe
On Mon, May 21 2007, Christoph Lameter wrote: > On Mon, 21 May 2007, Hugh Dickins wrote: > > > On Mon, 21 May 2007, Christoph Lameter wrote: > > > On Mon, 21 May 2007, Hugh Dickins wrote: > > > > > > > Yes, sounded the same to me too: I couldn't reproduce it or see anything > > > > wrong in the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Hugh Dickins wrote: > On Mon, 21 May 2007, Christoph Lameter wrote: > > On Mon, 21 May 2007, Hugh Dickins wrote: > > > > > Yes, sounded the same to me too: I couldn't reproduce it or see anything > > > wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: > On Mon, 21 May 2007, Hugh Dickins wrote: > > > Yes, sounded the same to me too: I couldn't reproduce it or see anything > > wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB > > off has helped a lot: I was then able to

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Hugh Dickins wrote: > Yes, sounded the same to me too: I couldn't reproduce it or see anything > wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB > off has helped a lot: I was then able to reproduce it on my x86_64, and > after a lot of staring at the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: > On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: > > > With no CONFIG_SLUB_DEBUG, things have slightly improved. No more panic. > > Good. > > Serial console is working. Good. But there is another problem: > > > Freeing unused kernel memory: 308k

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: > With no CONFIG_SLUB_DEBUG, things have slightly improved. No more panic. Good. > Serial console is working. Good. But there is another problem: Well this indicates that something destroys the sysfs pointer structure that SLUB is using. Could

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: With no CONFIG_SLUB_DEBUG, things have slightly improved. No more panic. Good. Serial console is working. Good. But there is another problem: Well this indicates that something destroys the sysfs pointer structure that SLUB is using. Could

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: With no CONFIG_SLUB_DEBUG, things have slightly improved. No more panic. Good. Serial console is working. Good. But there is another problem: Freeing unused kernel memory: 308k freed

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Hugh Dickins wrote: Yes, sounded the same to me too: I couldn't reproduce it or see anything wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB off has helped a lot: I was then able to reproduce it on my x86_64, and after a lot of staring at the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Hugh Dickins wrote: Yes, sounded the same to me too: I couldn't reproduce it or see anything wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB off has helped a lot: I was then able to reproduce it on

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Hugh Dickins wrote: On Mon, 21 May 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Hugh Dickins wrote: Yes, sounded the same to me too: I couldn't reproduce it or see anything wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB off has

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Jens Axboe
On Mon, May 21 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Hugh Dickins wrote: On Mon, 21 May 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Hugh Dickins wrote: Yes, sounded the same to me too: I couldn't reproduce it or see anything wrong in the code back then.

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Hugh Dickins wrote: On Mon, 21 May 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Hugh Dickins wrote: SLUB Debug: Fix object size calculation The object size calculation is wrong if !CONFIG_SLUB_DEBUG because

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Jens Axboe wrote: I can test whatever you want tomorrow morning, it was 100% repeatable here. So which one, your patch or Hughs? Great, thanks: Christoph's please - I'm sure he'll agree! Hugh - To unsubscribe from this list: send the line unsubscribe linux-kernel in the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Jens Axboe
On Mon, May 21 2007, Hugh Dickins wrote: On Mon, 21 May 2007, Jens Axboe wrote: I can test whatever you want tomorrow morning, it was 100% repeatable here. So which one, your patch or Hughs? Great, thanks: Christoph's please - I'm sure he'll agree! Alright, will do! -- Jens Axboe -

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Jens Axboe wrote: I can test whatever you want tomorrow morning, it was 100% repeatable here. So which one, your patch or Hughs? This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If you did not run without CONFIG_SLUB_DEBUG then you were not affected by

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Jens Axboe
On Mon, May 21 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Jens Axboe wrote: I can test whatever you want tomorrow morning, it was 100% repeatable here. So which one, your patch or Hughs? This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If you did not run

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Hugh Dickins
On Mon, 21 May 2007, Christoph Lameter wrote: On Mon, 21 May 2007, Jens Axboe wrote: I can test whatever you want tomorrow morning, it was 100% repeatable here. So which one, your patch or Hughs? This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If you did not run

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Hugh Dickins wrote: the problem. And the earlier case that you reported was a version of SLUB that did not have the capability to switch off SLUB_DEBUG. Are you sure? If CONFIG_EMBEDDED=y then -rc1 and -rc2 give you the choice to turn it off. Right. That went in

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
Ok booting SLUB without slub_debug and having CONFIG_DEBUG_LOCK_ALLOC CONFIG_LOCKDEP will trigger the problem. So I guess the issue is that lockdep does a slab alloc while we get the slab lock during slab alloc? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-21 Thread Christoph Lameter
On Mon, 21 May 2007, Christoph Lameter wrote: So I guess the issue is that lockdep does a slab alloc while we get the slab lock during slab alloc? Lockdep is not available on IA64 where I would be able to figure it out using a simulator. x86_64 early printk support seems to be broken? No

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Srihari Vijayaraghavan
--- Satyam Sharma <[EMAIL PROTECTED]> wrote: > On 5/20/07, Srihari Vijayaraghavan <[EMAIL PROTECTED]> > wrote: > > --- Christoph Lameter <[EMAIL PROTECTED]> wrote: > > > On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: > > > > > > > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Srihari Vijayaraghavan
--- Srihari Vijayaraghavan <[EMAIL PROTECTED]> wrote: > --- Christoph Lameter <[EMAIL PROTECTED]> wrote: > > On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: > > > > > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80 > > > RIP [...] slab_sysfs_init+0x49/0x98 > > > RSP [...] > >

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Satyam Sharma
On 5/20/07, Srihari Vijayaraghavan <[EMAIL PROTECTED]> wrote: --- Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: > > > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80 > > RIP [...] slab_sysfs_init+0x49/0x98 > > RSP [...] > >

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Srihari Vijayaraghavan
--- Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: > > > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80 > > RIP [...] slab_sysfs_init+0x49/0x98 > > RSP [...] > > kernel panic - not syncing: Attempted to kill init! > > sysfs?

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Christoph Lameter
On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80 > RIP [...] slab_sysfs_init+0x49/0x98 > RSP [...] > kernel panic - not syncing: Attempted to kill init! sysfs? If you build without CONFIG_SLUB_DEBUG then SLUB will not use

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Christoph Lameter
On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80 RIP [...] slab_sysfs_init+0x49/0x98 RSP [...] kernel panic - not syncing: Attempted to kill init! sysfs? If you build without CONFIG_SLUB_DEBUG then SLUB will not use sysfs.

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Srihari Vijayaraghavan
--- Christoph Lameter [EMAIL PROTECTED] wrote: On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80 RIP [...] slab_sysfs_init+0x49/0x98 RSP [...] kernel panic - not syncing: Attempted to kill init! sysfs? If you build

Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

2007-05-20 Thread Satyam Sharma
On 5/20/07, Srihari Vijayaraghavan [EMAIL PROTECTED] wrote: --- Christoph Lameter [EMAIL PROTECTED] wrote: On Sun, 20 May 2007, Srihari Vijayaraghavan wrote: Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80 RIP [...] slab_sysfs_init+0x49/0x98 RSP [...] kernel panic -

  1   2   >