Re: spin lock sched lock held for 5 seconds
On 08-Aug-2003 Lars Eggert wrote:
John Baldwin wrote:
On 01-Aug-2003 Lars Eggert wrote:
Hi,
got the following panic overnight running with all debugging options on
(WITNESS, MUTEX_DEBUG, DIAGNOSTIC, INVARIANTS; WITNESS_SKIPSPIN off):
panic: spin lock sched lock held by 0xc658e130 for 5 seconds
cpuid = 0; lapic.id =
Stack backtrace:
backtrace(c031d030,0,c031c4c5,df0dab8c,100) at backtrace+0x17
panic(c031c4c5,c031c62e,c658e130,c036f160,18b) at panic+0x13d
_mtx_lock_spin(c036f160,2,c031a229,18b,c21b2ab0) at _mtx_lock_spin+0x83
_mtx_lock_spin_flags(c036f160,2,c031a229,18b,df0dac0c) at
_mtx_lock_spin_flags+0xb9
statclock(df0dac00,df0dac44,c02d8a9c,0,c2198d00) at statclock+0x39
rtcintr(0) at rtcintr+0x4f
Xfastunpend8(df0dacb8,c02d1f05,8,608,c0372e60) at Xfastunpend8+0x1c
call_fast_unpend(8,608,c0372e60,ffc00034,0) at call_fast_unpend+0xd
i386_unpend(c036f160,c21b0790,df0dacd0,c01b1ae0,df0dacec) at
i386_unpend+0x8d
cpu_unpend(df0dacec,c01a2534,c036f160,1,c031c2e2) at cpu_unpend+0x2d
critical_exit(c036f160,1,c031c2e2,1bc,1) at critical_exit+0x2d
_mtx_unlock_spin_flags(c036f160,0,c031ac9f,7c,c21b2ab0) at
_mtx_unlock_spin_flags+0xbb
idle_proc(0,df0dad48,c031ab89,312,c21b2ab0) at idle_proc+0xb0
fork_exit(c0199972,0,df0dad48) at fork_exit+0xc3
fork_trampoline() at fork_trampoline+0x1a
--- trap 0x1, eip = 0, esp = 0xdf0dad7c, ebp = 0 ---
Debugger(panic)
timeout stopping cpus
Stopped at Debugger+0x4f: xchgl %ebx,in_Debugger.0
db
The machine is still in ddb, let me know if I can provide additional info.
Try updating to a more recent current. I recently added some extra
debugging here that will attempt to better show who owns the lock and
where it was acquired.
Same panic string, but a different call chain:
spin lock sched lock held by 0xc21b2d10 for 5 seconds
exclusive spin mutex sched lock r = 0 (0xc036efc0) locked @
/usr/src/sys/kern/kern_mutex.c:512
panic: spin lock held too long
cpuid = 3; lapic.id = 0300
Stack backtrace:
backtrace(c031ce60,300,c031c305,df0d0c30,100) at backtrace+0x17
panic(c031c305,c21b2d10,c21b2d10,c036efc0,bc) at panic+0x13d
_mtx_lock_spin(c036efc0,2,c031a037,bc,c21b2720) at _mtx_lock_spin+0xb4
_mtx_lock_spin_flags(c036efc0,2,c031a037,bc,c21b2720) at
_mtx_lock_spin_flags+0xb9
hardclock_process(df0d0ca0,df0d0ce4,c02d7062,0,c0390018) at
hardclock_process+0x3c
forwarded_hardclock(0) at forwarded_hardclock+0x11
Xhardclock(df0d0d10,c01998d7,c036f000,2,c031aaad) at Xhardclock+0x52
cpu_idle(c036f000,2,c031aaad,5f,c21b2720) at cpu_idle+0x8
idle_proc(0,df0d0d48,c031a997,30e,0) at idle_proc+0x45
fork_exit(c0199892,0,df0d0d48) at fork_exit+0xc3
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xdf0d0d7c, ebp = 0 ---
Debugger(panic)
timeout stopping cpus
Stopped at Debugger+0x4f: xchgl %ebx,in_Debugger.0
db
Machine is still in ddb, in case you'd like me to poke around some more.
do 'show pcpu' for each CPU. If this is a dual machine, then it might
be rather easy to figure out which CPU you are on ('show pcpu') then do
a 'show pcpu x' of the other CPU, find the it's curthread, and do a
'trace XXX' on it's PID to see where it is at. That might not work but
it's worth a shot I guess. Also, you can try 'option ADAPTIVE_MUTEXES'
and see if that helps.
--
John Baldwin [EMAIL PROTECTED]http://www.FreeBSD.org/~jhb/
Power Users Use the Power to Serve! - http://www.FreeBSD.org/
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: spin lock sched lock held for 5 seconds
John Baldwin wrote: On 01-Aug-2003 Lars Eggert wrote: Hi, got the following panic overnight running with all debugging options on (WITNESS, MUTEX_DEBUG, DIAGNOSTIC, INVARIANTS; WITNESS_SKIPSPIN off): panic: spin lock sched lock held by 0xc658e130 for 5 seconds cpuid = 0; lapic.id = Stack backtrace: backtrace(c031d030,0,c031c4c5,df0dab8c,100) at backtrace+0x17 panic(c031c4c5,c031c62e,c658e130,c036f160,18b) at panic+0x13d _mtx_lock_spin(c036f160,2,c031a229,18b,c21b2ab0) at _mtx_lock_spin+0x83 _mtx_lock_spin_flags(c036f160,2,c031a229,18b,df0dac0c) at _mtx_lock_spin_flags+0xb9 statclock(df0dac00,df0dac44,c02d8a9c,0,c2198d00) at statclock+0x39 rtcintr(0) at rtcintr+0x4f Xfastunpend8(df0dacb8,c02d1f05,8,608,c0372e60) at Xfastunpend8+0x1c call_fast_unpend(8,608,c0372e60,ffc00034,0) at call_fast_unpend+0xd i386_unpend(c036f160,c21b0790,df0dacd0,c01b1ae0,df0dacec) at i386_unpend+0x8d cpu_unpend(df0dacec,c01a2534,c036f160,1,c031c2e2) at cpu_unpend+0x2d critical_exit(c036f160,1,c031c2e2,1bc,1) at critical_exit+0x2d _mtx_unlock_spin_flags(c036f160,0,c031ac9f,7c,c21b2ab0) at _mtx_unlock_spin_flags+0xbb idle_proc(0,df0dad48,c031ab89,312,c21b2ab0) at idle_proc+0xb0 fork_exit(c0199972,0,df0dad48) at fork_exit+0xc3 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xdf0dad7c, ebp = 0 --- Debugger(panic) timeout stopping cpus Stopped at Debugger+0x4f: xchgl %ebx,in_Debugger.0 db The machine is still in ddb, let me know if I can provide additional info. Try updating to a more recent current. I recently added some extra debugging here that will attempt to better show who owns the lock and where it was acquired. Same panic string, but a different call chain: spin lock sched lock held by 0xc21b2d10 for 5 seconds exclusive spin mutex sched lock r = 0 (0xc036efc0) locked @ /usr/src/sys/kern/kern_mutex.c:512 panic: spin lock held too long cpuid = 3; lapic.id = 0300 Stack backtrace: backtrace(c031ce60,300,c031c305,df0d0c30,100) at backtrace+0x17 panic(c031c305,c21b2d10,c21b2d10,c036efc0,bc) at panic+0x13d _mtx_lock_spin(c036efc0,2,c031a037,bc,c21b2720) at _mtx_lock_spin+0xb4 _mtx_lock_spin_flags(c036efc0,2,c031a037,bc,c21b2720) at _mtx_lock_spin_flags+0xb9 hardclock_process(df0d0ca0,df0d0ce4,c02d7062,0,c0390018) at hardclock_process+0x3c forwarded_hardclock(0) at forwarded_hardclock+0x11 Xhardclock(df0d0d10,c01998d7,c036f000,2,c031aaad) at Xhardclock+0x52 cpu_idle(c036f000,2,c031aaad,5f,c21b2720) at cpu_idle+0x8 idle_proc(0,df0d0d48,c031a997,30e,0) at idle_proc+0x45 fork_exit(c0199892,0,df0d0d48) at fork_exit+0xc3 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xdf0d0d7c, ebp = 0 --- Debugger(panic) timeout stopping cpus Stopped at Debugger+0x4f: xchgl %ebx,in_Debugger.0 db Machine is still in ddb, in case you'd like me to poke around some more. Lars -- Lars Eggert [EMAIL PROTECTED] USC Information Sciences Institute smime.p7s Description: S/MIME Cryptographic Signature
RE: spin lock sched lock held for 5 seconds
On 01-Aug-2003 Lars Eggert wrote: Hi, got the following panic overnight running with all debugging options on (WITNESS, MUTEX_DEBUG, DIAGNOSTIC, INVARIANTS; WITNESS_SKIPSPIN off): panic: spin lock sched lock held by 0xc658e130 for 5 seconds cpuid = 0; lapic.id = Stack backtrace: backtrace(c031d030,0,c031c4c5,df0dab8c,100) at backtrace+0x17 panic(c031c4c5,c031c62e,c658e130,c036f160,18b) at panic+0x13d _mtx_lock_spin(c036f160,2,c031a229,18b,c21b2ab0) at _mtx_lock_spin+0x83 _mtx_lock_spin_flags(c036f160,2,c031a229,18b,df0dac0c) at _mtx_lock_spin_flags+0xb9 statclock(df0dac00,df0dac44,c02d8a9c,0,c2198d00) at statclock+0x39 rtcintr(0) at rtcintr+0x4f Xfastunpend8(df0dacb8,c02d1f05,8,608,c0372e60) at Xfastunpend8+0x1c call_fast_unpend(8,608,c0372e60,ffc00034,0) at call_fast_unpend+0xd i386_unpend(c036f160,c21b0790,df0dacd0,c01b1ae0,df0dacec) at i386_unpend+0x8d cpu_unpend(df0dacec,c01a2534,c036f160,1,c031c2e2) at cpu_unpend+0x2d critical_exit(c036f160,1,c031c2e2,1bc,1) at critical_exit+0x2d _mtx_unlock_spin_flags(c036f160,0,c031ac9f,7c,c21b2ab0) at _mtx_unlock_spin_flags+0xbb idle_proc(0,df0dad48,c031ab89,312,c21b2ab0) at idle_proc+0xb0 fork_exit(c0199972,0,df0dad48) at fork_exit+0xc3 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xdf0dad7c, ebp = 0 --- Debugger(panic) timeout stopping cpus Stopped at Debugger+0x4f: xchgl %ebx,in_Debugger.0 db The machine is still in ddb, let me know if I can provide additional info. Try updating to a more recent current. I recently added some extra debugging here that will attempt to better show who owns the lock and where it was acquired. -- John Baldwin [EMAIL PROTECTED]http://www.FreeBSD.org/~jhb/ Power Users Use the Power to Serve! - http://www.FreeBSD.org/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
