Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-10-15 Thread Anton Shterenlikht
From davide.itali...@gmail.com Fri Oct 11 15:39:49 2013

If you're not able to get a full dump, a textdump would be enough.
In your DDB scripts just remove the 'call doadump' step and you should
be done, unless I'm missing something.
Please adjust the script as well to include all the informations
requested as mentioned in my previous link, e.g. 'show lockedvnods' is
not mentioned on the example section of textdump(4) manpage but it
could be useful to ease debugging.

It seems call doadump is always needed.
At least it is included in textdump(4) examples.

Also, this tutorial incluldes it:

http://www.etinc.com/122/Using-FreeBSD-Text-Dumps

I think I still haven't got textdump right, because
instead of

savecore: writing core to textdump.tar.0

I see:

savecore: writing core to /var/crash/vmcore.9

Thanks

Anton
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-10-14 Thread Anton Shterenlikht
From davide.itali...@gmail.com Fri Oct 11 15:39:49 2013

If you're not able to get a full dump, a textdump would be enough.
In your DDB scripts just remove the 'call doadump' step and you should
be done, unless I'm missing something.
Please adjust the script as well to include all the informations
requested as mentioned in my previous link, e.g. 'show lockedvnods' is
not mentioned on the example section of textdump(4) manpage but it
could be useful to ease debugging.

I still haven't got it right, but I now collected manually
some (hopefully) useful info (see below).

- here's my /etc/ddb.conf:

script lockinfo=show locks; show alllocks; show lockedvnods
script kdb.enter.panic=textdump set; capture on; run lockinfo; show locks; show 
alllocks; show lockedvnods; show pcpu; bt; ps; alltrace; capture off; reset
script kdb.enter.witness=run lockinfo

Somehow when I get into a debugger, I can only
see lockinfo. Why?

- I've removed call doadump, as you see above.
  Still the dump is being written. I've in /etc/rc.conf:

dumpdev=/dev/da4p1
dumpdir=/var/crash
ddb_enable=YES

  Shall I remove dumpdev completely?
  And leave only dumpdir defined?

Thanks

Anton



db textdump set
textdump set
db capture on
db run lockinfo
db:0:lockinfo show locks
db:0:locks  show alllocks
db:0:alllocks  show lockedvnods
Locked vnodes

0xe000123799d0: tag devfs, type VCHR
usecount 1, writecount 0, refcount 408 mountedhere 0xe000126a4c00
flags (VI_ACTIVE)
v_object 0xe00012788100 ref 0 pages 21330
lock type devfs: EXCL by thread 0xe0001268e000 (pid 21, syncer, tid 
100062)
dev da5p1

0xe000127d5448: tag ufs, type VDIR
usecount 1, writecount 0, refcount 3 mountedhere 0
flags (VI_ACTIVE)
v_object 0xe0001277bc00 ref 0 pages 1
lock type ufs: EXCL by thread 0xe000127b5680 (pid 908, ntpd, tid 100080)
ino 2889216, on dev da3p1

0xe002203c0ce8: tag ufs, type VREG
usecount 1, writecount 1, refcount 16 mountedhere 0
flags (VI_ACTIVE)
v_object 0xe00223037700 ref 0 pages 56
lock type ufs: EXCL by thread 0xe0001557e480 (pid 45384, pkg-static, 
tid 100133)
ino 5780554, on dev da5p1
db 
db show pcpu
cpuid= 0
dynamic pcpu = 0x40040040ff13ca80
curthread= 0xe00011a26d80: pid 0 deadlkres
curpcb   = 0
fpcurthread  = none
idlethread   = 0xe00011178900: tid 13 idle: cpu0
MD: vhpt   = 0xe040ff00
MD: lid= 0
MD: clock  = 0x214563ccc206
MD: clock_mode = 2
MD: clock_load = 0
MD: stats  = 0x9ffc010ff130
MD: pmap   = 0x9ffc00eee760
spin locks held:
db 
db bt
Tracing pid 0 tid 100053 td 0xe00011a26d80
kdb_enter(0x9ffc00dc2fd0, 0x9ffc00dc2fd0, 0x9ffc0058e6b0, 0x40c) at 
kdb_enter+0x92
vpanic(0x9ffc00db8a18, 0xa0009dca7518) at vpanic+0x2b0
panic(0x9ffc00db8a18, 0x9ffc00db8c70, 0xe0001557fb00, 0xdbd38) at 
panic+0x80
deadlkres(0xdbd38, 0xe0001557fb00, 0x9ffc00dbb648, 0x9ffc00db89a8) 
at deadlkres+0x420
fork_exit(0x9ffc00e0fca0, 0x0, 0xa0009dca7550) at fork_exit+0x120
enter_userland() at enter_userland
db 
db ps
  pid  ppid  pgrp   uid   state   wmesg wchancmd
47482 47176  1092 0  LV+J   *pmap pv  0xe00012402540 sh
47476 35326  1092 0  D+  vm map ( 0xe000110adac0 sh
47475 35326  1092 0  S+  piperd   0xe00222b2f5a8 awk
47474 35326  1092 0  L+ *pmap pv  0xe00012402540 awk
47335 47329  1092 0  LL+J   *pmap pv  0xe00012402540 cc1
47329 34696  1092 0  SW+Jwait 0xe002233c3720 xgcc
47176 45981  1092 0  D+J ppwait   0xe000156af6d0 sh
46683 46681  1092 0  S+J piperd   0xe000127e3340 as
46682 46681  1092 0  LL+J   *pmap pv  0xe00012402540 cc1plus
46681 33402  1092 0  SW+Jwait 0xe000154c4000 g++
45981 44152  1092 0  SW+Jwait 0xe002233c3280 pkg-static
45384 45009  1092 0  L+J*vm page  0xe0001eff4f00 pkg-static
45009 44866  1092 0  SW+Jwait 0xe002233c2940 sh
44866 38577  1092 0  SW+Jwait 0xe0001547cde0 make
44152 43869  1092 0  SW+Jwait 0xe0001f2dcde0 sh
43869 37427  1092 0  SW+Jwait 0xe0001549d280 make
39289 38577  1092 0  S+  piperd   0xe0001556d0d8 tee
38577  1092  1092 0  SW+ wait 0xe000154f1280 sh
37813 37427  1092 0  S+  piperd   0xe00222b2f810 tee
37427  1092  1092 0  SW+ wait 0xe00012b3c4a0 sh
34696 89091  1092 0  SW+Jwait 0xe0001556e000 gmake
33402 33399  1092 0  SW+Jwait 0xe0001280e4a0 make
33399 9  1092 0  SW+Jwait 0xe000129e1720 sh
9 7  1092 0  SW+Jwait 0xe0001556e4a0 make
7 88092  1092 0  SW+ wait 0xe0001f2dd720 sh
90596 88092  1092 0  S+  piperd   0xe000127e3ce0 tee
89091 84462  1092 0  SW+Jwait 0xe00224d2 gmake
88092  1092  1092 

Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-10-14 Thread Davide Italiano
On Mon, Oct 14, 2013 at 10:21 AM, Anton Shterenlikht me...@bris.ac.uk wrote:
 From davide.itali...@gmail.com Fri Oct 11 15:39:49 2013

If you're not able to get a full dump, a textdump would be enough.
In your DDB scripts just remove the 'call doadump' step and you should
be done, unless I'm missing something.
Please adjust the script as well to include all the informations
requested as mentioned in my previous link, e.g. 'show lockedvnods' is
not mentioned on the example section of textdump(4) manpage but it
could be useful to ease debugging.

 I still haven't got it right, but I now collected manually
 some (hopefully) useful info (see below).


This is fair enough -- If you're still at the ddb prompt, please print
the whole panic message (or at least the address of the lock reported
as deadlocked by DEADLKRES), so that we can at least have a candidate.

-- 
Davide

There are no solved problems; there are only problems that are more
or less solved -- Henri Poincare
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-10-11 Thread Anton Shterenlikht
From davide.itali...@gmail.com Thu Sep 26 13:12:30 2013
On Thu, Sep 26, 2013 at 12:23 PM, Anton Shterenlikht me...@bris.ac.uk wrote:

 Regarding textdump(4), I'm not clear where the following
 scripts should be located and used:


[snip]

 Are these ddb(8) commands? Or do I set these in /etc/rc.conf?


The scripts (quotes unneeded) should be located in /etc/ddb.conf.
I think you want ddb_enable=YES in your /etc/rc.conf

I've set textdump following the man page.

I've got several more deadlocks, but
it seems savecore causes deadlock too!

# savecore
savecore: reboot after panic: deadlkres: possible deadlock detected for 
0xe000129d2000, blocked for 902608 ticks
savecore: writing core to ./vmcore.5

When the size of vmcore.x file gets to about 9GB,
I get a deadlock again.

What can I do now?

Thanks

Anton

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-10-11 Thread Davide Italiano
On Fri, Oct 11, 2013 at 11:26 AM, Anton Shterenlikht me...@bris.ac.uk wrote:
 From davide.itali...@gmail.com Thu Sep 26 13:12:30 2013
On Thu, Sep 26, 2013 at 12:23 PM, Anton Shterenlikht me...@bris.ac.uk wrote:

 Regarding textdump(4), I'm not clear where the following
 scripts should be located and used:


[snip]

 Are these ddb(8) commands? Or do I set these in /etc/rc.conf?


The scripts (quotes unneeded) should be located in /etc/ddb.conf.
I think you want ddb_enable=YES in your /etc/rc.conf

 I've set textdump following the man page.

 I've got several more deadlocks, but
 it seems savecore causes deadlock too!

 # savecore
 savecore: reboot after panic: deadlkres: possible deadlock detected for 
 0xe000129d2000, blocked for 902608 ticks
 savecore: writing core to ./vmcore.5

 When the size of vmcore.x file gets to about 9GB,
 I get a deadlock again.

 What can I do now?

 Thanks

 Anton


If you're not able to get a full dump, a textdump would be enough.
In your DDB scripts just remove the 'call doadump' step and you should
be done, unless I'm missing something.
Please adjust the script as well to include all the informations
requested as mentioned in my previous link, e.g. 'show lockedvnods' is
not mentioned on the example section of textdump(4) manpage but it
could be useful to ease debugging.

Thanks for your time,

-- 
Davide

There are no solved problems; there are only problems that are more
or less solved -- Henri Poincare
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-09-26 Thread Davide Italiano
On Thu, Sep 26, 2013 at 12:23 PM, Anton Shterenlikht me...@bris.ac.uk wrote:

 Regarding textdump(4), I'm not clear where the following
 scripts should be located and used:


[snip]

 Are these ddb(8) commands? Or do I set these in /etc/rc.conf?


The scripts (quotes unneeded) should be located in /etc/ddb.conf.
I think you want ddb_enable=YES in your /etc/rc.conf

-- 
Davide

There are no solved problems; there are only problems that are more
or less solved -- Henri Poincare
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-09-26 Thread Anton Shterenlikht
From davide.itali...@gmail.com Wed Sep 25 19:00:09 2013

On Wed, Sep 25, 2013 at 5:30 PM, Anton Shterenlikht me...@bris.ac.uk wrote:
 From davide.itali...@gmail.com Wed Sep 25 16:12:47 2013

Can you please paste the output of 'show locks', 'show alllocks',
'show lockedvnods' at least?
Ideally you should provide all the informations listed here.
http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-deadlocks.html

 ok, I'll need to study this.

 I've in the kernel:

 # Debugging support.  Always need this:
 options KDB # Enable kernel debugger support.
 options KDB_TRACE   # Print a stack trace for a panic.
 # For full debugger support use (turn off in stable branch):
 options DDB # Support DDB
 options GDB # Support remote GDB
 options DEADLKRES   # Enable the deadlock resolver
 options INVARIANTS  # Enable calls of extra sanity checking
 options INVARIANT_SUPPORT # required by INVARIANTS
 options WITNESS # Enable checks to detect deadlocks and 
 cycles
 options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed
 options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones

 so I'm missing DEBUG_LOCKS, DEBUG_VFS_LOCKS and DIAGNOSTIC
 from the handbook list.

 What about all debug options in GENERIC which are
 not mentioned in your link? Specifically, do I need
 to have DEADLKRES?


Yes, you need that option because it's DEADLKRES that triggers the panic.

 I've never used trace.
 Also, I'm getting a panic, so cannot run ps, I think.


You can run 'ps' from ddb prompt.
As an advice I suggest you to setup textdump(4) on your machine and
set up a script to gather the required informations, so that you can
get those informations pretty easily for report. The manpage has
detailed description about how to do this.

Regarding textdump(4), I'm not clear where the following
scripts should be located and used:

*quote*
EXAMPLES
 In the following example, the script kdb.enter.panic will run when the
 kernel debugger is entered as a result of a panic, enable output capture,
 dump several useful pieces of debugging information, and then invoke
 panic in order to force a kernel dump to be written out followed by a
 reboot:

   script kdb.enter.panic=textdump set; capture on; show allpcpu; bt;
 ps; alltrace; show alllocks; call doadump; reset

 In the following example, the script kdb.enter.witness will run when the
 kernel debugger is entered as a result of a witness violation, printing
 lock-related information for the user:

   script kdb.enter.witness=show locks

 These scripts may also be configured using the ddb(8) utility.
*end quote*

Are these ddb(8) commands? Or do I set these in /etc/rc.conf?

Please advise

Thanks

Anton

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-09-25 Thread Anton Shterenlikht
FreeBSD mech-as221.men.bris.ac.uk 10.0-ALPHA2 FreeBSD 10.0-ALPHA2 #8 r255811: 
Tue Sep 24 09:04:17 BST 2013 
r...@mech-as221.men.bris.ac.uk:/usr/obj/usr/src/sys/UZI  ia64

panic: deadlkres: possible deadlock detected for 0xe00012d07b00, blocked 
for 902743 ticks

cpuid = 1
KDB: stack backtrace:
db_trace_self(0x9ffc000c9ec0) at db_trace_self+0x40
db_trace_self_wrapper(0x9ffc004d40b0) at db_trace_self_wrapper+0x70
kdb_backtrace(0x9ffc00bfb030, 0x9ffc0045b350, 0x40c, 
0x9ffc00dd20a0) at kdb_backtrace+0xc0
vpanic(0x9ffc00aec840, 0xa05cb518) at vpanic+0x260
panic(0x9ffc00aec840, 0x9ffc00aecaa0, 0xe00012d07b00, 0xdc657) at 
panic+0x80
deadlkres(0xdc657, 0xe00012d07b00, 0x9ffc00aef478, 0x9ffc00aec7d0) 
at deadlkres+0x420
fork_exit(0x9ffc00b441e0, 0x0, 0xa05cb550) at fork_exit+0x120
enter_userland() at enter_userland
KDB: enter: panic
[ thread pid 0 tid 100047 ]
Stopped at  kdb_enter+0x92: [I2]addl r14=0xffe28fb0,gp ;;
db 
db show msgbuf

*skip*

118Sep 24 09:36:02 mech-as221 su: mexas to root on /dev/pts/0
lock order reversal:
 1st 0xa0005f0518b8 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3059
 2nd 0xe00012343000 dirhash (dirhash) @ 
/usr/src/sys/ufs/ufs/ufs_dirhash.c:284
KDB: stack backtrace:
db_trace_self(0x9ffc000c9ec0) at db_trace_self+0x40
db_trace_self_wrapper(0x9ffc004d40b0) at db_trace_self_wrapper+0x70
kdb_backtrace(0x9ffc00bfb030, 0x9ffc00509870) at kdb_backtrace+0xc0
_witness_debugger(0x1, 0x9ffc00b01d10, 0x9ffc0050d240, 0xb9d, 
0x9ffc00b2e268) at _witness_debugger+0x60
witness_checkorder(0xe00012343000, 0x9ffc00b01660, 0x9ffc00b2e268, 
0x11c, 0x0) at witness_checkorder+0x15b0
_sx_xlock(0xe00012343000, 0x0, 0x9ffc00b2e268, 0x11c) at _sx_xlock+0x120
ufsdirhash_acquire(0xe000123bd308, 0xe00012343000, 0x9ffc008d1920, 
0x38b) at ufsdirhash_acquire+0x50
ufsdirhash_remove(0xe000123bd308, 0xa00060ecbd08, 0x1d08, 
0xa0008fea51e8) at ufsdirhash_remove+0x20
ufs_dirremove(0xe000123d8000, 0xe000129e29d8, 0x0, 0x0) at 
ufs_dirremove+0x380
ufs_remove(0xa0008fea5380, 0xe000129e29d8, 0xa1c) at ufs_remove+0xe0
VOP_REMOVE_APV(0x9ffc00bc3180, 0xa0008fea5380, 0xe000122f8678, 0x0, 
0x9ffc005c2920, 0xa1c, 0x9ffc00dd20a0) at VOP_REMOVE_APV+0x220
kern_unlinkat(0xe000123e1200, 0xff9c, 0x7fffee36, 0x0, 
0x0) at kern_unlinkat+0x3f0
kern_unlink(0xe000123e1200, 0x7fffee36, 0x0) at kern_unlink+0x40
sys_unlink(0xe000123e1200, 0xa0008fea54e8, 0x9ffc00988c80, 0x48d) 
at sys_unlink+0x30
syscall(0xe000123de940, 0x7fffee36, 0x7fffeb00, 
0xe000123e1200, 0x0, 0x0, 0x9ffc00983f20, 0x8) at syscall+0x5e0
epc_syscall_return() at epc_syscall_return
6pid 52065 (conftest), uid 0: exited on signal 11 (core dumped)

db show thread
Thread 100047 at 0xe00011973b00:
 proc (pid 0): 0x9ffc00c15828
 name: deadlkres
 stack: 0xa05c4000-0xa05cbfff
 flags: 0x4  pflags: 0x20
 state: RUNNING (CPU 1)
 priority: 108
 container lock: sched lock 1 (0x9ffc00c3de80)
db 
db show proc
Process 0 (kernel) at 0x9ffc00c15828:
 state: NORMAL
 uid: 0  gids: 0
 ABI: null
 threads: 151
100354   D   -0xe00012697000 [zil_clean]
100353   D   -0xe0001230c100 [zil_clean]
100347   D   -0xe00012311c00 [zfs_vn_rele_taskq]
100345   D   -0xe00012313500 [zio_ioctl_intr]
100344   D   -0xe00012313400 [zio_ioctl_issue]
100343   D   -0xe00012313300 [zio_claim_intr]
100342   D   -0xe00012313200 [zio_claim_issue]
100341   D   -0xe00012313100 [zio_free_intr]
100340   D   -0xe00012313000 [zio_free_issue_99]
100339   D   -0xe00012313000 [zio_free_issue_98]
100338   D   -0xe00012313000 [zio_free_issue_97]
100337   D   -0xe00012313000 [zio_free_issue_96]
100336   D   -0xe00012313000 [zio_free_issue_95]
100335   D   -0xe00012313000 [zio_free_issue_94]
100334   D   -0xe00012313000 [zio_free_issue_93]
100333   D   -0xe00012313000 [zio_free_issue_92]
100332   D   -0xe00012313000 [zio_free_issue_91]
100331   D   -0xe00012313000 [zio_free_issue_90]
100330   D   -0xe00012313000 [zio_free_issue_89]
100329   D   -0xe00012313000 [zio_free_issue_88]
100328   D   -0xe00012313000 [zio_free_issue_87]
100327   D   

Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-09-25 Thread Davide Italiano
On Wed, Sep 25, 2013 at 11:11 AM, Anton Shterenlikht me...@bris.ac.uk wrote:
 FreeBSD mech-as221.men.bris.ac.uk 10.0-ALPHA2 FreeBSD 10.0-ALPHA2 #8 r255811: 
 Tue Sep 24 09:04:17 BST 2013 
 r...@mech-as221.men.bris.ac.uk:/usr/obj/usr/src/sys/UZI  ia64

 panic: deadlkres: possible deadlock detected for 0xe00012d07b00, blocked 
 for 902743 ticks

 cpuid = 1
 KDB: stack backtrace:
 db_trace_self(0x9ffc000c9ec0) at db_trace_self+0x40
 db_trace_self_wrapper(0x9ffc004d40b0) at db_trace_self_wrapper+0x70
 kdb_backtrace(0x9ffc00bfb030, 0x9ffc0045b350, 0x40c, 
 0x9ffc00dd20a0) at kdb_backtrace+0xc0
 vpanic(0x9ffc00aec840, 0xa05cb518) at vpanic+0x260
 panic(0x9ffc00aec840, 0x9ffc00aecaa0, 0xe00012d07b00, 0xdc657) at 
 panic+0x80
 deadlkres(0xdc657, 0xe00012d07b00, 0x9ffc00aef478, 
 0x9ffc00aec7d0) at deadlkres+0x420
 fork_exit(0x9ffc00b441e0, 0x0, 0xa05cb550) at fork_exit+0x120
 enter_userland() at enter_userland
 KDB: enter: panic
 [ thread pid 0 tid 100047 ]
 Stopped at  kdb_enter+0x92: [I2]addl r14=0xffe28fb0,gp ;;
 db
 db show msgbuf

 *skip*

 118Sep 24 09:36:02 mech-as221 su: mexas to root on /dev/pts/0
 lock order reversal:
  1st 0xa0005f0518b8 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3059
  2nd 0xe00012343000 dirhash (dirhash) @ 
 /usr/src/sys/ufs/ufs/ufs_dirhash.c:284
 KDB: stack backtrace:
 db_trace_self(0x9ffc000c9ec0) at db_trace_self+0x40
 db_trace_self_wrapper(0x9ffc004d40b0) at db_trace_self_wrapper+0x70
 kdb_backtrace(0x9ffc00bfb030, 0x9ffc00509870) at kdb_backtrace+0xc0
 _witness_debugger(0x1, 0x9ffc00b01d10, 0x9ffc0050d240, 0xb9d, 
 0x9ffc00b2e268) at _witness_debugger+0x60
 witness_checkorder(0xe00012343000, 0x9ffc00b01660, 
 0x9ffc00b2e268, 0x11c, 0x0) at witness_checkorder+0x15b0
 _sx_xlock(0xe00012343000, 0x0, 0x9ffc00b2e268, 0x11c) at 
 _sx_xlock+0x120
 ufsdirhash_acquire(0xe000123bd308, 0xe00012343000, 
 0x9ffc008d1920, 0x38b) at ufsdirhash_acquire+0x50
 ufsdirhash_remove(0xe000123bd308, 0xa00060ecbd08, 0x1d08, 
 0xa0008fea51e8) at ufsdirhash_remove+0x20
 ufs_dirremove(0xe000123d8000, 0xe000129e29d8, 0x0, 0x0) at 
 ufs_dirremove+0x380
 ufs_remove(0xa0008fea5380, 0xe000129e29d8, 0xa1c) at ufs_remove+0xe0
 VOP_REMOVE_APV(0x9ffc00bc3180, 0xa0008fea5380, 0xe000122f8678, 
 0x0, 0x9ffc005c2920, 0xa1c, 0x9ffc00dd20a0) at VOP_REMOVE_APV+0x220
 kern_unlinkat(0xe000123e1200, 0xff9c, 0x7fffee36, 
 0x0, 0x0) at kern_unlinkat+0x3f0
 kern_unlink(0xe000123e1200, 0x7fffee36, 0x0) at kern_unlink+0x40
 sys_unlink(0xe000123e1200, 0xa0008fea54e8, 0x9ffc00988c80, 0x48d) 
 at sys_unlink+0x30
 syscall(0xe000123de940, 0x7fffee36, 0x7fffeb00, 
 0xe000123e1200, 0x0, 0x0, 0x9ffc00983f20, 0x8) at syscall+0x5e0
 epc_syscall_return() at epc_syscall_return
 6pid 52065 (conftest), uid 0: exited on signal 11 (core dumped)

 db show thread
 Thread 100047 at 0xe00011973b00:
  proc (pid 0): 0x9ffc00c15828
  name: deadlkres
  stack: 0xa05c4000-0xa05cbfff
  flags: 0x4  pflags: 0x20
  state: RUNNING (CPU 1)
  priority: 108
  container lock: sched lock 1 (0x9ffc00c3de80)
 db

Can you please paste the output of 'show locks', 'show alllocks',
'show lockedvnods' at least?
Ideally you should provide all the informations listed here.
http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-deadlocks.html

Thanks,

-- 
Davide

A mathematical theory is not to be considered complete until you have
made it so clear that you can explain it to the first man whom you
meet on the street. (D. Hilbert)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-09-25 Thread Anton Shterenlikht
From davide.itali...@gmail.com Wed Sep 25 16:12:47 2013

Can you please paste the output of 'show locks', 'show alllocks',
'show lockedvnods' at least?
Ideally you should provide all the informations listed here.
http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-deadlocks.html

ok, I'll need to study this.

I've in the kernel:

# Debugging support.  Always need this:
options KDB # Enable kernel debugger support.
options KDB_TRACE   # Print a stack trace for a panic.
# For full debugger support use (turn off in stable branch):
options DDB # Support DDB
options GDB # Support remote GDB
options DEADLKRES   # Enable the deadlock resolver
options INVARIANTS  # Enable calls of extra sanity checking
options INVARIANT_SUPPORT # required by INVARIANTS
options WITNESS # Enable checks to detect deadlocks and cycles
options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed
options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones

so I'm missing DEBUG_LOCKS, DEBUG_VFS_LOCKS and DIAGNOSTIC
from the handbook list.

What about all debug options in GENERIC which are
not mentioned in your link? Specifically, do I need
to have DEADLKRES?

I've never used trace.
Also, I'm getting a panic, so cannot run ps, I think.

Thanks

Anton
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: ia64 r255811: deadlkres: possible deadlock detected for 0xe000000012d07b00, blocked for 902743 ticks

2013-09-25 Thread Davide Italiano
On Wed, Sep 25, 2013 at 5:30 PM, Anton Shterenlikht me...@bris.ac.uk wrote:
 From davide.itali...@gmail.com Wed Sep 25 16:12:47 2013

Can you please paste the output of 'show locks', 'show alllocks',
'show lockedvnods' at least?
Ideally you should provide all the informations listed here.
http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-deadlocks.html

 ok, I'll need to study this.

 I've in the kernel:

 # Debugging support.  Always need this:
 options KDB # Enable kernel debugger support.
 options KDB_TRACE   # Print a stack trace for a panic.
 # For full debugger support use (turn off in stable branch):
 options DDB # Support DDB
 options GDB # Support remote GDB
 options DEADLKRES   # Enable the deadlock resolver
 options INVARIANTS  # Enable calls of extra sanity checking
 options INVARIANT_SUPPORT # required by INVARIANTS
 options WITNESS # Enable checks to detect deadlocks and cycles
 options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed
 options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones

 so I'm missing DEBUG_LOCKS, DEBUG_VFS_LOCKS and DIAGNOSTIC
 from the handbook list.

 What about all debug options in GENERIC which are
 not mentioned in your link? Specifically, do I need
 to have DEADLKRES?


Yes, you need that option because it's DEADLKRES that triggers the panic.

 I've never used trace.
 Also, I'm getting a panic, so cannot run ps, I think.


You can run 'ps' from ddb prompt.
As an advice I suggest you to setup textdump(4) on your machine and
set up a script to gather the required informations, so that you can
get those informations pretty easily for report. The manpage has
detailed description about how to do this.

 Thanks

 Anton

-- 
Davide

There are no solved problems; there are only problems that are more
or less solved -- Henri Poincare
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org