Re: lock order reversals w/ backtrace
On Fri, Jan 24, 2014 at 11:47 PM, Thomas Hoffmann wrote: > On Fri, Jan 24, 2014 at 9:52 AM, Thomas Hoffmann wrote: > >> On Thu, Jan 23, 2014 at 11:49 AM, Hans Petter Selasky < >> hans.petter.sela...@bitfrost.no> wrote: >> >>> Hi, >>> >>> Can you see if you can snap some keywords of the backtraces, like usb_xxx >>> usbdx_xxx cam scsi or something like that. >>> >>> Else I believe there are some sysctl options to prevent the final reboot >>> somehow so that you can write down the messages. >>> >>> >>> --HPS >>> >>> >>> -----Original message----- >>> > From:Thomas Hoffmann >>> > Sent: Thursday 23rd January 2014 17:15 >>> >>> > To: freebsd-current >>> > Subject: lock order reversals w/ backtrace >>> > >>> > A few days ago I started running 11.0-CURRENT at r260971 for the first >>> > time. >>> >>> >>> > >>> > The last couple of times I shutdown my system I noticed 2 or 3 short "lock >>> > order reversal" messages with accompanying backtraces scroll by. Do these >>> > messages represent a problem that I should report or can I ignore them as >>> >>> >>> > debug in nature? If I should report them, how or where do these messages >>> > get logged? I can find no reference to them in syslog or anywhere else >>> > upon >>> > my subsequent reboot. >>> > >>> > I also had a couple of these messages pop up the other day while >>> >>> >>> > mounting/umounting USB thumb drives. I did not think anything of them at >>> > the time as the mounts/umounts completed successfully. >>> > >>> > Please advise. Thanks. >>> > >>> > -Tom >>> >>> I managed to snap a photo of my screen during shutdown. >> Here is the full text of the first of two lock order reversals I got last >> night during system shutdown, both of which are zfs-related to (it >> appears?) unmounts. A few lines got chopped as they were out-of-frame when >> I took the photo: >> >> shutting down local daemons: >> lock order reversal: >> 1st 0xf801194f67c8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1237 >> 2nd 0xf801194f6420 syncer (syncer) @ >> /usr/src/sys/kern/vfs_subr.c:22[..chopped...] >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_delf_wrapper+0x2b/frame >> 0xfe01ac78[...chopped...] >> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe01ac784650 >> witness_checkorder() at witness_checkorder+0xd23/frame 0xfe01ac7846e0 >> __lockmgr_args() __lockmgr_args+0x878/frame 0xfe01ac784810 >> vop_stdlock() at vop_stdlock+0x3c/frame 0xfe01ac784830 >> VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfe01ac784860 >> _vn_lock() at _vn_lock+0xab/frame 0xfe01ac7848d0 >> vputx() at vputx+0x240/frame 0xfe01ac784930 >> dounmount at dounmount+0x327/frame 0xfe01ac7849b0 >> sys_unmount() at sys_unmount+0x356/frame 0xfe01ac784ac0 >> amd64_syscall() at amd64_syscall+0x265/frame 0xfe01ac784bf0 >> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe01ac784bf0 >> --- syscall (22, FreeBSD ELF64, sys_unmount, rip = 0x80191c72a, >> rsp[...chopped...] >> >> I have a zpool on an external USB HDD that I export at shutdown via >> rc.shutdown.local. Don't know if that is contributing to this or not. When >> I get a chance to bring down the system I will manually export it before >> shutdown to see if that makes any difference. >> > > Was able to capture the full text of the lock order reversal I've been > experiencing at shutdown. Turns out to be associated with the export of one > of my zpools that is hosted on an external USB HDD. Normally I export the > zpool from rc.shutdown.local, but tonight I exported it manually before I > shutdown and got the following. > > lock order reversal: > : 1st 0xf8011952b5f0 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1237 > : 2nd 0xf8011952b068 syncer (syncer) @ > /usr/src/sys/kern/vfs_subr.c:2212 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfe01add6e5a0 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe01add6e650 > witness_checkorder() at witness_checkorder+0xd23/frame 0xfe01add6e6e0 > __lockmgr_args() at __lockmgr_args+0x878/frame 0xfe01add6e810 > vop_stdlock() at vop_stdlock+0x3c/frame 0xfe01add6e830 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfe01add
Re: lock order reversals w/ backtrace
On Fri, Jan 24, 2014 at 9:52 AM, Thomas Hoffmann wrote: > On Thu, Jan 23, 2014 at 11:49 AM, Hans Petter Selasky < > hans.petter.sela...@bitfrost.no> wrote: > >> Hi, >> >> Can you see if you can snap some keywords of the backtraces, like usb_xxx >> usbdx_xxx cam scsi or something like that. >> >> Else I believe there are some sysctl options to prevent the final reboot >> somehow so that you can write down the messages. >> >> --HPS >> >> >> -Original message- >> > From:Thomas Hoffmann >> > Sent: Thursday 23rd January 2014 17:15 >> >> > To: freebsd-current >> > Subject: lock order reversals w/ backtrace >> > >> > A few days ago I started running 11.0-CURRENT at r260971 for the first >> > time. >> >> > >> > The last couple of times I shutdown my system I noticed 2 or 3 short "lock >> > order reversal" messages with accompanying backtraces scroll by. Do these >> > messages represent a problem that I should report or can I ignore them as >> >> > debug in nature? If I should report them, how or where do these messages >> > get logged? I can find no reference to them in syslog or anywhere else upon >> > my subsequent reboot. >> > >> > I also had a couple of these messages pop up the other day while >> >> > mounting/umounting USB thumb drives. I did not think anything of them at >> > the time as the mounts/umounts completed successfully. >> > >> > Please advise. Thanks. >> > >> > -Tom >> >> I managed to snap a photo of my screen during shutdown. > Here is the full text of the first of two lock order reversals I got last > night during system shutdown, both of which are zfs-related to (it > appears?) unmounts. A few lines got chopped as they were out-of-frame when > I took the photo: > > shutting down local daemons: > lock order reversal: > 1st 0xf801194f67c8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1237 > 2nd 0xf801194f6420 syncer (syncer) @ > /usr/src/sys/kern/vfs_subr.c:22[..chopped...] > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_delf_wrapper+0x2b/frame > 0xfe01ac78[...chopped...] > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe01ac784650 > witness_checkorder() at witness_checkorder+0xd23/frame 0xfe01ac7846e0 > __lockmgr_args() __lockmgr_args+0x878/frame 0xfe01ac784810 > vop_stdlock() at vop_stdlock+0x3c/frame 0xfe01ac784830 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfe01ac784860 > _vn_lock() at _vn_lock+0xab/frame 0xfe01ac7848d0 > vputx() at vputx+0x240/frame 0xfe01ac784930 > dounmount at dounmount+0x327/frame 0xfe01ac7849b0 > sys_unmount() at sys_unmount+0x356/frame 0xfe01ac784ac0 > amd64_syscall() at amd64_syscall+0x265/frame 0xfe01ac784bf0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe01ac784bf0 > --- syscall (22, FreeBSD ELF64, sys_unmount, rip = 0x80191c72a, > rsp[...chopped...] > > I have a zpool on an external USB HDD that I export at shutdown via > rc.shutdown.local. Don't know if that is contributing to this or not. When > I get a chance to bring down the system I will manually export it before > shutdown to see if that makes any difference. > Was able to capture the full text of the lock order reversal I've been experiencing at shutdown. Turns out to be associated with the export of one of my zpools that is hosted on an external USB HDD. Normally I export the zpool from rc.shutdown.local, but tonight I exported it manually before I shutdown and got the following. lock order reversal: : 1st 0xf8011952b5f0 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1237 : 2nd 0xf8011952b068 syncer (syncer) @ /usr/src/sys/kern/vfs_subr.c:2212 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe01add6e5a0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe01add6e650 witness_checkorder() at witness_checkorder+0xd23/frame 0xfe01add6e6e0 __lockmgr_args() at __lockmgr_args+0x878/frame 0xfe01add6e810 vop_stdlock() at vop_stdlock+0x3c/frame 0xfe01add6e830 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfe01add6e860 _vn_lock() at _vn_lock+0xab/frame 0xfe01add6e8d0 vputx() at vputx+0x240/frame 0xfe01add6e930 dounmount() at dounmount+0x327/frame 0xfe01add6e9b0 sys_unmount() at sys_unmount+0x356/frame 0xfe01add6eae0 amd64_syscall() at amd64_syscall+0x265/frame 0xfe01add6ebf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe01add6ebf0 --- syscall (22, FreeBSD ELF64, sys_unmount), rip = 0x80191c72a, rsp = 0x7fffc4c8, rbp = 0x7fffc960 --- When imported and mounted, the zpool reports no errors and I have not experienced any anomalies reading or writing to the zpool. Should I be concerned? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversals w/ backtrace
On Thu, Jan 23, 2014 at 11:49 AM, Hans Petter Selasky < hans.petter.sela...@bitfrost.no> wrote: > Hi, > > Can you see if you can snap some keywords of the backtraces, like usb_xxx > usbdx_xxx cam scsi or something like that. > > Else I believe there are some sysctl options to prevent the final reboot > somehow so that you can write down the messages. > > --HPS > > > -Original message- > > From:Thomas Hoffmann > > Sent: Thursday 23rd January 2014 17:15 > > To: freebsd-current > > Subject: lock order reversals w/ backtrace > > > > A few days ago I started running 11.0-CURRENT at r260971 for the first time. > > > > The last couple of times I shutdown my system I noticed 2 or 3 short "lock > > order reversal" messages with accompanying backtraces scroll by. Do these > > messages represent a problem that I should report or can I ignore them as > > debug in nature? If I should report them, how or where do these messages > > get logged? I can find no reference to them in syslog or anywhere else upon > > my subsequent reboot. > > > > I also had a couple of these messages pop up the other day while > > mounting/umounting USB thumb drives. I did not think anything of them at > > the time as the mounts/umounts completed successfully. > > > > Please advise. Thanks. > > > > -Tom > > I managed to snap a photo of my screen during shutdown. Here is the full text of the first of two lock order reversals I got last night during system shutdown, both of which are zfs-related to (it appears?) unmounts. A few lines got chopped as they were out-of-frame when I took the photo: shutting down local daemons: lock order reversal: 1st 0xf801194f67c8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1237 2nd 0xf801194f6420 syncer (syncer) @ /usr/src/sys/kern/vfs_subr.c:22[..chopped...] KDB: stack backtrace: db_trace_self_wrapper() at db_trace_delf_wrapper+0x2b/frame 0xfe01ac78[...chopped...] kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe01ac784650 witness_checkorder() at witness_checkorder+0xd23/frame 0xfe01ac7846e0 __lockmgr_args() __lockmgr_args+0x878/frame 0xfe01ac784810 vop_stdlock() at vop_stdlock+0x3c/frame 0xfe01ac784830 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfe01ac784860 _vn_lock() at _vn_lock+0xab/frame 0xfe01ac7848d0 vputx() at vputx+0x240/frame 0xfe01ac784930 dounmount at dounmount+0x327/frame 0xfe01ac7849b0 sys_unmount() at sys_unmount+0x356/frame 0xfe01ac784ac0 amd64_syscall() at amd64_syscall+0x265/frame 0xfe01ac784bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe01ac784bf0 --- syscall (22, FreeBSD ELF64, sys_unmount, rip = 0x80191c72a, rsp[...chopped...] I have a zpool on an external USB HDD that I export at shutdown via rc.shutdown.local. Don't know if that is contributing to this or not. When I get a chance to bring down the system I will manually export it before shutdown to see if that makes any difference. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RE: lock order reversals w/ backtrace
Hi, Can you see if you can snap some keywords of the backtraces, like usb_xxx usbdx_xxx cam scsi or something like that. Else I believe there are some sysctl options to prevent the final reboot somehow so that you can write down the messages. --HPS -Original message- > From:Thomas Hoffmann mailto:trh...@gmail.com> > > Sent: Thursday 23rd January 2014 17:15 > To: freebsd-current <mailto:freebsd-current@freebsd.org> > > Subject: lock order reversals w/ backtrace > > A few days ago I started running 11.0-CURRENT at r260971 for the first time. > > The last couple of times I shutdown my system I noticed 2 or 3 short "lock > order reversal" messages with accompanying backtraces scroll by. Do these > messages represent a problem that I should report or can I ignore them as > debug in nature? If I should report them, how or where do these messages > get logged? I can find no reference to them in syslog or anywhere else upon > my subsequent reboot. > > I also had a couple of these messages pop up the other day while > mounting/umounting USB thumb drives. I did not think anything of them at > the time as the mounts/umounts completed successfully. > > Please advise. Thanks. > > -Tom > ___ > freebsd-current@freebsd.org <mailto:freebsd-current@freebsd.org> mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > <http://lists.freebsd.org/mailman/listinfo/freebsd-current> > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org > <mailto:freebsd-current-unsubscr...@freebsd.org> " > ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
lock order reversals w/ backtrace
A few days ago I started running 11.0-CURRENT at r260971 for the first time. The last couple of times I shutdown my system I noticed 2 or 3 short "lock order reversal" messages with accompanying backtraces scroll by. Do these messages represent a problem that I should report or can I ignore them as debug in nature? If I should report them, how or where do these messages get logged? I can find no reference to them in syslog or anywhere else upon my subsequent reboot. I also had a couple of these messages pop up the other day while mounting/umounting USB thumb drives. I did not think anything of them at the time as the mounts/umounts completed successfully. Please advise. Thanks. -Tom ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"