A lock order reversal that I've not seen before (zfs and tmpfs during poudriere bulk using USE_TMPFS=all)
I've never figured out how to tell important lock order reversal notices from unimportant ones. So I mostly report only unfamiliar ones. (But I normally do not do poudriere bulk builds with a debug kernel in use.) During a poudriere bulk that is using USE_TMPFS=all it reported: lock order reversal: 1st 0xa0027b7e83f0 zfs (zfs, lockmgr) @ /usr/src/sys/kern/vfs_mount.c:2240 2nd 0xa0023db44070 tmpfs (tmpfs, lockmgr) @ /usr/src/sys/kern/vfs_subr.c:3886 lock order tmpfs -> zfs established at: #0 0x004d7824 at witness_checkorder+0x304 #1 0x00435bfc at lockmgr_xlock+0x50 #2 0x00015d0ce814 at null_lock+0xb0 #3 0x0056cdf0 at _vn_lock+0x54 #4 0x00557170 at vflush+0x12c #5 0x00015d0cd6b0 at nullfs_unmount+0x40 #6 0x0054c318 at dounmount+0x714 #7 0x0054bb94 at kern_unmount+0x298 #8 0x007fe6ac at do_el0_sync+0x520 #9 0x007da110 at handle_el0_sync+0x44 lock order zfs -> tmpfs attempted at: #0 0x004d7fb8 at witness_checkorder+0xa98 #1 0x00434140 at lockmgr_lock_flags+0x1ec #2 0x0056cdf0 at _vn_lock+0x54 #3 0x00557170 at vflush+0x12c #4 0x003964d0 at tmpfs_unmount+0x60 #5 0x0054c318 at dounmount+0x714 #6 0x0054bb94 at kern_unmount+0x298 #7 0x007fe6ac at do_el0_sync+0x520 #8 0x007da110 at handle_el0_sync+0x44 It happens to be on aarch64, in case that matters for some odd reason. === Mark Millard marklmi at yahoo.com
Re: debug head -r368500 kernel (for example) : "lock order reversal: (sleepable after non-sleepable)" involving ure0 and a sysctl lock; more
On 2020-Dec-19, at 03:09, Hans Petter Selasky wrote: > Please test: > > https://svnweb.freebsd.org/changeset/base/368799 > https://svnweb.freebsd.org/changeset/base/368801 > > --HPS I grabbed a debug -r368803 kernel from artifacts (first arm64 snapshot available containing the above 2 updates): https://artifact.ci.freebsd.org/snapshot/13.0-CURRENT/r368803/arm64/aarch64/kernel.txz I used it to substitute in the updated debug kernel and booted the same configuration. It booted fine with no LOR or uma_zalloc_debug related console output. I've not tried my own non-debug build yet but that kind of context had never given me a clue of these issues anyway. (I am not expecting the above to change the non-debug "Root mount waiting for: usbus0" for the uefi/ACPI USB3 SSD based boot sequence.) === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: debug head -r368500 kernel (for example) : "lock order reversal: (sleepable after non-sleepable)" involving ure0 and a sysctl lock; more
Please test: https://svnweb.freebsd.org/changeset/base/368799 https://svnweb.freebsd.org/changeset/base/368801 --HPS ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: debug head -r368500 kernel (for example) : "lock order reversal: (sleepable after non-sleepable)" involving ure0 and a sysctl lock; more
[I managed to not cc the primary person that I intended but to cc the secondary person. So this resend just adds jmg and removes hps.] On 2020-Dec-18, at 22:27, Mark Millard wrote: > The following is from head -r368500's artifact kernel from: > > https://artifact.ci.freebsd.org/snapshot/13.0-CURRENT/r368500/arm64/aarch64/kernel.txz > > but the same sort of material showed for -r368000 . > (I was attempting a bisect for a different issue but > the debug kernels did not fail for what I was looking > for and all the debug versions that I tried reported > similarly to the below.) > > Note also the mixing in of "uma_zalloc_debug" activity > after the initial LOR backtrace, ure0 still involved. > > Autoloading module: if_ure.ko > ure0 on uhub0 > ure0: on > usbus0 > add host 127.0.0.1: gatelock order reversal: (sleepable after non-sleepable) > 1st 0xa2b2cea0 ure0 (ure0, sleep mutex) @ > /usr/src/sys/dev/usb/usb_request.c:714 > 2nd 0x00dd6858 sysctl lock (sysctl lock, sleepable rm) @ /usway lo0 > fib 0: route alrr/src/sys/kern/kern_sysctl.c:836 > lock order ure0 -> sysctl lock attempted at: > #0 0x0056d068 at witness_checkorder+0xc54 > #1 0x004f8f08 at _rm_wlock_debug+0x88 > #2 0x0050ee2c at sysctl_add_oid+0x60 > #3 0x9e415780 at ure_attach_post+0x1a78 > #4 0x00391d6c at ue_attach_post_task+0x3c > #5 0x003826e0 at usb_process+0x10c > #6 0x004baa1c at fork_exit+0x7c > #7 0x00816544 at fork_trampoline+0x10 > uma_zalloc_debug: zone "malloc-128eady in table > " with the following non-sleepable locks held: > exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ > /usr/src/sys/dev/usb/usb_request.c:714 > stack backtrace: > #0 0x0056d388 at witness_debugger+0x64 > #1 0x0056e518 at witness_warn+0x3ec > #2 0x00778f9c at uma_zalloc_debug+0x2c > #3 0x00778998 at uma_zalloc_arg+0x2c > #4 0x004d534c at malloc+0xa0 > #5 0x0050ee80 at sysctl_add_oid+0xb4 > #6 0x9e415780 at ure_attach_post+0x1a78 > #7 0x00391d6c at ue_attach_post_task+0x3c > #8 0x003826e0 at usb_process+0x10c > #9 0x004baa1c at fork_exit+0x7c > #10 0x00816544 at fork_trampoline+0x10 > uma_zalloc_debug: zone "malloc-16" with the following non-sleepable locks > held: > exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ > /usr/src/sys/dev/usb/usb_request.c:714 > stack backtrace: > #0 0x0056d388 at witness_debugger+0x64 > #1 0x0056e518 at witness_warn+0x3ec > #2 0x00778f9c at uma_zalloc_debug+0x2c > #3 0x00778998 at uma_zalloc_arg+0x2c > #4 0x004d534c at malloc+0xa0 > #5 0x005f8c80 at strdup+0x2c > #6 0x0050eeb8 at sysctl_add_oid+0xec > #7 0x9e415780 at ure_attach_post+0x1a78 > #8 0x00391d6c at ue_attach_post_task+0x3c > #9 0x003826e0 at usb_process+0x10c > #10 0x004baa1c at fork_exit+0x7c > #11 0x00816544 at fork_trampoline+0x10 > uma_zalloc_debug: zone "malloc-64" with the following non-sleepable locks > held: > exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ > /usr/src/sys/dev/usb/usb_request.c:714 > stack backtrace: > #0 0x0056d388 at witness_debugger+0x64 > #1 0x0056e518 at witness_warn+0x3ec > #2 0x00778f9c at uma_zalloc_debug+0x2c > #3 0x00778998 at uma_zalloc_arg+0x2c > #4 0x004d534c at malloc+0xa0 > #5 0x005f8c80 at strdup+0x2c > #6 0x0050eee4 at sysctl_add_oid+0x118 > #7 0x9e415780 at ure_attach_post+0x1a78 > #8 0x00391d6c at ue_attach_post_task+0x3c > #9 0x003826e0 at usb_process+0x10c > #10 0x004baa1c at fork_exit+0x7c > #11 0x00816544 at fork_trampoline+0x10 > uma_zalloc_debug: zone "malloc-32" with the following non-sleepable locks > held: > exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ > /usr/src/sys/dev/usb/usb_request.c:714 > stack backtrace: > #0 0x0056d388 at witness_debugger+0x64 > #1 0x0056e518 at witness_warn+0x3ec > #2 0x00778f9c at uma_zalloc_debug+0x2c > #3 0x00778998 at uma_zalloc_arg+0x2c > #4 0x004d534c at malloc+0xa0 > #5 0x0050ef3c at sysctl_add_oid+0x170 > #6 0x9e415780 at ure_attach_post+0x1a78 > #7 0x00391d6c at ue_attach_post_task+0x3c > #8 0x003826e0 at usb_process+0x10c > #9 0x004baa1c at fork_exit+0x7c > #10 0x00816544 at fork_trampoline+0x10 > miibus0: on ure0 > rgephy0: PHY 0 on miibus0 > add hrgephy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, > 1000baseT-FDX, 1000baseT-FDX-master, auto > 0 fib 0: route already iue0: on ure0 > > > > The context here is an RPi4 (aarch64 cortex-a72) with: > > # uname -apKU > FreeBSD RPi4B 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r368500: Thu Dec 10 > 07:52:39 UTC 2020 > r...@freebsd-head-aarch64-build.jail.ci.freebsd.org:/usr/obj/usr/sr
debug head -r368500 kernel (for example) : "lock order reversal: (sleepable after non-sleepable)" involving ure0 and a sysctl lock; more
The following is from head -r368500's artifact kernel from: https://artifact.ci.freebsd.org/snapshot/13.0-CURRENT/r368500/arm64/aarch64/kernel.txz but the same sort of material showed for -r368000 . (I was attempting a bisect for a different issue but the debug kernels did not fail for what I was looking for and all the debug versions that I tried reported similarly to the below.) Note also the mixing in of "uma_zalloc_debug" activity after the initial LOR backtrace, ure0 still involved. Autoloading module: if_ure.ko ure0 on uhub0 ure0: on usbus0 add host 127.0.0.1: gatelock order reversal: (sleepable after non-sleepable) 1st 0xa2b2cea0 ure0 (ure0, sleep mutex) @ /usr/src/sys/dev/usb/usb_request.c:714 2nd 0x00dd6858 sysctl lock (sysctl lock, sleepable rm) @ /usway lo0 fib 0: route alrr/src/sys/kern/kern_sysctl.c:836 lock order ure0 -> sysctl lock attempted at: #0 0x0056d068 at witness_checkorder+0xc54 #1 0x004f8f08 at _rm_wlock_debug+0x88 #2 0x0050ee2c at sysctl_add_oid+0x60 #3 0x9e415780 at ure_attach_post+0x1a78 #4 0x00391d6c at ue_attach_post_task+0x3c #5 0x003826e0 at usb_process+0x10c #6 0x004baa1c at fork_exit+0x7c #7 0x00816544 at fork_trampoline+0x10 uma_zalloc_debug: zone "malloc-128eady in table " with the following non-sleepable locks held: exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ /usr/src/sys/dev/usb/usb_request.c:714 stack backtrace: #0 0x0056d388 at witness_debugger+0x64 #1 0x0056e518 at witness_warn+0x3ec #2 0x00778f9c at uma_zalloc_debug+0x2c #3 0x00778998 at uma_zalloc_arg+0x2c #4 0x004d534c at malloc+0xa0 #5 0x0050ee80 at sysctl_add_oid+0xb4 #6 0x9e415780 at ure_attach_post+0x1a78 #7 0x00391d6c at ue_attach_post_task+0x3c #8 0x003826e0 at usb_process+0x10c #9 0x004baa1c at fork_exit+0x7c #10 0x00816544 at fork_trampoline+0x10 uma_zalloc_debug: zone "malloc-16" with the following non-sleepable locks held: exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ /usr/src/sys/dev/usb/usb_request.c:714 stack backtrace: #0 0x0056d388 at witness_debugger+0x64 #1 0x0056e518 at witness_warn+0x3ec #2 0x00778f9c at uma_zalloc_debug+0x2c #3 0x00778998 at uma_zalloc_arg+0x2c #4 0x004d534c at malloc+0xa0 #5 0x005f8c80 at strdup+0x2c #6 0x0050eeb8 at sysctl_add_oid+0xec #7 0x9e415780 at ure_attach_post+0x1a78 #8 0x00391d6c at ue_attach_post_task+0x3c #9 0x003826e0 at usb_process+0x10c #10 0x004baa1c at fork_exit+0x7c #11 0x00816544 at fork_trampoline+0x10 uma_zalloc_debug: zone "malloc-64" with the following non-sleepable locks held: exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ /usr/src/sys/dev/usb/usb_request.c:714 stack backtrace: #0 0x0056d388 at witness_debugger+0x64 #1 0x0056e518 at witness_warn+0x3ec #2 0x00778f9c at uma_zalloc_debug+0x2c #3 0x00778998 at uma_zalloc_arg+0x2c #4 0x004d534c at malloc+0xa0 #5 0x005f8c80 at strdup+0x2c #6 0x0050eee4 at sysctl_add_oid+0x118 #7 0x9e415780 at ure_attach_post+0x1a78 #8 0x00391d6c at ue_attach_post_task+0x3c #9 0x003826e0 at usb_process+0x10c #10 0x004baa1c at fork_exit+0x7c #11 0x00816544 at fork_trampoline+0x10 uma_zalloc_debug: zone "malloc-32" with the following non-sleepable locks held: exclusive sleep mutex ure0 (ure0) r = 0 (0xa2b2cea0) locked @ /usr/src/sys/dev/usb/usb_request.c:714 stack backtrace: #0 0x0056d388 at witness_debugger+0x64 #1 0x0056e518 at witness_warn+0x3ec #2 0x00778f9c at uma_zalloc_debug+0x2c #3 0x00778998 at uma_zalloc_arg+0x2c #4 0x004d534c at malloc+0xa0 #5 0x0050ef3c at sysctl_add_oid+0x170 #6 0x9e415780 at ure_attach_post+0x1a78 #7 0x00391d6c at ue_attach_post_task+0x3c #8 0x003826e0 at usb_process+0x10c #9 0x004baa1c at fork_exit+0x7c #10 0x00816544 at fork_trampoline+0x10 miibus0: on ure0 rgephy0: PHY 0 on miibus0 add hrgephy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto 0 fib 0: route already iue0: on ure0 The context here is an RPi4 (aarch64 cortex-a72) with: # uname -apKU FreeBSD RPi4B 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r368500: Thu Dec 10 07:52:39 UTC 2020 r...@freebsd-head-aarch64-build.jail.ci.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 aarch64 1300131 1300131 The boot attempts were via uefi/ACPI using https://github.com/pftf/RPi4 v1.21 materials, directly booting from the USB3 SSD, no microsd card involved. Some context in case it contributes something for the above (probably not) . . . The reason for the bisect was: such boot attempts fail to mount route with my non-debug head -r368500 kernel bui
r366623: lock order reversal in ZFS
On a recent CURRENT bos acting as poudriere host, we face after a crash under heavy load and extensive swap usage now this lock order reversal warning on the console: lock order reversal: 1st 0xf8007bdd9e00 zfs (zfs, lockmgr) @ /usr/src/sys/kern/vfs_mount.c:1018 2nd 0xf8034abfc070 devfs (devfs, lockmgr) @ /usr/src/sys/kern/vfs_mount.c:1029 lock order zfs -> devfs attempted at: #0 0x80f3380d at witness_checkorder+0xebd #1 0x80e8ba4b at lockmgr_xlock+0xab #2 0x80fc3e04 at _vn_lock+0x54 #3 0x80fa0e34 at vfs_domount+0xde4 #4 0x80f9f598 at vfs_donmount+0x898 #5 0x80f9ecb9 at sys_nmount+0x69 #6 0x81461255 at amd64_syscall+0x135 #7 0x814335ee at fast_syscall_common+0xf8 System: FreeBSD 13.0-CURRENT #71 r366623: Sun Oct 11 08:59:17 CEST 2020 pgp5mJtYDzyfv.pgp Description: OpenPGP digital signature
Re: lock order reversal and poudriere
On Sun, May 03, 2020 at 10:04:04AM -0600, Ian Lepore wrote: > That LOR site hasn't been updated in years. Many many years. If someone wants to help me set up a page on the wiki, let me know. (I have too much on my plate to do it myself.) mcl ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal and poudriere
On Sat, 2020-05-02 at 20:36 +0200, Kurt Jaeger wrote: > Hi! > > > > > I am compiling some packages with poudriere on 13-current kernel. I > > > > noticed some strange messages printed into the terminal and dmesg: > > > > > > > > lock order reversal: > > > > > > [...] > > > > Are those the debug messages that aren't visible on non-current kernel > > > > and should they be reported? > > > > > > Yes, they should be checked and reported. > > > > > > For more details see: > > > > > > http://sources.zabbadoz.net/freebsd/lor.html > > > > > > There's a webpage with a list of all known LORs and a way to > > > report new LORs. > > Thanks Kurt. I can't find those two specific LORs in the list on that > > page. The page also says to report them using a link, which leads to 404 > > :-), or on this mailing list, which I did. I am not sure what else should > > I do. > > I don't know, either 8-} bz@ is in Cc:, so he'll probably know what > to do. > That LOR site hasn't been updated in years. Many many years. The sad truth appears to be that nobody cares about LORs anymore. The same ones have been there for years. Nobody fixes them, nobody does anything to suppress reporting them. We just keep pointing new users to a dead website because that has always been the only response available. > > How do I know if I have got a backtrace? > > > > Are those errors: > > > > pid 43297 (conftest), jid 5, uid 0: exited on signal 11 > > > > related or it's a different issue? > > I think that's a different issue. > Segfaults and other problems with a program named "conftest" while building ports is normal. Autotools' configure script writes and runs programs named conftest to detect the presence or absence of features or bugs. That doesn't mean every failure of a program named conftest is normal and expected, but in general it's not a thing to worry about. -- Ian ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal and poudriere
On 03/05/2020 15:00, Niclas Zeising wrote: On 2020-05-02 20:36, Kurt Jaeger wrote: I don't know, either 8-} bz@ is in Cc:, so he'll probably know what to do. How do I know if I have got a backtrace? Are those errors: pid 43297 (conftest), jid 5, uid 0: exited on signal 11 related or it's a different issue? I think that's a different issue. conftest is when configure scripts do things. Configure works a lot by compiling (and sometimes running) small snippets of code to figure out what's going on. Sometimes those snippets core dump. It's all normal. Good to know. It's mostly conftest but sometimes others too: pid 37407 (cc), jid 9, uid 0: exited on signal 6 pid 95358 (conftest), jid 3, uid 0: exited on signal 11 pid 70242 (conftest), jid 9, uid 0: exited on signal 11 pid 27480 (ngc27183), jid 3, uid 0: exited on signal 11 Regards --GrzegorzJ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal and poudriere
On 2020-05-02 20:36, Kurt Jaeger wrote: Hi! I am compiling some packages with poudriere on 13-current kernel. I noticed some strange messages printed into the terminal and dmesg: lock order reversal: [...] Are those the debug messages that aren't visible on non-current kernel and should they be reported? Yes, they should be checked and reported. For more details see: http://sources.zabbadoz.net/freebsd/lor.html There's a webpage with a list of all known LORs and a way to report new LORs. Thanks Kurt. I can't find those two specific LORs in the list on that page. The page also says to report them using a link, which leads to 404 :-), or on this mailing list, which I did. I am not sure what else should I do. I don't know, either 8-} bz@ is in Cc:, so he'll probably know what to do. How do I know if I have got a backtrace? Are those errors: pid 43297 (conftest), jid 5, uid 0: exited on signal 11 related or it's a different issue? I think that's a different issue. conftest is when configure scripts do things. Configure works a lot by compiling (and sometimes running) small snippets of code to figure out what's going on. Sometimes those snippets core dump. It's all normal. Regards -- Niclas ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal and poudriere
On 02/05/2020 10:08, Grzegorz Junka wrote: I am compiling some packages with poudriere on 13-current kernel. I noticed some strange messages printed into the terminal and dmesg: lock order reversal: 1st 0xf8010ca78250 zfs (zfs) @ /usr/src-13/sys/kern/vfs_mount.c:1005 2nd 0xf8010cd37250 devfs (devfs) @ /usr/src-13/sys/kern/vfs_mount.c:1016 stack backtrace: #0 0x80c2d5f1 at witness_debugger+0x71 #1 0x80b92f18 at lockmgr_lock_flags+0x188 #2 0x80cae744 at _vn_lock+0x54 #3 0x80c90756 at vfs_domount+0xd16 #4 0x80c8efd1 at vfs_donmount+0x871 #5 0x80c8e729 at sys_nmount+0x69 #6 0x81060c40 at amd64_syscall+0x140 #7 0x810370a0 at fast_syscall_common+0x101 pid 17216 (conftest), jid 6, uid 0: exited on signal 11 pid 51159 (conftest), jid 6, uid 0: exited on signal 11 pid 23833 (conftest), jid 3, uid 0: exited on signal 11 pid 4916 (conftest), jid 3, uid 0: exited on signal 11 (... then there is a bunch of similar ones, then ...) pid 14504 (conftest), jid 3, uid 0: exited on signal 11 pid 27466 (conftest), jid 6, uid 0: exited on signal 11 pid 43297 (conftest), jid 5, uid 0: exited on signal 11 lock order reversal: 1st 0xfe00bc68c030 filedesc structure (filedesc structure) @ /usr/src-13/sys/kern/sys_generic.c:1557 2nd 0xf803baeddbd8 tmpfs (tmpfs) @ /usr/src-13/sys/kern/vfs_vnops.c:1553 stack backtrace: #0 0x80c2d5f1 at witness_debugger+0x71 #1 0x80b946b5 at lockmgr_xlock+0x55 #2 0x80cae744 at _vn_lock+0x54 #3 0x80cad0da at vn_poll+0x3a #4 0x80c33e19 at kern_poll+0x419 #5 0x80c340df at sys_ppoll+0x6f #6 0x81060c40 at amd64_syscall+0x140 #7 0x810370a0 at fast_syscall_common+0x101 pid 37533 (conftest), jid 5, uid 0: exited on signal 11 pid 43474 (conftest), jid 5, uid 0: exited on signal 11 I restarted the compilation and again seeing similar LORs: lock order reversal: 1st 0xf80115d32068 zfs (zfs) @ /usr/src-13/sys/kern/vfs_mount.c:1005 2nd 0xf800243d6808 devfs (devfs) @ /usr/src-13/sys/kern/vfs_mount.c:1016 stack backtrace: #0 0x80c2d5f1 at witness_debugger+0x71 #1 0x80b92f18 at lockmgr_lock_flags+0x188 #2 0x80cae744 at _vn_lock+0x54 #3 0x80c90756 at vfs_domount+0xd16 #4 0x80c8efd1 at vfs_donmount+0x871 #5 0x80c8e729 at sys_nmount+0x69 #6 0x81060c40 at amd64_syscall+0x140 #7 0x810370a0 at fast_syscall_common+0x101 lock order reversal: 1st 0xfe00a7aa49b0 filedesc structure (filedesc structure) @ /usr/src-13/sys/kern/sys_generic.c:1557 2nd 0xf800aa2cdbd8 zfs (zfs) @ /usr/src-13/sys/kern/vfs_vnops.c:1553 stack backtrace: #0 0x80c2d5f1 at witness_debugger+0x71 #1 0x80b946b5 at lockmgr_xlock+0x55 #2 0x80cae744 at _vn_lock+0x54 #3 0x80cad0da at vn_poll+0x3a #4 0x80c33e19 at kern_poll+0x419 #5 0x80c339f0 at sys_poll+0x50 #6 0x81060c40 at amd64_syscall+0x140 #7 0x810370a0 at fast_syscall_common+0x101 The page to report still returns 404 :) -- GrzegorzJ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal and poudriere
Hi! > > > I am compiling some packages with poudriere on 13-current kernel. I > > > noticed some strange messages printed into the terminal and dmesg: > > > > > > lock order reversal: > > [...] > > > Are those the debug messages that aren't visible on non-current kernel > > > and should they be reported? > > Yes, they should be checked and reported. > > > > For more details see: > > > > http://sources.zabbadoz.net/freebsd/lor.html > > > > There's a webpage with a list of all known LORs and a way to > > report new LORs. > Thanks Kurt. I can't find those two specific LORs in the list on that > page. The page also says to report them using a link, which leads to 404 > :-), or on this mailing list, which I did. I am not sure what else should > I do. I don't know, either 8-} bz@ is in Cc:, so he'll probably know what to do. > How do I know if I have got a backtrace? > > Are those errors: > > pid 43297 (conftest), jid 5, uid 0: exited on signal 11 > > related or it's a different issue? I think that's a different issue. -- p...@freebsd.org +49 171 3101372 Now what ? ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal and poudriere
On 02/05/2020 10:54, Kurt Jaeger wrote: Hi! I am compiling some packages with poudriere on 13-current kernel. I noticed some strange messages printed into the terminal and dmesg: lock order reversal: [...] Are those the debug messages that aren't visible on non-current kernel and should they be reported? Yes, they should be checked and reported. For more details see: http://sources.zabbadoz.net/freebsd/lor.html There's a webpage with a list of all known LORs and a way to report new LORs. Thanks Kurt. I can't find those two specific LORs in the list on that page. The page also says to report them using a link, which leads to 404 :-), or on this mailing list, which I did. I am not sure what else should I do. How do I know if I have got a backtrace? Are those errors: pid 43297 (conftest), jid 5, uid 0: exited on signal 11 related or it's a different issue? GrzegorzJ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal and poudriere
Hi! > I am compiling some packages with poudriere on 13-current kernel. I > noticed some strange messages printed into the terminal and dmesg: > > lock order reversal: [...] > Are those the debug messages that aren't visible on non-current kernel > and should they be reported? Yes, they should be checked and reported. For more details see: http://sources.zabbadoz.net/freebsd/lor.html There's a webpage with a list of all known LORs and a way to report new LORs. -- p...@opsec.eu+49 171 3101372Now what ? ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
lock order reversal and poudriere
I am compiling some packages with poudriere on 13-current kernel. I noticed some strange messages printed into the terminal and dmesg: lock order reversal: 1st 0xf8010ca78250 zfs (zfs) @ /usr/src-13/sys/kern/vfs_mount.c:1005 2nd 0xf8010cd37250 devfs (devfs) @ /usr/src-13/sys/kern/vfs_mount.c:1016 stack backtrace: #0 0x80c2d5f1 at witness_debugger+0x71 #1 0x80b92f18 at lockmgr_lock_flags+0x188 #2 0x80cae744 at _vn_lock+0x54 #3 0x80c90756 at vfs_domount+0xd16 #4 0x80c8efd1 at vfs_donmount+0x871 #5 0x80c8e729 at sys_nmount+0x69 #6 0x81060c40 at amd64_syscall+0x140 #7 0x810370a0 at fast_syscall_common+0x101 pid 17216 (conftest), jid 6, uid 0: exited on signal 11 pid 51159 (conftest), jid 6, uid 0: exited on signal 11 pid 23833 (conftest), jid 3, uid 0: exited on signal 11 pid 4916 (conftest), jid 3, uid 0: exited on signal 11 (... then there is a bunch of similar ones, then ...) pid 14504 (conftest), jid 3, uid 0: exited on signal 11 pid 27466 (conftest), jid 6, uid 0: exited on signal 11 pid 43297 (conftest), jid 5, uid 0: exited on signal 11 lock order reversal: 1st 0xfe00bc68c030 filedesc structure (filedesc structure) @ /usr/src-13/sys/kern/sys_generic.c:1557 2nd 0xf803baeddbd8 tmpfs (tmpfs) @ /usr/src-13/sys/kern/vfs_vnops.c:1553 stack backtrace: #0 0x80c2d5f1 at witness_debugger+0x71 #1 0x80b946b5 at lockmgr_xlock+0x55 #2 0x80cae744 at _vn_lock+0x54 #3 0x80cad0da at vn_poll+0x3a #4 0x80c33e19 at kern_poll+0x419 #5 0x80c340df at sys_ppoll+0x6f #6 0x81060c40 at amd64_syscall+0x140 #7 0x810370a0 at fast_syscall_common+0x101 pid 37533 (conftest), jid 5, uid 0: exited on signal 11 pid 43474 (conftest), jid 5, uid 0: exited on signal 11 Poudriere doesn't really report any problems: # poudriere status SET PORTS JAIL BUILD STATUS QUEUE BUILT FAIL SKIP IGNORE REMAIN TIME LOGS kde5 gui 13 2020-05-01_10h17m52s parallel_build 2040 792 0 0 0 1248 22:48:00 /usr/local/poudriere/data/logs/bulk/13-gui-kde5/2020-05-01_10h17m52s Are those the debug messages that aren't visible on non-current kernel and should they be reported? GrzegorzJ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 32-bit powerpc head -r360311: lock order reversal between: "PROC (UMA zone)" and "kernelpmap (kernelpmap)": Is this expected?
On 2020-Apr-30, at 18:30, Mark Millard wrote: > Using artifact.ci's head -r360311 debug-kernel materials: > > https://artifact.ci.freebsd.org/snapshot/head/r360311/powerpc/powerpc/kernel*.txz > > I got the following notice: > > lock order reversal: > 1st 0x1cbb680 PROC (UMA zone) @ /usr/src/sys/vm/uma_core.c:4387 > 2nd 0x113c99c kernelpmap (kernelpmap) @ > /usr/src/sys/powerpc/aim/mmu_oea.c:1524 > stack backtrace: > #0 0x5d1e5c at witness_debugger+0x94 > #1 0x5d1b34 at witness_checkorder+0xb50 > #2 0x51d774 at __mtx_lock_flags+0xcc > #3 0x90902c at moea_kextract+0x5c > #4 0x9462ac at pmap_kextract+0x98 > #5 0x8a417c at zone_release+0xf0 > #6 0x8abc14 at bucket_drain+0x2f0 > #7 0x8ab64c at bucket_free+0x54 > #8 0x8ab8bc at bucket_cache_reclaim+0x1bc > #9 0x8ab3c4 at zone_reclaim+0x128 > #10 0x8a7e60 at uma_reclaim+0x1d0 > #11 0x8d96ac at vm_pageout_worker+0x4d8 > #12 0x8d91c0 at vm_pageout+0x1b0 > #13 0x4f67a0 at fork_exit+0xb0 > #14 0x94892c at fork_trampoline+0xc > > Is the above interesting or is it one of the > known-safe lock order reversals that should > be ignored? > > (The notice is from something like 4.5 hours > before I noticed it.) > While running kyua to see what it might run into . . . lock order reversal: 1st 0x1c34800 filedesc0 (UMA zone) @ /usr/src/sys/vm/uma_core.c:4387 2nd 0x113c99c kernelpmap (kernelpmap) @ /usr/src/sys/powerpc/aim/mmu_oea.c:1524 stack backtrace: #0 0x5d1e5c at witness_debugger+0x94 #1 0x5d1b34 at witness_checkorder+0xb50 #2 0x51d774 at __mtx_lock_flags+0xcc #3 0x90902c at moea_kextract+0x5c #4 0x9462ac at pmap_kextract+0x98 #5 0x8a417c at zone_release+0xf0 #6 0x8abc14 at bucket_drain+0x2f0 #7 0x8ab64c at bucket_free+0x54 #8 0x8ab8bc at bucket_cache_reclaim+0x1bc #9 0x8ab3c4 at zone_reclaim+0x128 #10 0x8a7d58 at uma_reclaim+0xc8 #11 0x656d24 at vnlru_proc+0x908 #12 0x4f67a0 at fork_exit+0xb0 #13 0x94892c at fork_trampoline+0xc witness_debugger through zone_reclaim look the same as the prior report. uma_reclaim has different associated figures. There is also: lock order reversal: 1st 0xfbed24 allprison (allprison) @ /usr/src/sys/kern/kern_jail.c:984 2nd 0x10706c4 vnet_sysinit_sxlock (vnet_sysinit_sxlock) @ /usr/src/sys/net/vnet.c:577 stack backtrace: #0 0x5d1e5c at witness_debugger+0x94 #1 0x5d1b34 at witness_checkorder+0xb50 #2 0x555300 at _sx_slock_int+0xa0 #3 0x555b10 at _sx_slock+0x28 #4 0x6b7d84 at vnet_alloc+0xf4 #5 0x4fd09c at kern_jail_set+0x1868 #6 0x4fe938 at sys_jail_set+0x70 #7 0x9492fc at trap+0x748 #8 0x93d1c0 at powerpc_interrupt+0x178 And: lock order reversal: 1st 0x106f5d8 ifnet_sx (ifnet_sx) @ /usr/src/sys/netinet/in.c:914 2nd 0x107071c in_control (in_control) @ /usr/src/sys/netinet/in.c:243 stack backtrace: #0 0x5d1e5c at witness_debugger+0x94 #1 0x5d1b34 at witness_checkorder+0xb50 #2 0x553ca4 at _sx_xlock+0x98 #3 0x6c45b8 at in_ifscrub_all+0xec #4 0x6dbbc4 at ip_destroy+0xb0 #5 0x6b81c0 at vnet_destroy+0x154 #6 0x4fefb0 at prison_deref+0x2cc #7 0x5007dc at prison_remove_one+0x148 #8 0x500658 at sys_jail_remove+0x2a4 #9 0x9492fc at trap+0x748 #10 0x93d1c0 at powerpc_interrupt+0x178 I also do not know about the below GEOM topology related lock order reversals . . . lock order reversal: 1st 0xfbca1c GEOM topology (GEOM topology) @ /usr/src/sys/geom/eli/g_eli.c:746 2nd 0xd49000 allproc (allproc) @ /usr/src/sys/kern/kern_fork.c:382 stack backtrace: #0 0x5d1e5c at witness_debugger+0x94 #1 0x5d1b34 at witness_checkorder+0xb50 #2 0x553ca4 at _sx_xlock+0x98 #3 0x4f4fb4 at fork1+0x7dc #4 0x5041c8 at kproc_create+0xd4 #5 0xdd27c390 at g_eli_create+0x774 #6 0xdd281048 at g_eli_config+0x23fc #7 0x499188 at g_ctl_req+0x154 #8 0x49e784 at g_run_events+0x194 #9 0x4a1580 at g_event_procbody+0x74 #10 0x4f67a0 at fork_exit+0xb0 #11 0x94892c at fork_trampoline+0xc lock order reversal: 1st 0xfbca1c GEOM topology (GEOM topology) @ /usr/src/sys/geom/eli/g_eli.c:746 2nd 0xd2baccc8 filedesc structure (filedesc structure) @ /usr/src/sys/kern/kern_descrip.c:2064 stack backtrace: #0 0x5d1e5c at witness_debugger+0x94 #1 0x5d1b34 at witness_checkorder+0xb50 #2 0x555300 at _sx_slock_int+0xa0 #3 0x555b10 at _sx_slock+0x28 #4 0x4dc128 at fdinit+0xe8 #5 0x4dc638 at fdcopy+0x68 #6 0x4f52ac at fork1+0xad4 #7 0x5041c8 at kproc_create+0xd4 #8 0xdd27c390 at g_eli_create+0x774 #9 0xdd281048 at g_eli_config+0x23fc #10 0x499188 at g_ctl_req+0x154 #11 0x49e784 at g_run_events+0x194 #12 0x4a1580 at g_event_procbody+0x74 #13 0x4f67a0 at fork_exit+0xb0 #14 0x94892c at fork_trampoline+0xc lock order reversal: 1st 0xfbca1c GEOM topology (GEOM topology) @ /usr/src/sys/geom/eli/g_eli.c:746 2nd 0xd49080 proctree (proctree) @ /usr/src/sys/kern/kern_fork.c:557 stack backtrace: #0 0x5d1e5c at witness_debugger+0x94 #1 0x5d1b34 at witness_checkorder+0xb50 #2 0x553ca4 at _sx_xlock+0x98 #3 0x4f5650 at fork1+0xe78 #4 0x5041c8 at kproc_create+0xd4 #5 0xdd27c3
32-bit powerpc head -r360311: lock order reversal between: "PROC (UMA zone)" and "kernelpmap (kernelpmap)": Is this expected?
Using artifact.ci's head -r360311 debug-kernel materials: https://artifact.ci.freebsd.org/snapshot/head/r360311/powerpc/powerpc/kernel*.txz I got the following notice: lock order reversal: 1st 0x1cbb680 PROC (UMA zone) @ /usr/src/sys/vm/uma_core.c:4387 2nd 0x113c99c kernelpmap (kernelpmap) @ /usr/src/sys/powerpc/aim/mmu_oea.c:1524 stack backtrace: #0 0x5d1e5c at witness_debugger+0x94 #1 0x5d1b34 at witness_checkorder+0xb50 #2 0x51d774 at __mtx_lock_flags+0xcc #3 0x90902c at moea_kextract+0x5c #4 0x9462ac at pmap_kextract+0x98 #5 0x8a417c at zone_release+0xf0 #6 0x8abc14 at bucket_drain+0x2f0 #7 0x8ab64c at bucket_free+0x54 #8 0x8ab8bc at bucket_cache_reclaim+0x1bc #9 0x8ab3c4 at zone_reclaim+0x128 #10 0x8a7e60 at uma_reclaim+0x1d0 #11 0x8d96ac at vm_pageout_worker+0x4d8 #12 0x8d91c0 at vm_pageout+0x1b0 #13 0x4f67a0 at fork_exit+0xb0 #14 0x94892c at fork_trampoline+0xc Is the above interesting or is it one of the known-safe lock order reversals that should be ignored? (The notice is from something like 4.5 hours before I noticed it.) === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal
On Mon, 26 Feb 2018 08:42:14 +0200 "Andriy Gapon" said On 26/02/2018 07:18, Jon Brawn wrote: > Wotcha! > > So, I’ve been using FreeBSD 12-CURRENT at various svn releases for a while > now, and I get quite a few “lock order reversal” dumps. The one I’ve got > on my screen at the moment is for ufs / bufwait / ufs: > > root@brax:/usr/src/stand # lock order reversal: > 1st 0xfd0003ec17e8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2602 > 2nd 0x410efa20 bufwait (bufwait) @ > /usr/src/sys/ufs/ffs/ffs_vnops.c:282 > 3rd 0xfd00b83ca7e8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2602 > stack backtrace: > #0 0x003b59d4 at witness_debugger+0x64 > #1 0x0032bd34 at __lockmgr_args+0x6ac > #2 0x005c6af0 at ffs_lock+0x88 > #3 0x00679eb0 at VOP_LOCK1_APV+0xac > #4 0x00426fa8 at _vn_lock+0x64 > #5 0x00417550 at vget+0x78 > #6 0x00409fdc at vfs_hash_get+0xec > #7 0x005c2b94 at ffs_vgetf+0x44 > #8 0x005b96a8 at softdep_sync_buf+0x9f4 > #9 0x005c7834 at ffs_syncvnode+0x26c > #10 0x005a1b5c at ffs_truncate+0x6b0 > #11 0x005ce3cc at ufs_direnter+0x778 > #12 0x005d64bc at ufs_makeinode+0x4b8 > #13 0x005d2b90 at ufs_create+0x38 > #14 0x00677168 at VOP_CREATE_APV+0xac > #15 0x0042691c at vn_open_cred+0x264 > #16 0x0041fc84 at kern_openat+0x208 > #17 0x0064b59c at do_el0_sync+0x8bc > > Is there something I should be doing to help debug these? IMO, no. Please ignore LORs involving "bufwait", "filedesc structure", "syncer" unless you experience any real problem (like a lock up). Or build a kernel without WITNESS (and FULL_BUF_TRACKING?) :-) --Chris -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal
On 26/02/2018 07:18, Jon Brawn wrote: > Wotcha! > > So, I’ve been using FreeBSD 12-CURRENT at various svn releases for a while > now, and I get quite a few “lock order reversal” dumps. The one I’ve got on > my screen at the moment is for ufs / bufwait / ufs: > > root@brax:/usr/src/stand # lock order reversal: > 1st 0xfd0003ec17e8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2602 > 2nd 0x410efa20 bufwait (bufwait) @ > /usr/src/sys/ufs/ffs/ffs_vnops.c:282 > 3rd 0xfd00b83ca7e8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2602 > stack backtrace: > #0 0x003b59d4 at witness_debugger+0x64 > #1 0x0032bd34 at __lockmgr_args+0x6ac > #2 0x005c6af0 at ffs_lock+0x88 > #3 0x00679eb0 at VOP_LOCK1_APV+0xac > #4 0x00426fa8 at _vn_lock+0x64 > #5 0x00417550 at vget+0x78 > #6 0x00409fdc at vfs_hash_get+0xec > #7 0x005c2b94 at ffs_vgetf+0x44 > #8 0x005b96a8 at softdep_sync_buf+0x9f4 > #9 0x005c7834 at ffs_syncvnode+0x26c > #10 0x005a1b5c at ffs_truncate+0x6b0 > #11 0x005ce3cc at ufs_direnter+0x778 > #12 0x005d64bc at ufs_makeinode+0x4b8 > #13 0x005d2b90 at ufs_create+0x38 > #14 0x00677168 at VOP_CREATE_APV+0xac > #15 0x0042691c at vn_open_cred+0x264 > #16 0x0041fc84 at kern_openat+0x208 > #17 0x0064b59c at do_el0_sync+0x8bc > > Is there something I should be doing to help debug these? IMO, no. Please ignore LORs involving "bufwait", "filedesc structure", "syncer" unless you experience any real problem (like a lock up). -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
lock order reversal
Wotcha! So, I’ve been using FreeBSD 12-CURRENT at various svn releases for a while now, and I get quite a few “lock order reversal” dumps. The one I’ve got on my screen at the moment is for ufs / bufwait / ufs: root@brax:/usr/src/stand # lock order reversal: 1st 0xfd0003ec17e8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2602 2nd 0x410efa20 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:282 3rd 0xfd00b83ca7e8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2602 stack backtrace: #0 0x003b59d4 at witness_debugger+0x64 #1 0x0032bd34 at __lockmgr_args+0x6ac #2 0x005c6af0 at ffs_lock+0x88 #3 0x00679eb0 at VOP_LOCK1_APV+0xac #4 0x00426fa8 at _vn_lock+0x64 #5 0x00417550 at vget+0x78 #6 0x00409fdc at vfs_hash_get+0xec #7 0x005c2b94 at ffs_vgetf+0x44 #8 0x005b96a8 at softdep_sync_buf+0x9f4 #9 0x005c7834 at ffs_syncvnode+0x26c #10 0x005a1b5c at ffs_truncate+0x6b0 #11 0x005ce3cc at ufs_direnter+0x778 #12 0x005d64bc at ufs_makeinode+0x4b8 #13 0x005d2b90 at ufs_create+0x38 #14 0x00677168 at VOP_CREATE_APV+0xac #15 0x0042691c at vn_open_cred+0x264 #16 0x0041fc84 at kern_openat+0x208 #17 0x0064b59c at do_el0_sync+0x8bc Is there something I should be doing to help debug these? Jon. smime.p7s Description: S/MIME cryptographic signature
lock order reversal another, repeatable...
Mar 24 14:25:16 kernel: lock order reversal: Mar 24 14:25:16 kernel: 1st 0xc09cf6dc ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1277 Mar 24 14:25:16 kernel: 2nd 0xc0100a30 devfs (devfs) @ /usr/src/sys/ufs/ffs/ffs_softdep.c:1908 Mar 24 14:25:16 kernel: stack backtrace: Mar 24 14:25:16 kernel: #0 0xb5c22421 at witness_debugger+0x81 Mar 24 14:25:16 kernel: #1 0xb5c22342 at witness_checkorder+0xd12 Mar 24 14:25:16 kernel: #2 0xb5b9b5d4 at __lockmgr_args+0xa64 Mar 24 14:25:16 kernel: #3 0xb5c784ad at vop_stdlock+0x4d Mar 24 14:25:16 kernel: #4 0xb618e7f7 at VOP_LOCK1_APV+0xd7 Mar 24 14:25:16 kernel: #5 0xb5c9c137 at _vn_lock+0xb7 Mar 24 14:25:16 kernel: #6 0xb5e9d648 at softdep_flushworklist+0x68 Mar 24 14:25:16 kernel: #7 0xb5ebc892 at ffs_sync+0x2b2 Mar 24 14:25:16 kernel: #8 0xb5c82955 at dounmount+0x6c5 Mar 24 14:25:16 kernel: #9 0xb5c82185 at sys_unmount+0x315 Mar 24 14:25:16 kernel: #10 0xb6155fa5 at syscall+0x3b5 Mar 24 14:25:16 kernel: #11 0xb6140ede at Xint0x80_syscall+0x2e umounting a sata filesystem, the above. Useful? aside: twice recently dropped to debugger upon shutdown. kb> [ or whatever ] any useful commands here? I thought 'st' but have not read about a procedure... That occured upon umount of a 2nd sata disk's filesystem(s). On a second note, ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal [ newbie ] report [2nd one> more of ]
On Wed, 22 Feb 2017 21:06:32 -0600, Benjamin Kaduk wrote: > Hi Jeffrey, > > Thank you for your enthusiasm in reporting these. > Unfortunately, it is very likely that these two are "well-known" and > believed to be harmless, so you have not discovered something > terribly exciting. > > An old and no-longer particularly maintained listing of these and > other LORs is at: http://sources.zabbadoz.net/freebsd/lor.html > > -Ben > > On Wed, Feb 22, 2017 at 06:20:08PM -0800, Jeffrey Bouquet wrote: > > This one at boot: > > #0 to #10 > > bufwait > > /usr/src/sys/kern/vfs_bio.c:3500 > > dirhash > > /usr/src/sys/ufs/ufs/ufs_dirhash.c:201 > > > > r313487 12.0-CURRENT Feb 13 2017 > > 1200020 FWIW > > both the above and the below reports... > > > > > > > > > > On Wed, 22 Feb 2017 15:37:21 -0800 (PST), "Jeffrey Bouquet" > > wrote: > > > > > #0 #16 follow: > > > jotted down : > > > > > > 1. ufs /usr/src/sys/kern/vfs_syscalls.c:3364 > > > 2. bufwait /usr/src/sys/ufs/ffs/ffs_vnops.c:280 > > > 3. ufs /usr/src/sys/kern/vfs_subr.c:2600 > > > > > > [ took roxterm out of the xinitrc, system stable seems more than > > > yesterday... too > > > early to tell, which is/was a 2nd issue... put in urxvt and st... based > > > on TOP memory... ] > > > ___ > > > freebsd-current@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > > > > > ___ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" Found a few more in a debug custom kernel that has 9h 54m uptime as of now, using nextboot, so are maybe not important... just a toy maybe to examine if anyone has free time and/or has less than 9h uptime issues to remove a few lock order reversals, in, say, amd64 (i386 here) if they are common to both but would crash amd64 more reliably than the good uptime I have as of this morning... r318487 Feb 13 2017 12.0-CURRENT 1200020 Thought to email them only because different 'subsystem' messages occur during the boot verbose process... like 2016, 2017, 2016, 2015... Maybe just newbie stuff. Thanks... ignore the 'speaker' stuff in it... Jeff kernel log messages: +subsystem 100 + vm_mem_init(0)... done. + vm_page_init(0)... done. +subsystem 180 + sysctl_register_all(0)... done. + mallocinit(0)... done. + malloc_init(&M_ACPICA)... done. + malloc_init(&M_KBDMUX)... done. + malloc_init(&M_LED)... done. + malloc_init(&M_MALODEV)... done. + malloc_init(&M_MD)... done. + malloc_init(&M_MDSECT)... done. + malloc_init(&M_MFIBUF)... done. + malloc_init(&M_MPR)... done. + malloc_init(&M_MPRSAS)... done. + malloc_init(&M_MPRUSER)... done. + malloc_init(&M_MPT2)... done. + malloc_init(&M_MPSSAS)... done. + malloc_init(&M_MPSUSER)... done. + malloc_init(&M_ACPITASK)... done. + malloc_init(&M_MPTUSER)... done. + malloc_init(&M_MRSAS)... done. + malloc_init(&M_ACPISEM)... done. + malloc_init(&M_CAMCCBQ)... done. + malloc_init(&M_ACPIDEV)... done. + malloc_init(&M_MVS)... done. + malloc_init(&M_MWLDEV)... done. + malloc_init(&M_NETMAP)... done. + malloc_init(&M_PPBUSDEV)... done. + malloc_init(&M_PSTIOP)... done. + malloc_init(&M_PSTRAID)... done. + malloc_init(&M_PUC)... done. + malloc_init(&M_CAMSIM)... done. + malloc_init(&M_ENTROPY)... done. + malloc_init(&M_CAMXPT)... done. + malloc_init(&M_CAMDEV)... done. + malloc_init(&M_CAMCCB)... done. + malloc_init(&M_CAMPATH)... done. + malloc_init(&M_SIIS)... done. + malloc_init(&M_CAMPERIPH)... done. + malloc_init(&M_SNP)... done. + malloc_init(&M_ACPICMBAT)... done. + malloc_init(&M_AC97)... done. + malloc_init(&M_FEEDER)... done. + malloc_init(&M_MIXER)... done. + malloc_init(&M_MIDI)... done. + malloc_init(&M_TWA)... done. + malloc_init(&M_TWE)... done. + malloc_init(&M_TWS)... done. + malloc_init(&M_ACPIPERF)... done. + malloc_init(&M_ACPIPWR)... done. + malloc_init(&M_CAMSCHED)... done. + malloc_init(&M_CAMQ)... done. + malloc_init(&M_SCSICD)... done. + malloc_init(&M_UART)... done. + malloc_init(&M_AGP)... done. + malloc_init(&M_AHCI)... done. + malloc_init(&M_USB)... done. + malloc_init(&M_USBDEV)... done. + malloc_init(&M_SCSICH)... done. + malloc_init(&M_ATADA)... done. + malloc_init(&M_CAMDEVQ)... done. + malloc_init(&M_SCSIDA)... done. + malloc_init(&M_SCSILOW)... done. + malloc_init(&M_AMR)... done. + malloc_init(&M_ATA)... done. + malloc_init(&M_ATADMA)... done.
Re: Lock order reversal [ newbie ] report [2nd one]
Hi Jeffrey, Thank you for your enthusiasm in reporting these. Unfortunately, it is very likely that these two are "well-known" and believed to be harmless, so you have not discovered something terribly exciting. An old and no-longer particularly maintained listing of these and other LORs is at: http://sources.zabbadoz.net/freebsd/lor.html -Ben On Wed, Feb 22, 2017 at 06:20:08PM -0800, Jeffrey Bouquet wrote: > This one at boot: > #0 to #10 > bufwait > /usr/src/sys/kern/vfs_bio.c:3500 > dirhash > /usr/src/sys/ufs/ufs/ufs_dirhash.c:201 > > r313487 12.0-CURRENT Feb 13 2017 > 1200020 FWIW > both the above and the below reports... > > > > > On Wed, 22 Feb 2017 15:37:21 -0800 (PST), "Jeffrey Bouquet" > wrote: > > > #0 #16 follow: > > jotted down : > > > > 1. ufs /usr/src/sys/kern/vfs_syscalls.c:3364 > > 2. bufwait /usr/src/sys/ufs/ffs/ffs_vnops.c:280 > > 3. ufs /usr/src/sys/kern/vfs_subr.c:2600 > > > > [ took roxterm out of the xinitrc, system stable seems more than > > yesterday... too > > early to tell, which is/was a 2nd issue... put in urxvt and st... based on > > TOP memory... ] > > ___ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal [ newbie ] report [2nd one]
This one at boot: #0 to #10 bufwait /usr/src/sys/kern/vfs_bio.c:3500 dirhash /usr/src/sys/ufs/ufs/ufs_dirhash.c:201 r313487 12.0-CURRENT Feb 13 2017 1200020 FWIW both the above and the below reports... On Wed, 22 Feb 2017 15:37:21 -0800 (PST), "Jeffrey Bouquet" wrote: > #0 #16 follow: > jotted down : > > 1. ufs /usr/src/sys/kern/vfs_syscalls.c:3364 > 2. bufwait /usr/src/sys/ufs/ffs/ffs_vnops.c:280 > 3. ufs /usr/src/sys/kern/vfs_subr.c:2600 > > [ took roxterm out of the xinitrc, system stable seems more than yesterday... > too > early to tell, which is/was a 2nd issue... put in urxvt and st... based on > TOP memory... ] > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Lock order reversal [ newbie ] report
#0 #16 follow: jotted down : 1. ufs /usr/src/sys/kern/vfs_syscalls.c:3364 2. bufwait /usr/src/sys/ufs/ffs/ffs_vnops.c:280 3. ufs /usr/src/sys/kern/vfs_subr.c:2600 [ took roxterm out of the xinitrc, system stable seems more than yesterday... too early to tell, which is/was a 2nd issue... put in urxvt and st... based on TOP memory... ] ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RE: New Lock Order Reversal in 12.0?
Good, that makes me feel better :) The system is running fine, so I think you're right and it's nothing to worry about. Thanks again for your responses. Best, Anindya From: Benjamin Kaduk [ka...@mit.edu] Sent: January 9, 2017 1:53 PM To: Anindya Mukherjee Cc: freebsd-current@freebsd.org Subject: Re: New Lock Order Reversal in 12.0? On Mon, Jan 09, 2017 at 02:31:39AM +, Anindya Mukherjee wrote: > Hi Ben, > > Thanks for your reply, and yes, I did notice #238. I should say at this point > that I'm a newbie when it comes to kernel code and am trying to learn about > it (hence current). Understood. It's good that you are ready to try! > The entry you refer to does look a bit like the one I've got, but I'm not > totally sure if the same code path is being followed to arrive at this LOR. > An inode is being created in my case, vs a directory creation (entry + inode > probably) in #238, and then a sync is being attempted, which causes locks to > activate in the softdep code. I've read a bit about this from McCusick's book > but the details are still fuzzy. > > Perhaps you are trying to tell me that it's benign? I know that WITNESS has > false positives, an example being #236 where due to shared vs exclusive vnode > locks required on the two ways to lock bufwait and dirhash a deadlock never > happens. Well, it's hard to say for certain, but there are a few ufs/bufwait/ufs LORs that are very commonly reported as WITNESS output but have never (IIRC) been identified as causing a deadlock. So, it seems pretty likely that this one is similarly benign -- I, for one, do not plan to put in time trying to analyze it until there is a hang or deadlock associated with it. -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New Lock Order Reversal in 12.0?
On Mon, Jan 09, 2017 at 02:31:39AM +, Anindya Mukherjee wrote: > Hi Ben, > > Thanks for your reply, and yes, I did notice #238. I should say at this point > that I'm a newbie when it comes to kernel code and am trying to learn about > it (hence current). Understood. It's good that you are ready to try! > The entry you refer to does look a bit like the one I've got, but I'm not > totally sure if the same code path is being followed to arrive at this LOR. > An inode is being created in my case, vs a directory creation (entry + inode > probably) in #238, and then a sync is being attempted, which causes locks to > activate in the softdep code. I've read a bit about this from McCusick's book > but the details are still fuzzy. > > Perhaps you are trying to tell me that it's benign? I know that WITNESS has > false positives, an example being #236 where due to shared vs exclusive vnode > locks required on the two ways to lock bufwait and dirhash a deadlock never > happens. Well, it's hard to say for certain, but there are a few ufs/bufwait/ufs LORs that are very commonly reported as WITNESS output but have never (IIRC) been identified as causing a deadlock. So, it seems pretty likely that this one is similarly benign -- I, for one, do not plan to put in time trying to analyze it until there is a hang or deadlock associated with it. -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RE: New Lock Order Reversal in 12.0?
Hi Ben, Thanks for your reply, and yes, I did notice #238. I should say at this point that I'm a newbie when it comes to kernel code and am trying to learn about it (hence current). The entry you refer to does look a bit like the one I've got, but I'm not totally sure if the same code path is being followed to arrive at this LOR. An inode is being created in my case, vs a directory creation (entry + inode probably) in #238, and then a sync is being attempted, which causes locks to activate in the softdep code. I've read a bit about this from McCusick's book but the details are still fuzzy. Perhaps you are trying to tell me that it's benign? I know that WITNESS has false positives, an example being #236 where due to shared vs exclusive vnode locks required on the two ways to lock bufwait and dirhash a deadlock never happens. Best, Anindya From: Benjamin Kaduk [ka...@mit.edu] Sent: January 8, 2017 4:47 PM To: Anindya Mukherjee Cc: freebsd-current@freebsd.org Subject: Re: New Lock Order Reversal in 12.0? On Mon, Jan 09, 2017 at 12:32:28AM +, Anindya Mukherjee wrote: > Hi, I'm running 12.0-current and noticed a LOR message from WITNESS which I > couldn't find a report about. I looked at > http://sources.zabbadoz.net/freebsd/lor.html, among other places. > > system details: > root@triskelion:~ # uname -a > FreeBSD triskelion 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r311461: Thu Jan 5 > 22:46:38 UTC 2017 > r...@releng3.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > root@triskelion:~ # freebsd-version > 12.0-CURRENT > > > WITNESS report: > lock order reversal: > 1st 0xf8002e8049a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2598 > 2nd 0xfe01e7ce9b40 bufwait (bufwait) @ > /usr/src/sys/ufs/ffs/ffs_vnops.c:277 > 3rd 0xf8002ec7b9a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2598 > stack backtrace: > #0 0x80aa6fd0 at witness_debugger+0x70 > #1 0x80aa6ed3 at witness_checkorder+0xde3 > #2 0x80a20c15 at __lockmgr_args+0x725 > #3 0x80d06fc5 at ffs_lock+0xa5 > #4 0x8101c0c0 at VOP_LOCK1_APV+0xe0 > #5 0x80b1a6aa at _vn_lock+0x9a > #6 0x80b0ac94 at vget+0x64 > #7 0x80afd19c at vfs_hash_get+0xcc > #8 0x80d02e5e at ffs_vgetf+0x3e > #9 0x80cf9787 at softdep_sync_buf+0xc37 > #10 0x80d07c51 at ffs_syncvnode+0x2a1 > #11 0x80d06e60 at ffs_fsync+0x20 > #12 0x8101b110 at VOP_FSYNC_APV+0xe0 > #13 0x80d0f2f0 at ufs_direnter+0x870 > #14 0x80d18050 at ufs_makeinode+0x5c0 > #15 0x80d13d7a at ufs_create+0x3a > #16 0x810199ca at VOP_CREATE_APV+0xda > #17 0x80b19f77 at vn_open_cred+0x2c7 > > This is based on the FreeBSD-12.0-CURRENT-amd64-20170105-r311461-memstick.img > installer. Known issue? You do not think it looks like http://sources.zabbadoz.net/freebsd/lor/238.html ? -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New Lock Order Reversal in 12.0?
On Mon, Jan 09, 2017 at 12:32:28AM +, Anindya Mukherjee wrote: > Hi, I'm running 12.0-current and noticed a LOR message from WITNESS which I > couldn't find a report about. I looked at > http://sources.zabbadoz.net/freebsd/lor.html, among other places. > > system details: > root@triskelion:~ # uname -a > FreeBSD triskelion 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r311461: Thu Jan 5 > 22:46:38 UTC 2017 > r...@releng3.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > root@triskelion:~ # freebsd-version > 12.0-CURRENT > > > WITNESS report: > lock order reversal: > 1st 0xf8002e8049a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2598 > 2nd 0xfe01e7ce9b40 bufwait (bufwait) @ > /usr/src/sys/ufs/ffs/ffs_vnops.c:277 > 3rd 0xf8002ec7b9a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2598 > stack backtrace: > #0 0x80aa6fd0 at witness_debugger+0x70 > #1 0x80aa6ed3 at witness_checkorder+0xde3 > #2 0x80a20c15 at __lockmgr_args+0x725 > #3 0x80d06fc5 at ffs_lock+0xa5 > #4 0x8101c0c0 at VOP_LOCK1_APV+0xe0 > #5 0x80b1a6aa at _vn_lock+0x9a > #6 0x80b0ac94 at vget+0x64 > #7 0x80afd19c at vfs_hash_get+0xcc > #8 0x80d02e5e at ffs_vgetf+0x3e > #9 0x80cf9787 at softdep_sync_buf+0xc37 > #10 0x80d07c51 at ffs_syncvnode+0x2a1 > #11 0x80d06e60 at ffs_fsync+0x20 > #12 0x8101b110 at VOP_FSYNC_APV+0xe0 > #13 0x80d0f2f0 at ufs_direnter+0x870 > #14 0x80d18050 at ufs_makeinode+0x5c0 > #15 0x80d13d7a at ufs_create+0x3a > #16 0x810199ca at VOP_CREATE_APV+0xda > #17 0x80b19f77 at vn_open_cred+0x2c7 > > This is based on the FreeBSD-12.0-CURRENT-amd64-20170105-r311461-memstick.img > installer. Known issue? You do not think it looks like http://sources.zabbadoz.net/freebsd/lor/238.html ? -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
New Lock Order Reversal in 12.0?
Hi, I'm running 12.0-current and noticed a LOR message from WITNESS which I couldn't find a report about. I looked at http://sources.zabbadoz.net/freebsd/lor.html, among other places. system details: root@triskelion:~ # uname -a FreeBSD triskelion 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r311461: Thu Jan 5 22:46:38 UTC 2017 r...@releng3.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 root@triskelion:~ # freebsd-version 12.0-CURRENT WITNESS report: lock order reversal: 1st 0xf8002e8049a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2598 2nd 0xfe01e7ce9b40 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:277 3rd 0xf8002ec7b9a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2598 stack backtrace: #0 0x80aa6fd0 at witness_debugger+0x70 #1 0x80aa6ed3 at witness_checkorder+0xde3 #2 0x80a20c15 at __lockmgr_args+0x725 #3 0x80d06fc5 at ffs_lock+0xa5 #4 0x8101c0c0 at VOP_LOCK1_APV+0xe0 #5 0x80b1a6aa at _vn_lock+0x9a #6 0x80b0ac94 at vget+0x64 #7 0x80afd19c at vfs_hash_get+0xcc #8 0x80d02e5e at ffs_vgetf+0x3e #9 0x80cf9787 at softdep_sync_buf+0xc37 #10 0x80d07c51 at ffs_syncvnode+0x2a1 #11 0x80d06e60 at ffs_fsync+0x20 #12 0x8101b110 at VOP_FSYNC_APV+0xe0 #13 0x80d0f2f0 at ufs_direnter+0x870 #14 0x80d18050 at ufs_makeinode+0x5c0 #15 0x80d13d7a at ufs_create+0x3a #16 0x810199ca at VOP_CREATE_APV+0xda #17 0x80b19f77 at vn_open_cred+0x2c7 This is based on the FreeBSD-12.0-CURRENT-amd64-20170105-r311461-memstick.img installer. Known issue? It happened during a portsnap fetch. the filesystem is UFS with default mount options. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal errors during boot
I see this LOR often too. I thought maybe this is happening because of my Virtual Box setting :) (?!?) On 5/21/16, Will Brokenbourgh wrote: > On 05/16/16 00:44, Bjoern A. Zeeb wrote: >>> On 15 May 2016, at 07:44 , Kurt Jaeger wrote: >>> >>> Hi! >>> >>>> I am seeing 'lock order reversal' errors during boot on 11-CURRENT - >>>> r298793. A sample error: >>>> >>>> lock order reversal: >>>> 1st 0xf8011280d068 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 >>>> 2nd 0xfe01ca539400 bufwait (bufwait) @ >>>> /usr/src/sys/ufs/ffs/ffs_vnops.c:263 >>>> 3rd 0xf80112a23b78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 >>>> stack backtrace: >>>> #0 0x80a94bf0 at witness_debugger+0x70 >>>> #1 0x80a94ae4 at witness_checkorder+0xe54 >>>> #2 0x80a0fdd6 at __lockmgr_args+0x4d6 >>>> #3 0x80ce9626 at ffs_lock+0xa6 >>>> [...snip...] >>>> >>>> I'm not sure if this is even something I should report, but better safe >>>> than sorry, yes? >>> Yes, and there's even a page to compare your LOR against other ones: >>> >>> http://sources.zabbadoz.net/freebsd/lor.html >>> >>> Can you try if you find your LOR there ? If not, can you add it >>> to that page ? >> No he can’t and I haven’t in years either. Searching bugs might however >> be a good idea; there is a LOR category or tag somewhere. >> >> /bz > Thank you for the replies. > > I'm getting these LORs pretty frequently. Is this a normal occurrence > for 11-CURRENT or is it only me having this problem? > > Thanks > > Will Brokenbourgh > > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal errors during boot
On 05/16/16 00:44, Bjoern A. Zeeb wrote: On 15 May 2016, at 07:44 , Kurt Jaeger wrote: Hi! I am seeing 'lock order reversal' errors during boot on 11-CURRENT - r298793. A sample error: lock order reversal: 1st 0xf8011280d068 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 2nd 0xfe01ca539400 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:263 3rd 0xf80112a23b78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 stack backtrace: #0 0x80a94bf0 at witness_debugger+0x70 #1 0x80a94ae4 at witness_checkorder+0xe54 #2 0x80a0fdd6 at __lockmgr_args+0x4d6 #3 0x80ce9626 at ffs_lock+0xa6 [...snip...] I'm not sure if this is even something I should report, but better safe than sorry, yes? Yes, and there's even a page to compare your LOR against other ones: http://sources.zabbadoz.net/freebsd/lor.html Can you try if you find your LOR there ? If not, can you add it to that page ? No he can’t and I haven’t in years either. Searching bugs might however be a good idea; there is a LOR category or tag somewhere. /bz Thank you for the replies. I'm getting these LORs pretty frequently. Is this a normal occurrence for 11-CURRENT or is it only me having this problem? Thanks Will Brokenbourgh ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal errors during boot
> On 15 May 2016, at 07:44 , Kurt Jaeger wrote: > > Hi! > >> I am seeing 'lock order reversal' errors during boot on 11-CURRENT - >> r298793. A sample error: >> >> lock order reversal: >> 1st 0xf8011280d068 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 >> 2nd 0xfe01ca539400 bufwait (bufwait) @ >> /usr/src/sys/ufs/ffs/ffs_vnops.c:263 >> 3rd 0xf80112a23b78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 >> stack backtrace: >> #0 0x80a94bf0 at witness_debugger+0x70 >> #1 0x80a94ae4 at witness_checkorder+0xe54 >> #2 0x80a0fdd6 at __lockmgr_args+0x4d6 >> #3 0x80ce9626 at ffs_lock+0xa6 >> [...snip...] >> >> I'm not sure if this is even something I should report, but better safe >> than sorry, yes? > > Yes, and there's even a page to compare your LOR against other ones: > > http://sources.zabbadoz.net/freebsd/lor.html > > Can you try if you find your LOR there ? If not, can you add it > to that page ? No he can’t and I haven’t in years either. Searching bugs might however be a good idea; there is a LOR category or tag somewhere. /bz ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal errors during boot
Hi! > I am seeing 'lock order reversal' errors during boot on 11-CURRENT - > r298793. A sample error: > > lock order reversal: > 1st 0xf8011280d068 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 > 2nd 0xfe01ca539400 bufwait (bufwait) @ > /usr/src/sys/ufs/ffs/ffs_vnops.c:263 > 3rd 0xf80112a23b78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 > stack backtrace: > #0 0x80a94bf0 at witness_debugger+0x70 > #1 0x80a94ae4 at witness_checkorder+0xe54 > #2 0x80a0fdd6 at __lockmgr_args+0x4d6 > #3 0x80ce9626 at ffs_lock+0xa6 > [...snip...] > > I'm not sure if this is even something I should report, but better safe > than sorry, yes? Yes, and there's even a page to compare your LOR against other ones: http://sources.zabbadoz.net/freebsd/lor.html Can you try if you find your LOR there ? If not, can you add it to that page ? -- p...@opsec.eu+49 171 3101372 4 years to go ! ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Lock order reversal errors during boot
Greetings, My first post here so please be gentle. I am seeing 'lock order reversal' errors during boot on 11-CURRENT - r298793. A sample error: lock order reversal: 1st 0xf8011280d068 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 2nd 0xfe01ca539400 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:263 3rd 0xf80112a23b78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498 stack backtrace: #0 0x80a94bf0 at witness_debugger+0x70 #1 0x80a94ae4 at witness_checkorder+0xe54 #2 0x80a0fdd6 at __lockmgr_args+0x4d6 #3 0x80ce9626 at ffs_lock+0xa6 [...snip...] I'm not sure if this is even something I should report, but better safe than sorry, yes? I'm attaching the full dmesg for more information. Sincerely, Will Brokenbourgh Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #0 r298793: Fri Apr 29 20:48:00 UTC 2016 r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) WARNING: WITNESS option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz (3300.06-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3 Features=0xbfebfbff Features2=0x7ffafbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x27ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 8589934592 (8192 MB) avail memory = 7625789440 (7272 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: unblocking device. ioapic0 irqs 0-23 on motherboard random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module module_register_init: MOD_LOAD (vesa, 0x80f10fb0, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 hpet0: iomem 0xfed0-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 550 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 Event timer "HPET3" frequency 14318180 Hz quality 440 Event timer "HPET4" frequency 14318180 Hz quality 440 atrtc0: port 0x70-0x77 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 vgapci0: port 0xf000-0xf03f mem 0xf780-0xf7bf,0xe000-0xefff irq 16 at device 2.0 on pci0 vgapci0: Boot video device hdac0: mem 0xf7d04000-0xf7d07fff irq 16 at device 3.0 on pci0 pci0: at device 22.0 (no driver attached) ehci0: mem 0xf7d0b000-0xf7d0b3ff irq 16 at device 26.0 on pci0 usbus0: EHCI version 1.0 usbus0 on ehci0 hdac1: mem 0xf7d0-0xf7d03fff irq 22 at device 27.0 on pci0 pcib1: irq 16 at device 28.0 on pci0 pci1: on pcib1 pcib2: irq 18 at device 28.2 on pci0 pci2: on pcib2 re0: port 0xe000-0xe0ff mem 0xf7c0-0xf7c00fff,0xf000-0xf0003fff irq 18 at device 0.0 on pci2 re0: Using 1 MSI-X message re0: ASPM disabled re0: Chip rev. 0x4c00 re0: MAC rev. 0x miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re0: Using defaults for TSO: 65518/35/2048 re0: Ethernet address: 40:8d:5c:38:04:9d re0: netmap queues/slots: TX 1/256, RX 1/256 pcib3: irq 19 at device 28.3 on pci0 pci3: on pcib3 pcib4: irq 19 at device 0.0 on pci3 pci4: on pcib4 ehci1: mem 0xf7d0a000-0xf7d0a3ff irq 23 at device 29.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci1 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xf7d09000-0xf7d097ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2:
Re: Lock order reversal in CURRENT
On Wed, 24 Feb 2016, Eax Melanhovich wrote: > Hello. > > Yesterday I built a kernel and world using master branch (6a8922d3) of > FreeBSD GitHub repository (a mirror of SVN as I understand?). Today I > got a "lock order reversal" report: > > http://i.imgur.com/jDZ4A3O.png > > According to this FAQ it could be a sign of a deadlock: > > https://www.freebsd.org/doc/faq/troubleshoot.html#idp63366608 > > Do I have to report this issue anywhere except freebsd-current@ ? That particular one, though not in the table at http://sources.zabbadoz.net/freebsd/lor.html, seems similar enough to a ufs/devfs entry that shows up very often and is harmless, to make me think that just the mail you have sent is quite sufficient. Thanks, Ben Kaduk ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Lock order reversal in CURRENT
Hello. Yesterday I built a kernel and world using master branch (6a8922d3) of FreeBSD GitHub repository (a mirror of SVN as I understand?). Today I got a "lock order reversal" report: http://i.imgur.com/jDZ4A3O.png According to this FAQ it could be a sign of a deadlock: https://www.freebsd.org/doc/faq/troubleshoot.html#idp63366608 Do I have to report this issue anywhere except freebsd-current@ ? -- Best regards, Eax Melanhovich http://eax.me/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ufs/devfs "lock order reversal" on poweroff
On Wed, Feb 18, 2015 at 9:53 AM, Damjan Jovanovic wrote: > Hi > > On r278909 (and probably earlier) I get the following when I run > "poweroff" (retyped from a video of it I had to record, since it > disappears very quickly): Hi Damjan, This is a known LOR. Thanks! ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
ufs/devfs "lock order reversal" on poweroff
Hi On r278909 (and probably earlier) I get the following when I run "poweroff" (retyped from a video of it I had to record, since it disappears very quickly): Syncing disks, vnodes remaining...4 1 0 0 done All buffers synced. lock order reversal: 1st 0xf80014d4d060 ufs (ufs) 0 /usr/src/sys/kern/vfs_mount.c:1229 2nd 0xf00014a695f0 devfs (devfs) 0 /usr/src/sys/kern/vfs_subr.c:2176 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame ... witness_checkorder() at witness_checkorder+... __lockmgr_args() at __lockmgr_args+... vop_stdlock() at vop_stdlock+0x3c/frame .. VOP_LOOCK1_AVP() _vm_lock() vget() devfs_allocv() devfs_root() dounmount() vfs_unmountall() kern_reboot() sys_reboot() amd64_syscall() Xfast_syscall() --- syscall (55, FreeBSD ELF64, sys_reboot), ript = 0x40fd1c, rsp = ..., rbp= ... Uptime: ... Thank you Damjan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: UFS lock order reversal stack trace with r264677 on i386
On Sat, 19 Apr 2014, R. Tyler Croy wrote: I've noticed this as of late on my i386 -CURRENT Thinkpad T43 when I perform some file operations, but an exact reproduction case I've not yet stumbled upon: Apr 20 01:29:32 lemon kernel: lock order reversal: Apr 20 01:29:32 lemon kernel: 1st 0xc5832358 bufwait (bufwait) @ /usr/home/tyler/source/github/freebsd/sys/kern/vfs_bio.c:3081 Apr 20 01:29:32 lemon kernel: 2nd 0xc6f1d600 dirhash (dirhash) @ /usr/home/tyler/source/github/freebsd/sys/ufs/ufs/ufs_dirhash.c:284 [...] Apr 20 01:29:54 lemon kernel: lock order reversal: Apr 20 01:29:54 lemon kernel: 1st 0xc699e388 ufs (ufs) @ /usr/home/tyler/source/github/freebsd/sys/kern/vfs_subr.c:2101 Apr 20 01:29:54 lemon kernel: 2nd 0xc5859e98 bufwait (bufwait) @ /usr/home/tyler/source/github/freebsd/sys/ufs/ffs/ffs_vnops.c:262 Apr 20 01:29:54 lemon kernel: 3rd 0xc7d27c68 ufs (ufs) @ /usr/home/tyler/source/github/freebsd/sys/kern/vfs_subr.c:2101 These are "well known", #261 and #285 at http://sources.zabbadoz.net/freebsd/lor.html . -Ben Kaduk ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
UFS lock order reversal stack trace with r264677 on i386
I've noticed this as of late on my i386 -CURRENT Thinkpad T43 when I perform some file operations, but an exact reproduction case I've not yet stumbled upon: Apr 20 01:29:32 lemon kernel: lock order reversal: Apr 20 01:29:32 lemon kernel: 1st 0xc5832358 bufwait (bufwait) @ /usr/home/tyler/source/github/freebsd/sys/kern/vfs_bio.c:3081 Apr 20 01:29:32 lemon kernel: 2nd 0xc6f1d600 dirhash (dirhash) @ /usr/home/tyler/source/github/freebsd/sys/ufs/ufs/ufs_dirhash.c:284 Apr 20 01:29:32 lemon kernel: KDB: stack backtrace: Apr 20 01:29:32 lemon kernel: db_trace_self_wrapper(c115aa9d,79732f64,66752f73,66752f73,66752f73,...) at db_trace_self_wrapper+0x2d/frame 0xe93ba918 Apr 20 01:29:32 lemon kernel: kdb_backtrace(c115e882,c6f1d600,c1192fe7,c5dae950,c1192c15,...) at kdb_backtrace+0x30/frame 0xe93ba980 Apr 20 01:29:32 lemon kernel: witness_checkorder(c6f1d600,9,c1192c15,11c,0,...) at witness_checkorder+0xd04/frame 0xe93ba9cc Apr 20 01:29:32 lemon kernel: _sx_xlock(c6f1d600,0,c1192c15,11c,c5832300,...) at _sx_xlock+0x75/frame 0xe93ba9fc Apr 20 01:29:32 lemon kernel: ufsdirhash_remove(c6c26bc8,db6e02fc,2fc,e93baa58,e93baa54,...) at ufsdirhash_remove+0x37/frame 0xe93baa24 Apr 20 01:29:32 lemon kernel: ufs_dirremove(c6c1f238,c73c0d98,400800c,0,c6c1f238,...) at ufs_dirremove+0x12c/frame 0xe93baa68 Apr 20 01:29:32 lemon kernel: ufs_remove(e93bac00,c11683ae,a8b,e93babfc,0,...) at ufs_remove+0x76/frame 0xe93baaac Apr 20 01:29:32 lemon kernel: VOP_REMOVE_APV(c13e8d1c,e93bac00,c73c2e6c,e93babd4,81a5b18,...) at VOP_REMOVE_APV+0xfe/frame 0xe93baad8 Apr 20 01:29:32 lemon kernel: kern_unlinkat(c6958000,ff9c,81a5b18,0,0) at kern_unlinkat+0x27a/frame 0xe93bac24 Apr 20 01:29:32 lemon kernel: sys_unlink(c6958000,e93bacc8,c0feeaef,c1c25c90,0,...) at sys_unlink+0x32/frame 0xe93bac40 Apr 20 01:29:32 lemon kernel: syscall(e93bad08) at syscall+0x30c/frame 0xe93bacfc Apr 20 01:29:32 lemon kernel: Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xe93bacfc Apr 20 01:29:32 lemon kernel: --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0x2848bd37, esp = 0xbfbfd1bc, ebp = 0xbfbfd248 --- Apr 20 01:29:54 lemon kernel: lock order reversal: Apr 20 01:29:54 lemon kernel: 1st 0xc699e388 ufs (ufs) @ /usr/home/tyler/source/github/freebsd/sys/kern/vfs_subr.c:2101 Apr 20 01:29:54 lemon kernel: 2nd 0xc5859e98 bufwait (bufwait) @ /usr/home/tyler/source/github/freebsd/sys/ufs/ffs/ffs_vnops.c:262 Apr 20 01:29:54 lemon kernel: 3rd 0xc7d27c68 ufs (ufs) @ /usr/home/tyler/source/github/freebsd/sys/kern/vfs_subr.c:2101 Apr 20 01:29:54 lemon kernel: KDB: stack backtrace: Apr 20 01:29:54 lemon kernel: db_trace_self_wrapper(c115aa9d,762f6e72,735f7366,2e726275,31323a63,...) at db_trace_self_wrapper+0x2d/frame 0xe93ba260 Apr 20 01:29:54 lemon kernel: kdb_backtrace(c115e89b,c7d27c68,c1144d4e,c5dae8e8,c11683ae,...) at kdb_backtrace+0x30/frame 0xe93ba2c4 Apr 20 01:29:54 lemon kernel: witness_checkorder(c7d27c68,9,c11683ae,835,c7d27c88,...) at witness_checkorder+0xd04/frame 0xe93ba310 Apr 20 01:29:54 lemon kernel: __lockmgr_args(c7d27c68,80100,c7d27c88,0,0,...) at __lockmgr_args+0x8f3/frame 0xe93ba3f0 Apr 20 01:29:54 lemon kernel: ffs_lock(e93ba470,c116537f,c5d8d110,c5d93840,c5d8d110,...) at ffs_lock+0x87/frame 0xe93ba42c Apr 20 01:29:54 lemon kernel: VOP_LOCK1_APV(c13e8d1c,e93ba470,c116537f,227,c13fe1f0,...) at VOP_LOCK1_APV+0x10a/frame 0xe93ba458 Apr 20 01:29:54 lemon kernel: _vn_lock(c7d27c34,80100,c11683ae,835,c11675a0,...) at Apr 20 01:29:54 lemon kernel: _vn_lock+0xa6/frame 0xe93ba498 Apr 20 01:29:54 lemon kernel: vget(c7d27c34,80100,c6958000,57,0,...) at vget+0x74/frame 0xe93ba4d0 Apr 20 01:29:54 lemon kernel: vfs_hash_get(c63fad20,70aa44,8,c6958000,e93ba5d0,...) at vfs_hash_get+0xfc/frame 0xe93ba4fc Apr 20 01:29:54 lemon kernel: ffs_vgetf(c63fad20,70aa44,8,e93ba5d0,1,...) at ffs_vgetf+0x44/frame 0xe93ba558 Apr 20 01:29:54 lemon kernel: softdep_sync_buf(c699e354,c5859e40,1,0,0,...) at softdep_sync_buf+0xbdf/frame 0xe93ba5e8 Apr 20 01:29:54 lemon kernel: ffs_syncvnode(c699e354,1,0,c13c3cdc,0,...) at ffs_syncvnode+0x2dd/frame 0xe93ba640 Apr 20 01:29:54 lemon kernel: ffs_truncate(c699e354,200,0,880,c5ed0300,...) at ffs_truncate+0x6eb/frame 0xe93ba Apr 20 01:29:54 lemon kernel: 7f0 Apr 20 01:29:54 lemon kernel: ufs_direnter(c699e354,c7d27c34,e93ba8b8,e93babcc,0,...) at ufs_direnter+0x79e/fra Apr 20 01:29:54 lemon kernel: me 0xe93ba870 Apr 20 01:29:54 lemon kernel: ufs_makeinode(e93babb8,e93babcc) at Apr 20 01:29:54 lemon kernel: ufs_makeinode+0x534/frame 0xe93ba9f0 Apr 20 01:29:54 lemon kernel: ufs_create(e93baad8,614,c63fad30,2,c63fad74,...) at Apr 20 01:29:54 lemon kernel: ufs_create+0x2f/frame 0xe93baa04 Apr 20 01:29:54 lemon kernel: VOP_CREATE_APV(c13e8d1c,e93baad8,e93babcc,e93baa68,c0acc930,...) at VOP_CREATE_APV+0xfe/frame 0xe93baa30 Apr 20 01:29:55 lemon kernel: vn_open_cred(e93bab70,e93babfc,180,0,c5ed0300,c6960118) at vn_open_cred+0x2f0/frame 0xe93bab00 Apr 20 01:2
10-CURRENT - LOR (lock order reversal)
Hi, these represent machine lockups, if not false positives. http://www.freebsd.org/cgi/query-pr.cgi?pr=182139&cat= jb ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Witness message about lock order reversal on 10 (head)
It might be the same false positive I saw a couple weeks ago ... Davide said to me: | The LOR is a false positive. | See the comment in sys/ufs/ufs/ufs_dirhash.c | Also, switching motherboards is not related to this in any way. You'll | eventually hit that LOR report, unless you disabled WITNESS. Thanks, -- Davide On Mon, 19 Aug 2013, Yuri wrote: I got these messages on 10 head, rev.254235, during 'filesystem full' condition. Yuri = log ===== lock order reversal: 1st 0xff80f7432470 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3054 2nd 0xfe00075b5600 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff8000284440 kdb_backtrace() at kdb_backtrace+0x39/frame 0xff80002844f0 witness_checkorder() at witness_checkorder+0xd4f/frame 0xff8000284580 _sx_xlock() at _sx_xlock+0x75/frame 0xff80002845c0 ufsdirhash_add() at ufsdirhash_add+0x3b/frame 0xff8000284600 ufs_direnter() at ufs_direnter+0x688/frame 0xff80002846c0 ufs_mkdir() at ufs_mkdir+0x863/frame 0xff80002848c0 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xf0/frame 0xff80002848f0 kern_mkdirat() at kern_mkdirat+0x21a/frame 0xff8000284ae0 amd64_syscall() at amd64_syscall+0x265/frame 0xff8000284bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xff8000284bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x800931e9a, rsp = 0x7fffd798, rbp = 0x7fffdc70 --- lock order reversal: 1st 0xfe0007633840 filedesc structure (filedesc structure) @ /usr/src/sys/kern/kern_descrip.c:1184 2nd 0xfe0007a45240 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:4346 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff800031f6b0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xff800031f760 witness_checkorder() at witness_checkorder+0xd4f/frame 0xff800031f7f0 __lockmgr_args() at __lockmgr_args+0x6f2/frame 0xff800031f920 ffs_lock() at ffs_lock+0x84/frame 0xff800031f970 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xff800031f9a0 _vn_lock() at _vn_lock+0xab/frame 0xff800031fa10 knlist_remove_kq() at knlist_remove_kq+0x82/frame 0xff800031fa40 knote_fdclose() at knote_fdclose+0xc8/frame 0xff800031fa90 closefp() at closefp+0x64/frame 0xff800031fae0 amd64_syscall() at amd64_syscall+0x265/frame 0xff800031fbf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xff800031fbf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x80164537a, rsp = 0x7f7fbf08, rbp = 0x7f7fbf20 --- pid 983 (sendmail), uid 25 inumber 473785 on /: filesystem full pid 1101 (dd), uid 2 inumber 426338 on /: filesystem full lock order reversal: 1st 0xfe0007cbc240 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099 2nd 0xff80f7894338 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:262 3rd 0xfe003b531418 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff8000378e60 kdb_backtrace() at kdb_backtrace+0x39/frame 0xff8000378f10 witness_checkorder() at witness_checkorder+0xd4f/frame 0xff8000378fa0 __lockmgr_args() at __lockmgr_args+0x6f2/frame 0xff80003790d0 ffs_lock() at ffs_lock+0x84/frame 0xff8000379120 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xff8000379150 _vn_lock() at _vn_lock+0xab/frame 0xff80003791c0 vget() at vget+0x70/frame 0xff8000379210 vfs_hash_get() at vfs_hash_get+0xf5/frame 0xff8000379260 ffs_vgetf() at ffs_vgetf+0x41/frame 0xff80003792f0 softdep_sync_buf() at softdep_sync_buf+0x2e4/frame 0xff80003793a0 ffs_syncvnode() at ffs_syncvnode+0x258/frame 0xff8000379420 ffs_truncate() at ffs_truncate+0x5ca/frame 0xff8000379600 ufs_direnter() at ufs_direnter+0x891/frame 0xff80003796c0 ufs_mkdir() at ufs_mkdir+0x863/frame 0xff80003798c0 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xf0/frame 0xff80003798f0 kern_mkdirat() at kern_mkdirat+0x21a/frame 0xff8000379ae0 amd64_syscall() at amd64_syscall+0x265/frame 0xff8000379bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xff8000379bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x800931e9a, rsp = 0x7fffd898, rbp = 0x7fffd970 --- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" dan -- Dan Mack ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Witness message about lock order reversal on 10 (head)
I got these messages on 10 head, rev.254235, during 'filesystem full' condition. Yuri = log = lock order reversal: 1st 0xff80f7432470 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3054 2nd 0xfe00075b5600 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff8000284440 kdb_backtrace() at kdb_backtrace+0x39/frame 0xff80002844f0 witness_checkorder() at witness_checkorder+0xd4f/frame 0xff8000284580 _sx_xlock() at _sx_xlock+0x75/frame 0xff80002845c0 ufsdirhash_add() at ufsdirhash_add+0x3b/frame 0xff8000284600 ufs_direnter() at ufs_direnter+0x688/frame 0xff80002846c0 ufs_mkdir() at ufs_mkdir+0x863/frame 0xff80002848c0 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xf0/frame 0xff80002848f0 kern_mkdirat() at kern_mkdirat+0x21a/frame 0xff8000284ae0 amd64_syscall() at amd64_syscall+0x265/frame 0xff8000284bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xff8000284bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x800931e9a, rsp = 0x7fffd798, rbp = 0x7fffdc70 --- lock order reversal: 1st 0xfe0007633840 filedesc structure (filedesc structure) @ /usr/src/sys/kern/kern_descrip.c:1184 2nd 0xfe0007a45240 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:4346 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff800031f6b0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xff800031f760 witness_checkorder() at witness_checkorder+0xd4f/frame 0xff800031f7f0 __lockmgr_args() at __lockmgr_args+0x6f2/frame 0xff800031f920 ffs_lock() at ffs_lock+0x84/frame 0xff800031f970 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xff800031f9a0 _vn_lock() at _vn_lock+0xab/frame 0xff800031fa10 knlist_remove_kq() at knlist_remove_kq+0x82/frame 0xff800031fa40 knote_fdclose() at knote_fdclose+0xc8/frame 0xff800031fa90 closefp() at closefp+0x64/frame 0xff800031fae0 amd64_syscall() at amd64_syscall+0x265/frame 0xff800031fbf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xff800031fbf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x80164537a, rsp = 0x7f7fbf08, rbp = 0x7f7fbf20 --- pid 983 (sendmail), uid 25 inumber 473785 on /: filesystem full pid 1101 (dd), uid 2 inumber 426338 on /: filesystem full lock order reversal: 1st 0xfe0007cbc240 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099 2nd 0xff80f7894338 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:262 3rd 0xfe003b531418 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff8000378e60 kdb_backtrace() at kdb_backtrace+0x39/frame 0xff8000378f10 witness_checkorder() at witness_checkorder+0xd4f/frame 0xff8000378fa0 __lockmgr_args() at __lockmgr_args+0x6f2/frame 0xff80003790d0 ffs_lock() at ffs_lock+0x84/frame 0xff8000379120 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xff8000379150 _vn_lock() at _vn_lock+0xab/frame 0xff80003791c0 vget() at vget+0x70/frame 0xff8000379210 vfs_hash_get() at vfs_hash_get+0xf5/frame 0xff8000379260 ffs_vgetf() at ffs_vgetf+0x41/frame 0xff80003792f0 softdep_sync_buf() at softdep_sync_buf+0x2e4/frame 0xff80003793a0 ffs_syncvnode() at ffs_syncvnode+0x258/frame 0xff8000379420 ffs_truncate() at ffs_truncate+0x5ca/frame 0xff8000379600 ufs_direnter() at ufs_direnter+0x891/frame 0xff80003796c0 ufs_mkdir() at ufs_mkdir+0x863/frame 0xff80003798c0 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xf0/frame 0xff80003798f0 kern_mkdirat() at kern_mkdirat+0x21a/frame 0xff8000379ae0 amd64_syscall() at amd64_syscall+0x265/frame 0xff8000379bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xff8000379bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x800931e9a, rsp = 0x7fffd898, rbp = 0x7fffd970 --- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
lock order reversal in r252098
Hello- I am running 10.0-CURRENT #0 r252098 on a beaglebone and I am getting the occasional LOR when reading/writing in an nfs mount lock order reversal: 1st 0xc1c0a150 newnfs (newnfs) @ /usr/src/sys/kern/vfs_syscalls.c:4064 2nd 0xc6a5e278 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2311 3rd 0xc1c12150 newnfs (newnfs) @ /usr/src/sys/kern/vfs_subr.c:2099 KDB: stack backtrace: end() at 0xd52035b0 scp=0xd52035b0 rlv=0xc05108e0 (db_trace_self+0x14) rsp=0xd5203498 rfp=0xc057394f Bad frame pointer: 0xc057394f Shortly after, the beaglebone locks up and requires a power cycle If anyone has any hints or can spare some cycles to help me debug this issue, please let me know Many Thanks ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal in sys/kern/vfs_mount.c
John Baldwin wrote on 08.06.2012 18:45: On Friday, June 08, 2012 4:21:34 am Ruslan Mahmatkhanov wrote: Ruslan Mahmatkhanov wrote on 08.06.2012 12:10: Good day, After updating to yesterdays -current, I got this on boot: lock order reversal: 1st 0xfe0007b04c38 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1254 2nd 0xfe0007ed9478 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2158 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2c witness_checkorder() at witness_checkorder+0x853 __lockmgr_args() at __lockmgr_args+0x113a vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xbf _vn_lock() at _vn_lock+0x47 vget() at vget+0x7b devfs_allocv() at devfs_allocv+0x13f devfs_root() at devfs_root+0x4d dounmount() at dounmount+0x45c vfs_unmountall() at vfs_unmountall+0x4c kern_reboot() at kern_reboot+0x84b sys_reboot() at sys_reboot+0x68 amd64_syscall() at amd64_syscall+0x2e0 Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (55, FreeBSD ELF64, sys_reboot), rip = 0x40ebbc, rsp = 0x7fffd6c8, rbp = 0x65 --- Reverting to old kernel (that was built about a week ago) helped to avoid this. Any thoughts? And this one comes up with the recent update too: driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) driver bug: Unable to set devclass (class: acpi_timer devname: (unknown)) cpu0: on acpi0 driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) cpu1: on acpi0 driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) cpu2: on acpi0 driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) cpu3: on acpi0 What the additional info should I supply to understand what's wrong? Here is my full dmesg; http://people.freebsd.org/~rm/dmesg.txt Is this a verbose boot? Also, can you try this to get more details in the log message: Sorry for delay. This particular panic had triggered by intel video driver (x11-drivers/xf86-video-intel v 2.19.0). After I rebuild it with new kernel, all is going fine. The panic appeared after GDM login, and I was able to catch some driver-related errors in xorg log. Thanks John. -- Regards, Ruslan Tinderboxing kills... the drives. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal in sys/kern/vfs_mount.c
Am 08.06.2012 10:10, schrieb Ruslan Mahmatkhanov: > Good day, > > After updating to yesterdays -current, I got this on boot: > > lock order reversal: > 1st 0xfe0007b04c38 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1254 > 2nd 0xfe0007ed9478 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2158 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > kdb_backtrace() at kdb_backtrace+0x37 > _witness_debugger() at _witness_debugger+0x2c > witness_checkorder() at witness_checkorder+0x853 > __lockmgr_args() at __lockmgr_args+0x113a > vop_stdlock() at vop_stdlock+0x39 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xbf > _vn_lock() at _vn_lock+0x47 > vget() at vget+0x7b > devfs_allocv() at devfs_allocv+0x13f > devfs_root() at devfs_root+0x4d > dounmount() at dounmount+0x45c > vfs_unmountall() at vfs_unmountall+0x4c > kern_reboot() at kern_reboot+0x84b > sys_reboot() at sys_reboot+0x68 > amd64_syscall() at amd64_syscall+0x2e0 > Xfast_syscall() at Xfast_syscall+0xf7 > --- syscall (55, FreeBSD ELF64, sys_reboot), rip = 0x40ebbc, rsp = > 0x7fffd6c8, rbp = 0x65 --- > > Reverting to old kernel (that was built about a week ago) helped to > avoid this. Any thoughts? I have been getting those for a long time in -CURRENT, but with ZFS instead of UFS, and during system shutdown, not startup. And this LOR seems to cause the system to lock-up on shutdown with relatively high probability (I often have to remove power to halt the system, Since the ACPI power-off is not issued because of the LOR). My system is only rebooted from the local console and thus the LOR is not that much of a problem, and I did not bother to further look into the cause. But in case of remote system that is rebooted via SSH, the LOR could be quite a nuisance ... Regards, STefan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal in sys/kern/vfs_mount.c
On Friday, June 08, 2012 4:21:34 am Ruslan Mahmatkhanov wrote: > Ruslan Mahmatkhanov wrote on 08.06.2012 12:10: > > Good day, > > > > After updating to yesterdays -current, I got this on boot: > > > > lock order reversal: > > 1st 0xfe0007b04c38 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1254 > > 2nd 0xfe0007ed9478 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2158 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > > kdb_backtrace() at kdb_backtrace+0x37 > > _witness_debugger() at _witness_debugger+0x2c > > witness_checkorder() at witness_checkorder+0x853 > > __lockmgr_args() at __lockmgr_args+0x113a > > vop_stdlock() at vop_stdlock+0x39 > > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xbf > > _vn_lock() at _vn_lock+0x47 > > vget() at vget+0x7b > > devfs_allocv() at devfs_allocv+0x13f > > devfs_root() at devfs_root+0x4d > > dounmount() at dounmount+0x45c > > vfs_unmountall() at vfs_unmountall+0x4c > > kern_reboot() at kern_reboot+0x84b > > sys_reboot() at sys_reboot+0x68 > > amd64_syscall() at amd64_syscall+0x2e0 > > Xfast_syscall() at Xfast_syscall+0xf7 > > --- syscall (55, FreeBSD ELF64, sys_reboot), rip = 0x40ebbc, rsp = > > 0x7fffd6c8, rbp = 0x65 --- > > > > Reverting to old kernel (that was built about a week ago) helped to > > avoid this. Any thoughts? > > And this one comes up with the recent update too: > > driver bug: Unable to set devclass (class: acpi_sysresource devname: > (unknown)) > driver bug: Unable to set devclass (class: acpi_timer devname: (unknown)) > cpu0: on acpi0 > driver bug: Unable to set devclass (class: acpi_sysresource devname: > (unknown)) > cpu1: on acpi0 > driver bug: Unable to set devclass (class: acpi_sysresource devname: > (unknown)) > cpu2: on acpi0 > driver bug: Unable to set devclass (class: acpi_sysresource devname: > (unknown)) > cpu3: on acpi0 > > What the additional info should I supply to understand what's wrong? > Here is my full dmesg; http://people.freebsd.org/~rm/dmesg.txt Is this a verbose boot? Also, can you try this to get more details in the log message: Index: subr_bus.c === --- subr_bus.c (revision 236680) +++ subr_bus.c (working copy) @@ -1988,7 +1990,7 @@ device_probe_child(device_t dev, device_t child) devclass_t dc; driverlink_t best = NULL; driverlink_t dl; - int result, pri = 0; + int error, result, pri = 0; int hasclass = (child->devclass != NULL); GIANT_REQUIRED; @@ -2019,17 +2021,18 @@ device_probe_child(device_t dev, device_t child) else if (result != 0) continue; if (!hasclass) { - if (device_set_devclass(child, - dl->driver->name) != 0) { + error = device_set_devclass(child, + dl->driver->name); + if (error != 0) { char const * devname = device_get_name(child); if (devname == NULL) devname = "(unknown)"; printf("driver bug: Unable to set " "devclass (class: %s " - "devname: %s)\n", + "devname: %s): %d\n", dl->driver->name, - devname); + devname, error); (void)device_set_driver(child, NULL); continue; } -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal in sys/kern/vfs_mount.c
Ruslan Mahmatkhanov wrote on 08.06.2012 12:10: Good day, After updating to yesterdays -current, I got this on boot: lock order reversal: 1st 0xfe0007b04c38 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1254 2nd 0xfe0007ed9478 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2158 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2c witness_checkorder() at witness_checkorder+0x853 __lockmgr_args() at __lockmgr_args+0x113a vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xbf _vn_lock() at _vn_lock+0x47 vget() at vget+0x7b devfs_allocv() at devfs_allocv+0x13f devfs_root() at devfs_root+0x4d dounmount() at dounmount+0x45c vfs_unmountall() at vfs_unmountall+0x4c kern_reboot() at kern_reboot+0x84b sys_reboot() at sys_reboot+0x68 amd64_syscall() at amd64_syscall+0x2e0 Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (55, FreeBSD ELF64, sys_reboot), rip = 0x40ebbc, rsp = 0x7fffd6c8, rbp = 0x65 --- Reverting to old kernel (that was built about a week ago) helped to avoid this. Any thoughts? And this one comes up with the recent update too: driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) driver bug: Unable to set devclass (class: acpi_timer devname: (unknown)) cpu0: on acpi0 driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) cpu1: on acpi0 driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) cpu2: on acpi0 driver bug: Unable to set devclass (class: acpi_sysresource devname: (unknown)) cpu3: on acpi0 What the additional info should I supply to understand what's wrong? Here is my full dmesg; http://people.freebsd.org/~rm/dmesg.txt -- Regards, Ruslan Tinderboxing kills... the drives. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
lock order reversal in sys/kern/vfs_mount.c
Good day, After updating to yesterdays -current, I got this on boot: lock order reversal: 1st 0xfe0007b04c38 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1254 2nd 0xfe0007ed9478 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2158 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2c witness_checkorder() at witness_checkorder+0x853 __lockmgr_args() at __lockmgr_args+0x113a vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xbf _vn_lock() at _vn_lock+0x47 vget() at vget+0x7b devfs_allocv() at devfs_allocv+0x13f devfs_root() at devfs_root+0x4d dounmount() at dounmount+0x45c vfs_unmountall() at vfs_unmountall+0x4c kern_reboot() at kern_reboot+0x84b sys_reboot() at sys_reboot+0x68 amd64_syscall() at amd64_syscall+0x2e0 Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (55, FreeBSD ELF64, sys_reboot), rip = 0x40ebbc, rsp = 0x7fffd6c8, rbp = 0x65 --- Reverting to old kernel (that was built about a week ago) helped to avoid this. Any thoughts? -- Regards, Ruslan Tinderboxing kills... the drives. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 9.0-BETA3 lock order reversal in mount_smbfs
On Wed, 5 Oct 2011 19:39:56 -0700 Bob Finch wrote: BF> Attempting to mount a remote SMB share with mount_smbfs fails: BF> freebsd9b3# uname -a BF> FreeBSD freebsd9b3 9.0-BETA3 FreeBSD 9.0-BETA3 #0: Sat Sep 24 20:46:57 UTC 2011 r...@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 BF> freebsd9b3# mount_smbfs -I smbhost -U xxx -W domain //smbhost/xxx /mnt BF> Password: BF> mount_smbfs: unable to open connection: syserr = No such file or directory BF> and displays the following kernel messages: BF> smb_co_lock: recursive lock for object 1 BF> lock order reversal: BF> 1st 0xc2ef4608 smb_vc (smb_vc) @ /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:325 BF> 2nd 0xc2ffbc28 smbsm (smbsm) @ /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:348 BF> KDB: stack backtrace: BF> db_trace_self_wrapper(c0eff6ac,626d732f,2e2f7366,2e2e2f2e,74656e2f,...) at db_trace_self_wrapper+0x26 BF> kdb_backtrace(c0a42bdb,c0f0300f,c29697b0,c29696e0,c76d298c,...) at kdb_backtrace+0x2a BF> _witness_debugger(c0f0300f,c2ffbc28,c2ff93df,c29696e0,c2ff9320,...) at _witness_debugger+0x25 BF> witness_checkorder(c2ffbc28,9,c2ff9320,15c,c2ffbc48,...) at witness_checkorder+0x839 BF> __lockmgr_args(c2ffbc28,8,c2ffbc48,0,0,...) at __lockmgr_args+0x824 BF> smb_co_lock(c2ffbc20,8,2,2,c76d2b30,...) at smb_co_lock+0x73 BF> smb_co_gone(c2ef4600,c76d2b88,c76d2b88,c76d2aac,c2ad5b00,...) at smb_co_gone+0x34 BF> smb_sm_lookup(c76d2ad8,c76d2b14,c76d2b88,c76d2b30,c29f041c,...) at smb_sm_lookup+0xf0 BF> smb_usr_lookup(c29f0400,c76d2b88,c76d2b94,c76d2b90,c76d2b7c,...) at smb_usr_lookup+0x98 BF> nsmb_dev_ioctl(c2f76700,82fc6e6a,c29f0400,3,c2fce8a0,...) at nsmb_dev_ioctl+0x1d9 BF> giant_ioctl(c2f76700,82fc6e6a,c29f0400,3,c2fce8a0,...) at giant_ioctl+0x75 BF> devfs_ioctl_f(c2d417a8,82fc6e6a,c29f0400,c2c05e00,c2fce8a0,...) at devfs_ioctl_f+0x10b BF> kern_ioctl(c2fce8a0,3,82fc6e6a,c29f0400,6d2cec,...) at kern_ioctl+0x21d BF> sys_ioctl(c2fce8a0,c76d2cec,c0f493b6,c0eebb0e,246,...) at sys_ioctl+0x134 BF> syscall(c76d2d28) at syscall+0x284 BF> Xint0x80_syscall() at Xint0x80_syscall+0x21 BF> --- syscall (54, FreeBSD ELF32, sys_ioctl), eip = 0x28193283, esp = 0xbfbfe35c, ebp = 0xbfbfe688 --- The LOR appears after the problem (the connection gone) happened, on error handling. So although it indicates that there is something wrong with our smb locking this does not look like a cause of your issue. BF> Anything further I can do to help debug this problem? Is this specific for 9.0-BETA3? Have you tried mounting the share with the same parameters on another systems? I think it could be useful to tcpdump the session and look at it in wireshark, which understands the SMB protocol. Also you might want to rebuild the kernel with this options in /etc/src.conf: DEBUG_FLAGS=-g -DSMB_SOCKET_DEBUG -DSMB_IOD_DEBUG -DNB_DEBUG -DSMB_VNODE_DEBUG which will add many debugging messages. This might be helpful for troubleshooting. -- Mikolaj Golub ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 9.0-BETA 3 lock order reversal
On Sat, 8 Oct 2011, Roar Pettersen wrote: Hello ! Just did a new build of world & kernel, and the error message have changed a bit : lock order reversal: 1st 0xddf0c4cc bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 2nd 0xc4996200 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284 Hi Roar, Both of these look to be "well-known" (i.e. trivially reproduced) LORs. No one seems to have been willing to step up to disable the warnings for them or otherwise eliminate them, though, so they still pop up and surprise people. Thanks for taking the time to report them, and sorry that they've been sitting untouched for so long. -Ben Kaduk [0] http://ipv4.sources.zabbadoz.net/freebsd/lor.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
9.0-BETA 3 lock order reversal
Hello ! Just did a new build of world & kernel, and the error message have changed a bit : lock order reversal: 1st 0xddf0c4cc bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 2nd 0xc4996200 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0a19f6c,7366752f,7366752f,7269645f,68736168,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0757c8b,c0a1d8cf,c45452a0,c4548f90,de2bf898,...) at kdb_backtrace+0x2a _witness_debugger(c0a1d8cf,c4996200,c0a42e72,c4548f90,c0a42af7,...) at _witness_debugger+0x25 witness_checkorder(c4996200,9,c0a42af7,11c,0,...) at witness_checkorder+0x839 _sx_xlock(c4996200,0,c0a42af7,11c,c4ccfd24,...) at _sx_xlock+0x85 ufsdirhash_acquire(ddf0c46c,c4ccfd24,de2bf9f4,deb3ca54,de2bf968,...) at ufsdirhash_acquire+0x35 ufsdirhash_add(c4ccfd24,de2bf9f4,a54,de2bf954,de2bf958,...) at ufsdirhash_add+0x13 ufs_direnter(c4cd8cc0,c4cd8bb0,de2bf9f4,de2bfb84,ddf0c868,...) at ufs_direnter+0x739 ufs_mkdir(de2bfc14,de2bfc28,0,0,de2bfbac,...) at ufs_mkdir+0x8ef VOP_MKDIR_APV(c0ab18c0,de2bfc14,de2bfb84,de2bfbac,0,...) at VOP_MKDIR_APV+0xa5 kern_mkdirat(c4cb38a0,ff9c,28404020,0,1c0,...) at kern_mkdirat+0x2a1 kern_mkdir(c4cb38a0,28404020,0,1c0,de2bfd1c,...) at kern_mkdir+0x2e sys_mkdir(c4cb38a0,de2bfcec,c0a568d6,c0a1e49e,202,...) at sys_mkdir+0x29 syscall(de2bfd28) at syscall+0x284 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28174303, esp = 0xbfbfe8cc, ebp = 0xbfbfed78 --- FreeBSD 9.0-BETA3 #1: Sat Oct 8 12:19:13 CEST 2011 i386 Regards Roar Pettersen Bergen - Norway ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
9.0-BETA 3 lock order reversal
Hello ! After upgrading my system from 8.2 to 9.0-BETA3 I now see this error message during boot : lock order reversal: 1st 0xc536e8d8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2134 2nd 0xde0077b4 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:260 3rd 0xc7b088d8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper(c0a19f6c,632e7262,3331323a,6f000a34,632e7370,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0757c8b,c0a1d8e8,c45452a0,c4548f28,de2ea3f4,...) at kdb_backtrace+0x2a _witness_debugger(c0a1d8e8,c7b088d8,c0a0cac0,c4548f28,c0a25674,...) at _witness_debugger+0x25 witness_checkorder(c7b088d8,9,c0a25674,856,0,...) at witness_checkorder+0x839 __lockmgr_args(c7b088d8,80100,c7b088f8,0,0,...) at __lockmgr_args+0x824 ffs_lock(de2ea51c,c0768ecb,c0a2495e,80100,c7b08880,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0ab18c0,de2ea51c,c4e36c30,c0ac00e0,c7b08880,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7b08880,80100,c0a25674,856,4,...) at _vn_lock+0x5e vget(c7b08880,80100,c4e36b80,50,0,...) at vget+0xb9 vfs_hash_get(c494c798,62d401,8,c4e36b80,de2ea660,...) at vfs_hash_get+0xe6 ffs_vgetf(c494c798,62d401,8,de2ea660,1,...) at ffs_vgetf+0x49 softdep_sync_buf(c536e880,de007754,1,106,0,...) at softdep_sync_buf+0x4a3 ffs_syncvnode(c536e880,1,c0afe61c,4,c0a1456b,...) at ffs_syncvnode+0x24c ffs_truncate(c536e880,200,0,880,c49f6500,...) at ffs_truncate+0x7a3 ufs_direnter(c536e880,c7b08880,de2ea9f4,de2eab84,ddfeb7c0,...) at ufs_direnter+0x921 ufs_mkdir(de2eac14,de2eac28,0,0,de2eabac,...) at ufs_mkdir+0x8ef VOP_MKDIR_APV(c0ab18c0,de2eac14,de2eab84,de2eabac,0,...) at VOP_MKDIR_APV+0xa5 kern_mkdirat(c4e36b80,ff9c,bfbfea6d,0,1ff,...) at kern_mkdirat+0x2a1 kern_mkdir(c4e36b80,bfbfea6d,0,1ff,de2ead1c,...) at kern_mkdir+0x2e sys_mkdir(c4e36b80,de2eacec,c0a568d6,c0a1e526,202,...) at sys_mkdir+0x29 syscall(de2ead28) at syscall+0x284 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28174303, esp = 0xbfbfe7dc, ebp = 0xbfbfe8a8 --- Regards Roar Pettersen Bergen - Norway ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
9.0-BETA3 lock order reversal in mount_smbfs
Attempting to mount a remote SMB share with mount_smbfs fails: freebsd9b3# uname -a FreeBSD freebsd9b3 9.0-BETA3 FreeBSD 9.0-BETA3 #0: Sat Sep 24 20:46:57 UTC 2011 r...@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 freebsd9b3# mount_smbfs -I smbhost -U xxx -W domain //smbhost/xxx /mnt Password: mount_smbfs: unable to open connection: syserr = No such file or directory and displays the following kernel messages: smb_co_lock: recursive lock for object 1 lock order reversal: 1st 0xc2ef4608 smb_vc (smb_vc) @ /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:325 2nd 0xc2ffbc28 smbsm (smbsm) @ /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:348 KDB: stack backtrace: db_trace_self_wrapper(c0eff6ac,626d732f,2e2f7366,2e2e2f2e,74656e2f,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a42bdb,c0f0300f,c29697b0,c29696e0,c76d298c,...) at kdb_backtrace+0x2a _witness_debugger(c0f0300f,c2ffbc28,c2ff93df,c29696e0,c2ff9320,...) at _witness_debugger+0x25 witness_checkorder(c2ffbc28,9,c2ff9320,15c,c2ffbc48,...) at witness_checkorder+0x839 __lockmgr_args(c2ffbc28,8,c2ffbc48,0,0,...) at __lockmgr_args+0x824 smb_co_lock(c2ffbc20,8,2,2,c76d2b30,...) at smb_co_lock+0x73 smb_co_gone(c2ef4600,c76d2b88,c76d2b88,c76d2aac,c2ad5b00,...) at smb_co_gone+0x34 smb_sm_lookup(c76d2ad8,c76d2b14,c76d2b88,c76d2b30,c29f041c,...) at smb_sm_lookup+0xf0 smb_usr_lookup(c29f0400,c76d2b88,c76d2b94,c76d2b90,c76d2b7c,...) at smb_usr_lookup+0x98 nsmb_dev_ioctl(c2f76700,82fc6e6a,c29f0400,3,c2fce8a0,...) at nsmb_dev_ioctl+0x1d9 giant_ioctl(c2f76700,82fc6e6a,c29f0400,3,c2fce8a0,...) at giant_ioctl+0x75 devfs_ioctl_f(c2d417a8,82fc6e6a,c29f0400,c2c05e00,c2fce8a0,...) at devfs_ioctl_f+0x10b kern_ioctl(c2fce8a0,3,82fc6e6a,c29f0400,6d2cec,...) at kern_ioctl+0x21d sys_ioctl(c2fce8a0,c76d2cec,c0f493b6,c0eebb0e,246,...) at sys_ioctl+0x134 syscall(c76d2d28) at syscall+0x284 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (54, FreeBSD ELF32, sys_ioctl), eip = 0x28193283, esp = 0xbfbfe35c, ebp = 0xbfbfe688 --- Anything further I can do to help debug this problem? -- Bob ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal
Thanks, K. Macy Ok Sorry to disturb K. Macy wrote: When WITNESS support was added to lockmgr locks a number of longstanding LORs were exposed in the process. I can't comment on whether or not they'll be fixed or the warnings will some day be silenced. On Tue, Sep 6, 2011 at 2:58 PM, Vadim Denisov wrote: I install current on last week I have some messages in dmesg -a: lock order reversal: 1st 0xc81ba278 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:425 2nd 0xe0f566d0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 3rd 0xc7d89168 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:546 KDB: stack backtrace: db_trace_self_wrapper(c0c353ac,616e735f,6f687370,3a632e74,a363435,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c088dd7b,c0c38d9c,c7535098,c75385d0,ef9e1404,...) at kdb_backtrace+0x2a _witness_debugger(c0c38d9c,c7d89168,c0c2816c,c75385d0,c0c56442,...) at _witness_debugger+0x25 witness_checkorder(c7d89168,9,c0c56442,222,0,...) at witness_checkorder+0x839 __lockmgr_args(c7d89168,80100,c7d89188,0,0,...) at __lockmgr_args+0x824 ffs_lock(ef9e152c,c0ecca08,c7dc1c30,80100,c7d89110,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0d3c680,ef9e152c,ef9e154c,c0d4caa0,c7d89110,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7d89110,80100,c0c56442,222,c755ce80,...) at _vn_lock+0x5e ffs_snapshot(c7beba20,c7b753e0,c0c59850,1a2,0,...) at ffs_snapshot+0x14fc ffs_mount(c7beba20,c7bf5d00,ff,394,c7dc1c30,...) at ffs_mount+0x1c13 vfs_donmount(c7dc1b80,211000,c7787780,c7787780,c7dbb588,...) at vfs_donmount+0x1219 nmount(c7dc1b80,ef9e1cec,c087a16c,ef9e1d80,0,...) at nmount+0x84 syscallenter(c7dc1b80,ef9e1ce4,ef9e1d1c,c0acbf86,c0d90c80,...) at syscallenter+0x263 syscall(ef9e1d28) at syscall+0x34 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280fb61b, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- lock order reversal: 1st 0xe0f566d0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 2nd 0xc7563c9c snaplk (snaplk) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:818 KDB: stack backtrace: db_trace_self_wrapper(c0c353ac,662f7366,735f7366,7370616e,2e746f68,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c088dd7b,c0c38d83,c7535098,c7538978,ef9e1404,...) at kdb_backtrace+0x2a _witness_debugger(c0c38d83,c7563c9c,c0c564a4,c7538978,c0c56442,...) at _witness_debugger+0x25 witness_checkorder(c7563c9c,9,c0c56442,332,c81ba298,...) at witness_checkorder+0x839 __lockmgr_args(c7563c9c,80400,c81ba298,0,0,...) at __lockmgr_args+0x824 ffs_lock(ef9e152c,e0ef5790,10,80400,c81ba220,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0d3c680,ef9e152c,e0ef57ec,c0d4caa0,c81ba220,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c81ba220,80400,c0c56442,332,0,...) at _vn_lock+0x5e ffs_snapshot(c7beba20,c7b753e0,c0c59850,1a2,0,...) at ffs_snapshot+0x298e ffs_mount(c7beba20,c7bf5d00,ff,394,c7dc1c30,...) at ffs_mount+0x1c13 vfs_donmount(c7dc1b80,211000,c7787780,c7787780,c7dbb588,...) at vfs_donmount+0x1219 nmount(c7dc1b80,ef9e1cec,c087a16c,ef9e1d80,0,...) at nmount+0x84 syscallenter(c7dc1b80,ef9e1ce4,ef9e1d1c,c0acbf86,c0d90c80,...) at syscallenter+0x263 syscall(ef9e1d28) at syscall+0x34 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280fb61b, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- lock order reversal: 1st 0xc7563c9c snaplk (snaplk) @ /usr/src/sys/kern/vfs_vnops.c:307 2nd 0xc81ba278 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:1620 KDB: stack backtrace: db_trace_self_wrapper(c0c353ac,616e735f,6f687370,3a632e74,30323631,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c088dd7b,c0c38d83,c7538978,c75385d0,ef9e16c8,...) at kdb_backtrace+0x2a _witness_debugger(c0c38d83,c81ba278,c0c2816c,c75385d0,c0c56442,...) at _witness_debugger+0x25 witness_checkorder(c81ba278,9,c0c56442,654,0,...) at witness_checkorder+0x839 __lockmgr_args(c81ba278,8,0,0,0,...) at __lockmgr_args+0x824 ffs_snapremove(c81ba220,8,c0c3e10a,5e8,c7535098,...) at ffs_snapremove+0x11f ffs_truncate(c81ba220,0,0,c00,0,...) at ffs_truncate+0x577 ufs_inactive(ef9e1a9c,c81ba298,c81ba220,c81ba298,ef9e1ab4,...) at ufs_inactive+0x1f8 VOP_INACTIVE_APV(c0d3c680,ef9e1a9c,c0c40a67,94e,c0d4ca60,...) at VOP_INACTIVE_APV+0xa5 vinactive(c0d3c680,ef9e1ad0,c0c40a67,8a5,0,...) at vinactive+0x8e vputx(ef9e1b38,c08febda,c81ba220,ef9e1b14,c0c41f00,...) at vputx+0x2f8 vput(c81ba220,ef9e1b14,c0c41f00,133,0,...) at vput+0x10 vn_close(c81ba220,1,c755ce00,c7dc1b80,0,...) at vn_close+0x19a vn_closefile(c7c300a8,c7dc1b80,c0c2706c,0,c7c300a8,...) at vn_closefile+0xe4 _fdrop(c7c300a8,c7dc1b80,0,ef9e1bfc,0,c0ecc9d8,c7dc1c30,c0d28da0,c0ecc9d8,c0c2af54,c7dc1b80,c7b05d2c,4ce,ef9e1c0c,c085d087,c7b05d2c,8,c0c2af54,c7c300a8) at _fdrop+0x43 closef(c7c300a8,c7dc1b80,4ce,ef9e1c30,c7b05d2c,...) at closef+0x2b0 kern_close(c7dc1b80,4,ef9e1c7c,c0897ff3,c7dc1b80,...) at kern_close+0x139 close(c7dc1b80,ef9e1cec,ef9e1d28,c0c3767a,0,...) at close+0x1a syscallenter(c7dc1b80,ef9e1ce4,ef9e1ce4,0,c0d819f0,...) at syscallenter+0x263 syscall(ef9e1d28) at syscall+0x34 Xint0x80_sy
Re: lock order reversal
When WITNESS support was added to lockmgr locks a number of longstanding LORs were exposed in the process. I can't comment on whether or not they'll be fixed or the warnings will some day be silenced. On Tue, Sep 6, 2011 at 2:58 PM, Vadim Denisov wrote: > I install current on last week > I have some messages in dmesg -a: > lock order reversal: > 1st 0xc81ba278 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:425 > 2nd 0xe0f566d0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 > 3rd 0xc7d89168 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:546 > KDB: stack backtrace: > db_trace_self_wrapper(c0c353ac,616e735f,6f687370,3a632e74,a363435,...) at > db_trace_self_wrapper+0x26 > kdb_backtrace(c088dd7b,c0c38d9c,c7535098,c75385d0,ef9e1404,...) at > kdb_backtrace+0x2a > _witness_debugger(c0c38d9c,c7d89168,c0c2816c,c75385d0,c0c56442,...) at > _witness_debugger+0x25 > witness_checkorder(c7d89168,9,c0c56442,222,0,...) at > witness_checkorder+0x839 > __lockmgr_args(c7d89168,80100,c7d89188,0,0,...) at __lockmgr_args+0x824 > ffs_lock(ef9e152c,c0ecca08,c7dc1c30,80100,c7d89110,...) at ffs_lock+0x8a > VOP_LOCK1_APV(c0d3c680,ef9e152c,ef9e154c,c0d4caa0,c7d89110,...) at > VOP_LOCK1_APV+0xb5 > _vn_lock(c7d89110,80100,c0c56442,222,c755ce80,...) at _vn_lock+0x5e > ffs_snapshot(c7beba20,c7b753e0,c0c59850,1a2,0,...) at ffs_snapshot+0x14fc > ffs_mount(c7beba20,c7bf5d00,ff,394,c7dc1c30,...) at ffs_mount+0x1c13 > vfs_donmount(c7dc1b80,211000,c7787780,c7787780,c7dbb588,...) at > vfs_donmount+0x1219 > nmount(c7dc1b80,ef9e1cec,c087a16c,ef9e1d80,0,...) at nmount+0x84 > syscallenter(c7dc1b80,ef9e1ce4,ef9e1d1c,c0acbf86,c0d90c80,...) at > syscallenter+0x263 > syscall(ef9e1d28) at syscall+0x34 > Xint0x80_syscall() at Xint0x80_syscall+0x21 > --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280fb61b, esp = > 0xbfbfea9c, ebp = 0xbfbfede8 --- > lock order reversal: > 1st 0xe0f566d0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 > 2nd 0xc7563c9c snaplk (snaplk) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:818 > KDB: stack backtrace: > db_trace_self_wrapper(c0c353ac,662f7366,735f7366,7370616e,2e746f68,...) at > db_trace_self_wrapper+0x26 > kdb_backtrace(c088dd7b,c0c38d83,c7535098,c7538978,ef9e1404,...) at > kdb_backtrace+0x2a > _witness_debugger(c0c38d83,c7563c9c,c0c564a4,c7538978,c0c56442,...) at > _witness_debugger+0x25 > witness_checkorder(c7563c9c,9,c0c56442,332,c81ba298,...) at > witness_checkorder+0x839 > __lockmgr_args(c7563c9c,80400,c81ba298,0,0,...) at __lockmgr_args+0x824 > ffs_lock(ef9e152c,e0ef5790,10,80400,c81ba220,...) at ffs_lock+0x8a > VOP_LOCK1_APV(c0d3c680,ef9e152c,e0ef57ec,c0d4caa0,c81ba220,...) at > VOP_LOCK1_APV+0xb5 > _vn_lock(c81ba220,80400,c0c56442,332,0,...) at _vn_lock+0x5e > ffs_snapshot(c7beba20,c7b753e0,c0c59850,1a2,0,...) at ffs_snapshot+0x298e > ffs_mount(c7beba20,c7bf5d00,ff,394,c7dc1c30,...) at ffs_mount+0x1c13 > vfs_donmount(c7dc1b80,211000,c7787780,c7787780,c7dbb588,...) at > vfs_donmount+0x1219 > nmount(c7dc1b80,ef9e1cec,c087a16c,ef9e1d80,0,...) at nmount+0x84 > syscallenter(c7dc1b80,ef9e1ce4,ef9e1d1c,c0acbf86,c0d90c80,...) at > syscallenter+0x263 > syscall(ef9e1d28) at syscall+0x34 > Xint0x80_syscall() at Xint0x80_syscall+0x21 > --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280fb61b, esp = > 0xbfbfea9c, ebp = 0xbfbfede8 --- > lock order reversal: > 1st 0xc7563c9c snaplk (snaplk) @ /usr/src/sys/kern/vfs_vnops.c:307 > 2nd 0xc81ba278 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:1620 > KDB: stack backtrace: > db_trace_self_wrapper(c0c353ac,616e735f,6f687370,3a632e74,30323631,...) at > db_trace_self_wrapper+0x26 > kdb_backtrace(c088dd7b,c0c38d83,c7538978,c75385d0,ef9e16c8,...) at > kdb_backtrace+0x2a > _witness_debugger(c0c38d83,c81ba278,c0c2816c,c75385d0,c0c56442,...) at > _witness_debugger+0x25 > witness_checkorder(c81ba278,9,c0c56442,654,0,...) at > witness_checkorder+0x839 > __lockmgr_args(c81ba278,8,0,0,0,...) at __lockmgr_args+0x824 > ffs_snapremove(c81ba220,8,c0c3e10a,5e8,c7535098,...) at ffs_snapremove+0x11f > ffs_truncate(c81ba220,0,0,c00,0,...) at ffs_truncate+0x577 > ufs_inactive(ef9e1a9c,c81ba298,c81ba220,c81ba298,ef9e1ab4,...) at > ufs_inactive+0x1f8 > VOP_INACTIVE_APV(c0d3c680,ef9e1a9c,c0c40a67,94e,c0d4ca60,...) at > VOP_INACTIVE_APV+0xa5 > vinactive(c0d3c680,ef9e1ad0,c0c40a67,8a5,0,...) at vinactive+0x8e > vputx(ef9e1b38,c08febda,c81ba220,ef9e1b14,c0c41f00,...) at vputx+0x2f8 > vput(c81ba220,ef9e1b14,c0c41f00,133,0,...) at vput+0x10 > vn_close(c81ba220,1,c755ce00,c7dc1b80,0,...) at vn_close+0x19a > vn_closefile(c7c300a8,c7dc1b80,c0c2706c,0,c7c300a8,...) at vn_closefile+0xe4 > _fdrop(c7c300a8,c7dc1b80,0,ef9e1bfc,0,c0ecc9d8,c7dc1c30,c0d28da0,c0ecc9d8,c0c2af54,c7dc1b80,c7b05d2c,4ce,ef9e1c0c,c085d087,c7b05d2c,8,c0c2af54
lock order reversal
I install current on last week I have some messages in dmesg -a: lock order reversal: 1st 0xc81ba278 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:425 2nd 0xe0f566d0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 3rd 0xc7d89168 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:546 KDB: stack backtrace: db_trace_self_wrapper(c0c353ac,616e735f,6f687370,3a632e74,a363435,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c088dd7b,c0c38d9c,c7535098,c75385d0,ef9e1404,...) at kdb_backtrace+0x2a _witness_debugger(c0c38d9c,c7d89168,c0c2816c,c75385d0,c0c56442,...) at _witness_debugger+0x25 witness_checkorder(c7d89168,9,c0c56442,222,0,...) at witness_checkorder+0x839 __lockmgr_args(c7d89168,80100,c7d89188,0,0,...) at __lockmgr_args+0x824 ffs_lock(ef9e152c,c0ecca08,c7dc1c30,80100,c7d89110,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0d3c680,ef9e152c,ef9e154c,c0d4caa0,c7d89110,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7d89110,80100,c0c56442,222,c755ce80,...) at _vn_lock+0x5e ffs_snapshot(c7beba20,c7b753e0,c0c59850,1a2,0,...) at ffs_snapshot+0x14fc ffs_mount(c7beba20,c7bf5d00,ff,394,c7dc1c30,...) at ffs_mount+0x1c13 vfs_donmount(c7dc1b80,211000,c7787780,c7787780,c7dbb588,...) at vfs_donmount+0x1219 nmount(c7dc1b80,ef9e1cec,c087a16c,ef9e1d80,0,...) at nmount+0x84 syscallenter(c7dc1b80,ef9e1ce4,ef9e1d1c,c0acbf86,c0d90c80,...) at syscallenter+0x263 syscall(ef9e1d28) at syscall+0x34 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280fb61b, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- lock order reversal: 1st 0xe0f566d0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 2nd 0xc7563c9c snaplk (snaplk) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:818 KDB: stack backtrace: db_trace_self_wrapper(c0c353ac,662f7366,735f7366,7370616e,2e746f68,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c088dd7b,c0c38d83,c7535098,c7538978,ef9e1404,...) at kdb_backtrace+0x2a _witness_debugger(c0c38d83,c7563c9c,c0c564a4,c7538978,c0c56442,...) at _witness_debugger+0x25 witness_checkorder(c7563c9c,9,c0c56442,332,c81ba298,...) at witness_checkorder+0x839 __lockmgr_args(c7563c9c,80400,c81ba298,0,0,...) at __lockmgr_args+0x824 ffs_lock(ef9e152c,e0ef5790,10,80400,c81ba220,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0d3c680,ef9e152c,e0ef57ec,c0d4caa0,c81ba220,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c81ba220,80400,c0c56442,332,0,...) at _vn_lock+0x5e ffs_snapshot(c7beba20,c7b753e0,c0c59850,1a2,0,...) at ffs_snapshot+0x298e ffs_mount(c7beba20,c7bf5d00,ff,394,c7dc1c30,...) at ffs_mount+0x1c13 vfs_donmount(c7dc1b80,211000,c7787780,c7787780,c7dbb588,...) at vfs_donmount+0x1219 nmount(c7dc1b80,ef9e1cec,c087a16c,ef9e1d80,0,...) at nmount+0x84 syscallenter(c7dc1b80,ef9e1ce4,ef9e1d1c,c0acbf86,c0d90c80,...) at syscallenter+0x263 syscall(ef9e1d28) at syscall+0x34 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280fb61b, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- lock order reversal: 1st 0xc7563c9c snaplk (snaplk) @ /usr/src/sys/kern/vfs_vnops.c:307 2nd 0xc81ba278 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:1620 KDB: stack backtrace: db_trace_self_wrapper(c0c353ac,616e735f,6f687370,3a632e74,30323631,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c088dd7b,c0c38d83,c7538978,c75385d0,ef9e16c8,...) at kdb_backtrace+0x2a _witness_debugger(c0c38d83,c81ba278,c0c2816c,c75385d0,c0c56442,...) at _witness_debugger+0x25 witness_checkorder(c81ba278,9,c0c56442,654,0,...) at witness_checkorder+0x839 __lockmgr_args(c81ba278,8,0,0,0,...) at __lockmgr_args+0x824 ffs_snapremove(c81ba220,8,c0c3e10a,5e8,c7535098,...) at ffs_snapremove+0x11f ffs_truncate(c81ba220,0,0,c00,0,...) at ffs_truncate+0x577 ufs_inactive(ef9e1a9c,c81ba298,c81ba220,c81ba298,ef9e1ab4,...) at ufs_inactive+0x1f8 VOP_INACTIVE_APV(c0d3c680,ef9e1a9c,c0c40a67,94e,c0d4ca60,...) at VOP_INACTIVE_APV+0xa5 vinactive(c0d3c680,ef9e1ad0,c0c40a67,8a5,0,...) at vinactive+0x8e vputx(ef9e1b38,c08febda,c81ba220,ef9e1b14,c0c41f00,...) at vputx+0x2f8 vput(c81ba220,ef9e1b14,c0c41f00,133,0,...) at vput+0x10 vn_close(c81ba220,1,c755ce00,c7dc1b80,0,...) at vn_close+0x19a vn_closefile(c7c300a8,c7dc1b80,c0c2706c,0,c7c300a8,...) at vn_closefile+0xe4 _fdrop(c7c300a8,c7dc1b80,0,ef9e1bfc,0,c0ecc9d8,c7dc1c30,c0d28da0,c0ecc9d8,c0c2af54,c7dc1b80,c7b05d2c,4ce,ef9e1c0c,c085d087,c7b05d2c,8,c0c2af54,c7c300a8) at _fdrop+0x43 closef(c7c300a8,c7dc1b80,4ce,ef9e1c30,c7b05d2c,...) at closef+0x2b0 kern_close(c7dc1b80,4,ef9e1c7c,c0897ff3,c7dc1b80,...) at kern_close+0x139 close(c7dc1b80,ef9e1cec,ef9e1d28,c0c3767a,0,...) at close+0x1a syscallenter(c7dc1b80,ef9e1ce4,ef9e1ce4,0,c0d819f0,...) at syscallenter+0x263 syscall(ef9e1d28) at syscall+0x34 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (6, FreeBSD ELF32, close), eip = 0x281a3283, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Sep 5 12:38:10 su: denisov to root on /dev/pts/0 lock order reversal: 1st 0xe10fbc60 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 2nd 0xc7c5d600 dirhash
Re: Lock order reversal .
On Tue, Dec 7, 2010 at 9:18 AM, Julian Elischer wrote: > On 12/7/10 3:41 AM, Attilio Rao wrote: >> >> 2010/12/7 Erik Cederstrand: >>> >>> Den 07/12/2010 kl. 10.20 skrev Garrett Cooper: >>> >>>> On Dec 7, 2010, at 12:26 AM, Mehmet Erol >>>> Sanliturk wrote: >>>> >>>>> A Dmesg.TXT is attached having a lock order reversal . >>>> >>>> The mount LOR is well known. >>> >>> I see that this is the standard response to lot's of LOR reports. It >>> seems to be one of the most-reported errors on CURRENT (and it's certainly a >>> loud one), but I think a lot of people waste time researching the error and >>> browsing Bjoerns LOR page, only to get the above response (not picking on >>> you, Garrett). >>> >>> Do we have the possibility of silencing well-known and presumably >>> harmless LOR's if there isn't sufficient motivation to fix the source? >> >> Witness has an 'internal blessing list' we never wanted to use in >> order to keep them popping up as reminder. >> Actually, the fact the LOR is 'known' doesn't mean it is 'analyzed'. >> The very few 'Analyzed but harmless' cases in the past have been >> handled via _NOWITNESS flags I guess. > > the problem is that the witness output tells you the second case (the > reversed case) > but it doesn't have any clues about the first case (the one that wsa the > other way around). > > An extended witness might use a lot of memory but associate with each lock a > 'last place called when a lock was already held' > that might give a clue as to where the other instance was. I'm not > volunteering to write it, > but it might be very worth while.. I'd certainly like to hear other ideas as > well. I have a small patch against stable/7 that adds a single bit to each witness structure so that, if the "normal" lock order is ever encountered after a reversal, the stack is printed. It doesn't help when the order is defined statically, though. I could try to roll this up against -CURRENT this weekend. Thanks, matthew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal .
On 12/7/10 3:41 AM, Attilio Rao wrote: 2010/12/7 Erik Cederstrand: Den 07/12/2010 kl. 10.20 skrev Garrett Cooper: On Dec 7, 2010, at 12:26 AM, Mehmet Erol Sanliturk wrote: A Dmesg.TXT is attached having a lock order reversal . The mount LOR is well known. I see that this is the standard response to lot's of LOR reports. It seems to be one of the most-reported errors on CURRENT (and it's certainly a loud one), but I think a lot of people waste time researching the error and browsing Bjoerns LOR page, only to get the above response (not picking on you, Garrett). Do we have the possibility of silencing well-known and presumably harmless LOR's if there isn't sufficient motivation to fix the source? Witness has an 'internal blessing list' we never wanted to use in order to keep them popping up as reminder. Actually, the fact the LOR is 'known' doesn't mean it is 'analyzed'. The very few 'Analyzed but harmless' cases in the past have been handled via _NOWITNESS flags I guess. the problem is that the witness output tells you the second case (the reversed case) but it doesn't have any clues about the first case (the one that wsa the other way around). An extended witness might use a lot of memory but associate with each lock a 'last place called when a lock was already held' that might give a clue as to where the other instance was. I'm not volunteering to write it, but it might be very worth while.. I'd certainly like to hear other ideas as well. Attilio ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal .
2010/12/7 Erik Cederstrand : > > Den 07/12/2010 kl. 10.20 skrev Garrett Cooper: > >> On Dec 7, 2010, at 12:26 AM, Mehmet Erol Sanliturk >> wrote: >> >>> A Dmesg.TXT is attached having a lock order reversal . >> >> The mount LOR is well known. > > I see that this is the standard response to lot's of LOR reports. It seems to > be one of the most-reported errors on CURRENT (and it's certainly a loud > one), but I think a lot of people waste time researching the error and > browsing Bjoerns LOR page, only to get the above response (not picking on > you, Garrett). > > Do we have the possibility of silencing well-known and presumably harmless > LOR's if there isn't sufficient motivation to fix the source? Witness has an 'internal blessing list' we never wanted to use in order to keep them popping up as reminder. Actually, the fact the LOR is 'known' doesn't mean it is 'analyzed'. The very few 'Analyzed but harmless' cases in the past have been handled via _NOWITNESS flags I guess. Attilio -- Peace can only be achieved by understanding - A. Einstein ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Lock order reversal .
Den 07/12/2010 kl. 10.20 skrev Garrett Cooper: > On Dec 7, 2010, at 12:26 AM, Mehmet Erol Sanliturk > wrote: > >> A Dmesg.TXT is attached having a lock order reversal . > >The mount LOR is well known. I see that this is the standard response to lot's of LOR reports. It seems to be one of the most-reported errors on CURRENT (and it's certainly a loud one), but I think a lot of people waste time researching the error and browsing Bjoerns LOR page, only to get the above response (not picking on you, Garrett). Do we have the possibility of silencing well-known and presumably harmless LOR's if there isn't sufficient motivation to fix the source? Erik
Re: Lock order reversal .
On Dec 7, 2010, at 12:26 AM, Mehmet Erol Sanliturk wrote: > A Dmesg.TXT is attached having a lock order reversal . The mount LOR is well known. The duplicate lock held WITNESS warning with em(4) might be of interest though. Thanks, -Garrett___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Lock order reversal .
A Dmesg.TXT is attached having a lock order reversal message . Thank you very much . Mehmet Erol Sanliturk Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT-201011 #0: Wed Nov 3 17:44:48 UTC 2010 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 WARNING: WITNESS option enabled, expect reduced performance. CPU: Intel(R) Core(TM)2 Quad CPUQ6600 @ 2.40GHz (2397.65-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x6fb Family = 6 Model = f Stepping = 11 Features=0xbfebfbff Features2=0xe3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant real memory = 2147483648 (2048 MB) avail memory = 2024038400 (1930 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 ioapic0: Changing APIC ID to 2 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 vgapci0: port 0x3410-0x3417 mem 0x9020-0x902f,0x8000-0x8fff irq 16 at device 2.0 on pci0 agp0: on vgapci0 agp0: aperture size is 256M, detected 7676k stolen memory pci0: at device 3.0 (no driver attached) em0: port 0x30e0-0x30ff mem 0x9030-0x9031,0x90324000-0x90324fff irq 20 at device 25.0 on pci0 em0: Using an MSI interrupt acquiring duplicate lock of same type: "network driver" 1st &dev_spec->swflag_mutex @ /usr/src/sys/dev/e1000/e1000_ich8lan.c:778 2nd &dev_spec->nvm_mutex @ /usr/src/sys/dev/e1000/e1000_ich8lan.c:744 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x8de _mtx_lock_flags() at _mtx_lock_flags+0x79 e1000_acquire_nvm_ich8lan() at e1000_acquire_nvm_ich8lan+0x1e e1000_read_nvm_ich8lan() at e1000_read_nvm_ich8lan+0x76 e1000_post_phy_reset_ich8lan() at e1000_post_phy_reset_ich8lan+0x1b1 e1000_reset_hw_ich8lan() at e1000_reset_hw_ich8lan+0x4c1 em_attach() at em_attach+0x120f device_attach() at device_attach+0x69 bus_generic_attach() at bus_generic_attach+0x1a acpi_pci_attach() at acpi_pci_attach+0x14f device_attach() at device_attach+0x69 bus_generic_attach() at bus_generic_attach+0x1a acpi_pcib_attach() at acpi_pcib_attach+0x1a7 acpi_pcib_acpi_attach() at acpi_pcib_acpi_attach+0x1fd device_attach() at device_attach+0x69 bus_generic_attach() at bus_generic_attach+0x1a acpi_attach() at acpi_attach+0xaa0 device_attach() at device_attach+0x69 bus_generic_attach() at bus_generic_attach+0x1a nexus_acpi_attach() at nexus_acpi_attach+0x69 device_attach() at device_attach+0x69 bus_generic_new_pass() at bus_generic_new_pass+0xd6 bus_set_pass() at bus_set_pass+0x7a configure() at configure+0xa mi_startup() at mi_startup+0x77 btext() at btext+0x2c em0: Ethernet address: 00:1c:c0:1e:c4:05 uhci0: port 0x30c0-0x30df irq 16 at device 26.0 on pci0 uhci0: LegSup = 0x2f00 usbus0: on uhci0 uhci1: port 0x30a0-0x30bf irq 21 at device 26.1 on pci0 uhci1: LegSup = 0x2f00 usbus1: on uhci1 ehci0: mem 0x90325c00-0x90325fff irq 18 at device 26.7 on pci0 usbus2: EHCI version 1.0 usbus2: on ehci0 pci0: at device 27.0 (no driver attached) pcib1: at device 28.0 on pci0 pci1: on pcib1 pcib2: at device 28.1 on pci0 pci2: on pcib2 atapci0: port 0x2018-0x201f,0x2024-0x2027,0x2010-0x2017,0x2020-0x2023,0x2000-0x200f mem 0x9010-0x901001ff irq 17 at device 0.0 on pci2 ata2: on atapci0 pcib3: at device 28.2 on pci0 pci3: on pcib3 pcib4: at device 28.3 on pci0 pci4: on pcib4 pcib5: at device 28.4 on pci0 pci5: on pcib5 uhci2: port 0x3080-0x309f irq 23 at device 29.0 on pci0 uhci2: LegSup = 0x2f00 usbus3: on uhci2 uhci3: port 0x3060-0x307f irq 19 at device 29.1 on pci0 uhci3: LegSup = 0x2f00 usbus4: on uhci3 uhci4: port 0x3040-0x305f irq 18 at device 29.2 on pci0 uhci4: LegSup = 0x2f00 usbus5: on uhci4 ehci1: mem 0x90325800-0x90325bff irq 23 at device 29.7 on pci0 usbus6: EHCI version 1.0 usbus6: on ehci1 pcib6: at device 30.0 on pci0 pci6: on pcib6 rl0: port 0x1000-0x10ff mem 0x90004800-0x900048ff irq 18 at device 2.0 on pci6 miibus0: on rl0 rlphy0: PHY 0 on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl0: Ethernet address: 00:1f:1f:38:bd:9c fwohci0: mem 0x90004000-0x900047ff,0x9000-0x90003fff irq 19 at device 3.
Re: lock order reversal bufwait/dirhash
On 9 Jun 2010, at 12:58, John Baldwin wrote: > On Wednesday 09 June 2010 3:51:52 am Bernd Walter wrote: >> Got this during installworld (source on NFS, destination UFS on CF-card) >> Source is current checked out yesterday. >> >> lock order reversal: >> 1st 0xc28b85b4 bufwait (bufwait) @ >> /data/builder/c13-2010-06-07/head/sys/kern/vfs_bio.c:2575 >> 2nd 0xc343f000 dirhash (dirhash) @ >> /data/builder/c13-2010-06-07/head/sys/ufs/ufs/ufs_dirhash.c:283 >> KDB: stack backtrace: > > Known false positive. From ufs_dirhash.c: > > /* > * Locking: > * > * The relationship between inode and dirhash is protected either by an > * exclusive vnode lock or the vnode interlock where a shared vnode lock > * may be used. The dirhash_mtx is acquired after the dirhash lock. To > * handle teardown races, code wishing to lock the dirhash for an inode > * when using a shared vnode lock must obtain a private reference on the > * dirhash while holding the vnode interlock. They can drop it once they > * have obtained the dirhash lock and verified that the dirhash wasn't > * recycled while they waited for the dirhash lock. > * > * ufsdirhash_build() acquires a shared lock on the dirhash when it is > * successful. This lock is released after a call to ufsdirhash_lookup(). > * > * Functions requiring exclusive access use ufsdirhash_acquire() which may > * free a dirhash structure that was recycled by ufsdirhash_recycle(). > * > * The dirhash lock may be held across io operations. > * > * WITNESS reports a lock order reversal between the "bufwait" lock > * and the "dirhash" lock. However, this specific reversal will not > * cause a deadlock. To get a deadlock, one would have to lock a > * buffer followed by the dirhash while a second thread locked a > * buffer while holding the dirhash lock. The second order can happen > * under a shared or exclusive vnode lock for the associated directory > * in lookup(). The first order, however, can only happen under an > * exclusive vnode lock (e.g. unlink(), rename(), etc.). Thus, for > * a thread to be doing a "bufwait" -> "dirhash" order, it has to hold > * an exclusive vnode lock. That exclusive vnode lock will prevent > * any other threads from doing a "dirhash" -> "bufwait" order. > */ Can we tell witness not to complain then? Regards, -- Rui Paulo ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal bufwait/dirhash
On Wednesday 09 June 2010 3:51:52 am Bernd Walter wrote: > Got this during installworld (source on NFS, destination UFS on CF-card) > Source is current checked out yesterday. > > lock order reversal: > 1st 0xc28b85b4 bufwait (bufwait) @ > /data/builder/c13-2010-06-07/head/sys/kern/vfs_bio.c:2575 > 2nd 0xc343f000 dirhash (dirhash) @ > /data/builder/c13-2010-06-07/head/sys/ufs/ufs/ufs_dirhash.c:283 > KDB: stack backtrace: Known false positive. From ufs_dirhash.c: /* * Locking: * * The relationship between inode and dirhash is protected either by an * exclusive vnode lock or the vnode interlock where a shared vnode lock * may be used. The dirhash_mtx is acquired after the dirhash lock. To * handle teardown races, code wishing to lock the dirhash for an inode * when using a shared vnode lock must obtain a private reference on the * dirhash while holding the vnode interlock. They can drop it once they * have obtained the dirhash lock and verified that the dirhash wasn't * recycled while they waited for the dirhash lock. * * ufsdirhash_build() acquires a shared lock on the dirhash when it is * successful. This lock is released after a call to ufsdirhash_lookup(). * * Functions requiring exclusive access use ufsdirhash_acquire() which may * free a dirhash structure that was recycled by ufsdirhash_recycle(). * * The dirhash lock may be held across io operations. * * WITNESS reports a lock order reversal between the "bufwait" lock * and the "dirhash" lock. However, this specific reversal will not * cause a deadlock. To get a deadlock, one would have to lock a * buffer followed by the dirhash while a second thread locked a * buffer while holding the dirhash lock. The second order can happen * under a shared or exclusive vnode lock for the associated directory * in lookup(). The first order, however, can only happen under an * exclusive vnode lock (e.g. unlink(), rename(), etc.). Thus, for * a thread to be doing a "bufwait" -> "dirhash" order, it has to hold * an exclusive vnode lock. That exclusive vnode lock will prevent * any other threads from doing a "dirhash" -> "bufwait" order. */ -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: lock order reversal bufwait/dirhash
On Jun 9, 2010, at 12:51 AM, Bernd Walter wrote: > Got this during installworld (source on NFS, destination UFS on CF-card) > Source is current checked out yesterday. > > lock order reversal: > 1st 0xc28b85b4 bufwait (bufwait) @ > /data/builder/c13-2010-06-07/head/sys/kern/vfs_bio.c:2575 > 2nd 0xc343f000 dirhash (dirhash) @ > /data/builder/c13-2010-06-07/head/sys/ufs/ufs/ufs_dirhash.c:283 > KDB: stack backtrace: > db_trace_self_wrapper(c0905b2b,c0d02c56,c2d3d030,c2d40500,ca657890,...) at > db_trace_self_wrapper+0x26 > _witness_debugger(c0d02c56,c343f000,c0d279c2,c2d40500,c0d2762e,...) at > _witness_debugger+0x25 > witness_checkorder(c343f000,9,c0d2762e,11b,0,...) at witness_checkorder+0x73c > _sx_xlock(c343f000,0,c0d2762e,11b,c3a07e80,...) at _sx_xlock+0x51 > ufsdirhash_acquire(e,1828,ca6579f0,c3a07e80,cb2bd814,...) at > ufsdirhash_acquire+0x35 > ufsdirhash_add(c3a07e80,ca6579f0,1828,ca657950,ca657954,...) at > ufsdirhash_add+0x1f > ufs_direnter(c3944dd0,c423e220,ca6579f0,ca657bd0,c292ccb0,...) at > ufs_direnter+0x894 > ufs_mkdir(ca657bf8,ca657c0c,2,0,0,...) at ufs_mkdir+0x51c > VOP_MKDIR_APV(c0e0f4c0,ca657bf8,ca657bd0,ca657b3c,2,...) at VOP_MKDIR_APV+0x8e > kern_mkdirat(c39264e0,ff9c,80513e0,0,1c0,...) at kern_mkdirat+0x240 > kern_mkdir(c39264e0,80513e0,0,1c0,ca657c7c,...) at kern_mkdir+0x2f > mkdir(c39264e0,ca657cec,ca657d28,c0d014f7,0,...) at mkdir+0x27 > syscallenter(c39264e0,ca657ce4,ca657ce4,0,c0e630c0,...) at syscallenter+0xc6 > syscall(ca657d28) at syscall+0x3a > Xint0x80_syscall() at Xint0x80_syscall+0x20 > --- syscall (136, FreeBSD ELF32, mkdir), eip = 0x281817cf, esp = 0xbfbfe30c, > ebp = 0xbfbfe398 --- I've seen this for a while, but it was definitely present on the kernel that I tried working off of that was from Saturday. Thanks, -Garrett___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
lock order reversal bufwait/dirhash
Got this during installworld (source on NFS, destination UFS on CF-card) Source is current checked out yesterday. lock order reversal: 1st 0xc28b85b4 bufwait (bufwait) @ /data/builder/c13-2010-06-07/head/sys/kern/vfs_bio.c:2575 2nd 0xc343f000 dirhash (dirhash) @ /data/builder/c13-2010-06-07/head/sys/ufs/ufs/ufs_dirhash.c:283 KDB: stack backtrace: db_trace_self_wrapper(c0905b2b,c0d02c56,c2d3d030,c2d40500,ca657890,...) at db_trace_self_wrapper+0x26 _witness_debugger(c0d02c56,c343f000,c0d279c2,c2d40500,c0d2762e,...) at _witness_debugger+0x25 witness_checkorder(c343f000,9,c0d2762e,11b,0,...) at witness_checkorder+0x73c _sx_xlock(c343f000,0,c0d2762e,11b,c3a07e80,...) at _sx_xlock+0x51 ufsdirhash_acquire(e,1828,ca6579f0,c3a07e80,cb2bd814,...) at ufsdirhash_acquire+0x35 ufsdirhash_add(c3a07e80,ca6579f0,1828,ca657950,ca657954,...) at ufsdirhash_add+0x1f ufs_direnter(c3944dd0,c423e220,ca6579f0,ca657bd0,c292ccb0,...) at ufs_direnter+0x894 ufs_mkdir(ca657bf8,ca657c0c,2,0,0,...) at ufs_mkdir+0x51c VOP_MKDIR_APV(c0e0f4c0,ca657bf8,ca657bd0,ca657b3c,2,...) at VOP_MKDIR_APV+0x8e kern_mkdirat(c39264e0,ff9c,80513e0,0,1c0,...) at kern_mkdirat+0x240 kern_mkdir(c39264e0,80513e0,0,1c0,ca657c7c,...) at kern_mkdir+0x2f mkdir(c39264e0,ca657cec,ca657d28,c0d014f7,0,...) at mkdir+0x27 syscallenter(c39264e0,ca657ce4,ca657ce4,0,c0e630c0,...) at syscallenter+0xc6 syscall(ca657d28) at syscall+0x3a Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (136, FreeBSD ELF32, mkdir), eip = 0x281817cf, esp = 0xbfbfe30c, ebp = 0xbfbfe398 --- -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: PF not working, with lock order reversal
On Wed, Mar 10, 2010 at 8:43 AM, Rob Farmer wrote: > Hi, > > I just updated a sparc64 Sun Netra X1 running current. I am using PF > (built into the kernel) and now I cannot connect to the machine while > PF is enabled (but outbound traffic from the machine works). The same > ruleset has worked fine for me for several years on this and other > systems. I'm getting the following LOR at boot and wonder if it is > related? > > lock order reversal: > 1st 0xc0424d28 pf task mtx (pf task mtx) @ > /usr/src/sys/contrib/pf/net/pf.c:6929 > 2nd 0xf800011954f8 radix node head (radix node head) @ > /usr/src/sys/net/route.c:360 > KDB: stack backtrace: > _witness_debugger() at _witness_debugger+0x84 > witness_checkorder() at witness_checkorder+0xafc > _rw_rlock() at _rw_rlock+0x44 > rtalloc1_fib() at rtalloc1_fib+0x124 > rtalloc_ign_fib() at rtalloc_ign_fib+0xac > pf_calc_mss() at pf_calc_mss+0xbc > pf_test_tcp() at pf_test_tcp+0xf04 > pf_test() at pf_test+0x10e8 > pf_check_in() at pf_check_in+0x14 > pfil_run_hooks() at pfil_run_hooks+0xb8 > ip_input() at ip_input+0x488 > netisr_dispatch_src() at netisr_dispatch_src+0xf0 > ether_demux() at ether_demux+0x2ac > ether_input() at ether_input+0x24c > dc_rxeof() at dc_rxeof+0x350 > dc_intr() at dc_intr+0x310 > intr_event_execute_handlers() at intr_event_execute_handlers+0xc4 > ithread_loop() at ithread_loop+0xe4 > fork_exit() at fork_exit+0x6c > fork_trampoline() at fork_trampoline+0x8 > > My pf.conf: > http://www.predatorlabs.net/dl/pf.conf > My kernel config: > http://www.predatorlabs.net/dl/NETRA > > Thanks, > -- > Rob Farmer > To follow up on this issue: I tried using the route.h patch Qing Li posted in another thread and I can access the system now with PF running. I still get the LOR but otherwise everything is working normally. -- Rob Farmer ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
PF not working, with lock order reversal
Hi, I just updated a sparc64 Sun Netra X1 running current. I am using PF (built into the kernel) and now I cannot connect to the machine while PF is enabled (but outbound traffic from the machine works). The same ruleset has worked fine for me for several years on this and other systems. I'm getting the following LOR at boot and wonder if it is related? lock order reversal: 1st 0xc0424d28 pf task mtx (pf task mtx) @ /usr/src/sys/contrib/pf/net/pf.c:6929 2nd 0xf800011954f8 radix node head (radix node head) @ /usr/src/sys/net/route.c:360 KDB: stack backtrace: _witness_debugger() at _witness_debugger+0x84 witness_checkorder() at witness_checkorder+0xafc _rw_rlock() at _rw_rlock+0x44 rtalloc1_fib() at rtalloc1_fib+0x124 rtalloc_ign_fib() at rtalloc_ign_fib+0xac pf_calc_mss() at pf_calc_mss+0xbc pf_test_tcp() at pf_test_tcp+0xf04 pf_test() at pf_test+0x10e8 pf_check_in() at pf_check_in+0x14 pfil_run_hooks() at pfil_run_hooks+0xb8 ip_input() at ip_input+0x488 netisr_dispatch_src() at netisr_dispatch_src+0xf0 ether_demux() at ether_demux+0x2ac ether_input() at ether_input+0x24c dc_rxeof() at dc_rxeof+0x350 dc_intr() at dc_intr+0x310 intr_event_execute_handlers() at intr_event_execute_handlers+0xc4 ithread_loop() at ithread_loop+0xe4 fork_exit() at fork_exit+0x6c fork_trampoline() at fork_trampoline+0x8 My pf.conf: http://www.predatorlabs.net/dl/pf.conf My kernel config: http://www.predatorlabs.net/dl/NETRA Thanks, -- Rob Farmer ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
5.2-BETA: IA32 SMP: atkbd gone, lock order reversal swap_pager/uma_core
I have made some experiences on a Fujitsu-Siemens Primergy RX300 (Serverworks, Xeon with HyperThreading, MegaRAID) today. I was able to boot the 5.2-BETA ISO from CD-RW after being unsuccessful with floppies yesterday (not that I'd particularly trust floppies...) Here's what I've found so far: 1. KEYBOARD LOST I need to explicitly escape to the loader prompt and set hint.atkbd.0.flags=0 -- with the default of 0x1, I am unable to use the keyboard (tried twice, without and with re-plugging the keyboard, to no avail). The exact reason is unknown, the machine has remote management hardware on board which might interfere. Can this flags=0 be made the default or would that interfere with USB keyboard and headless configurations? 2. LOCK ORDER REVERSAL I let the machine buildworld, build a kernel and cvsup ports+doc at the same time through screen. Both buildworld and the kernel make were launched with "-j2". The machine then caught a LOR, but it may have happened after these tasks completed, neither make nor the cvsup reported an error. Here's the dmesg: FreeBSD 5.2-BETA #0: Tue Nov 25 08:24:08 GMT 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC Preloaded elf kernel "/boot/kernel/kernel" at 0xc0a76000. ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2800.13-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff Hyperthreading: 2 logical CPUs real memory = 536870912 (512 MB) avail memory = 511750144 (488 MB) FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ... SMP: AP CPU #1 Launched! Mounting root from ufs:/dev/amrd0s4a ... Lock order reversal 1st 0xc55acdec vm object (vm object) @ /usr/src/sys/vm/swap_pager.c:1323 2nd 0xc098cf60 swap_pager swhash (swap_pager swhash) @ /usr/src/sys/vm/swap_pager.c:1838 3rd 0xc10398c4 vm object (vm object) @ /usr/src/sys/vm/uma_core.c:876 Stack backtrace: (no backtrace) Since then, no further problems have been logged in the past three and a half hours. -- Matthias Andree Encrypt your mail: my GnuPG key ID is 0x052E7D95 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: lock order reversal
On Sun, Nov 30, 2003 at 07:46:55PM -0500, [EMAIL PROTECTED] wrote: > Is this a known issue on 5.2 beta 6 sup'ed nov 29/03? Yes, it's reported on a daily basis and is harmless. Kris pgp0.pgp Description: PGP signature
lock order reversal
Is this a known issue on 5.2 beta 6 sup'ed nov 29/03? lock order reveral 1st 0xc2121b58 vm object (vm object) @ /usr/src/sys/vm/swap_pager.c:1323 2nd 0xc0987b00 swap_pager swhash (swap_pager swhash) @ /usr/src/sys/vm/swap_pager.c:1838 3rd 0xc1036948 vm object (vm object) @ /usr/src/sys/vm/uma_core.c:876 Stack backtrace: backtrace(c0897ea1,c1036948,c08ac9f5,c08ac9f5,c08ad8d6) at backtrace+0x17 witness_lock(c1036948,8,c08ad8d6,36c,1021540) at witness_lock+0x672 _mtx_lock_flags(c1036948,0,c08ad8d6,36c,c1021554) at _mtx_lock_flags+0xba obj_alloc(c1021540,1000,c8f879f7,101,c0956320) at obj_alloc+0x3f slab_zalloc(c1021540,1,8,c08ad8d6,68c) at slab_zalloc+0xb3 uma_zone_slab(c1021540,1,c08ad8d6,68c,c10215e0) at uma_zone_slab+0xd6 uma_zalloc_internal(c1021540,0,1,5c1,c08ad7ef,72e) at uma_zalloc_internal+0x3e uma_zalloc_arg(c1021540,0,1,72e,2) at uma_zalloc_arg+0x3ab swp_pager_meta_build(c2121b58,5,0,2,0) at swp_pager_meta_build+0x174 swap_pager_putpages(c2121b58,c8f87bd0,4,0,c8f87b30) at swap_pager_putpages+0x32d default_pager_putpages(c2121b58,c8f87bd0,4,0,c8f87b30) at default_pager_putpages+0x2e vm_pageout_flush(c8f87bd0,4,0,eb,c0958020) at vm_pageout_flush+0x17a vm_pageout_clean(c119f608,0,c08ad6f1,32a,0) vm_pageout_clean+0x305 vm_pageout_scan(0,0,c08ad6f1,5a9,1f4) at vm_pageout_scan+0x64c vm_pageout(0,c8f87d48,c08927ba,311,0) at vm_pageout+0x31b fork_exit(c07eb910,0,c8f87d48) at fork_exit+0xb4 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xc8f87d7c, ebd = 0 --- BTW then terminal then freezes and the intranet connection is frozen. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: another 5.2-BETA lock order reversal
On Fri, Nov 28, 2003 at 04:37:49PM -0500, Jesse Guardiani wrote: > Checked the archive and didn't see this one listed yet: > > lock order reversal > ?1st?0xc4047134?filedesc?structure?(filedesc?structure)[EMAIL > PROTECTED]/usr/src/sys/kern/sys_ > generic.c:896 > ?2nd?0xc0956a80?Giant?(Giant)[EMAIL PROTECTED]/usr/src/sys/fs/specfs/spec_vnops.c:377 Known problem, frequently reported. Thanks anyway, though. Kris pgp0.pgp Description: PGP signature
another 5.2-BETA lock order reversal
Checked the archive and didn't see this one listed yet: lock order reversal 1st 0xc4047134 filedesc structure (filedesc structure)[EMAIL PROTECTED]/usr/src/sys/kern/sys_ generic.c:896 2nd 0xc0956a80 Giant (Giant)[EMAIL PROTECTED]/usr/src/sys/fs/specfs/spec_vnops.c:377 Stack backtrace: backtrace(c089b89c,c0956a80,c0897b1f,c0897b1f,c0893352) at backtrace+0x17 witness_lock(c0956a80,8,c0893352,179,0) at witness_lock+0x672 _mtx_lock_flags(c0956a80,0,c0893352,179,c089bea3) at _mtx_lock_flags+0xba spec_poll(e476fafc,e476fb1c,c06d600c,e476fafc,c0937ca0) at spec_poll+0x134 spec_vnoperate(e476fafc,c0937ca0,c3c7e410,40,c435d680) at spec_vnoperate+0x18 vn_poll(c45992a8,40,c435d680,c43323c0,c435d680) at vn_poll+0x3c selscan(c43323c0,e476fb9c,e476fb8c,1,4) at selscan+0x141 kern_select(c43323c0,1,bfbfe890,0,bfbfe810) at kern_select+0x37f select(c43323c0,e476fd14,c08b631d,3ee,5) at select+0x66 syscall(2f,2f,2f,0,0) at syscall+0x2c0 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (93), eip = 0x2845b94f, esp = 0xbfbfe7cc, ebp = 0xbfbfe928 --- -- Jesse Guardiani, Systems Administrator WingNET Internet Services, P.O. Box 2605 // Cleveland, TN 37320-2605 423-559-LINK (v) 423-559-5145 (f) http://www.wingnet.net ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: lock order reversal is 5.2-BETA Nov 26
On Thu, Nov 27, 2003 at 10:00:54PM -0500, Mikhail Teterin wrote: > > On Thu, Nov 27, 2003 at 07:16:05PM -0500, Mikhail Teterin wrote: > > > It did not crash or anything, but the following is printed on > > > console now (addresses ommitted): > > > > > > lock order reversal > > >1st ... UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 > > >2nd ... system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 > > > > Thanks, this was reported several times already. > > How about this one? > > lock order reversal > 1st 0xc37abad4 vm object (vm object) @ /usr/src/sys/vm/swap_pager.c:1323 > 2nd 0xc098d020 swap_pager swhash (swap_pager swhash) @ > /usr/src/sys/vm/swap_pager.c:1838 > 3rd 0xc1036948 vm object (vm object) @ /usr/src/sys/vm/uma_core.c:876 Even more frequently ;-) Kris pgp0.pgp Description: PGP signature
Re: lock order reversal is 5.2-BETA Nov 26
> On Thu, Nov 27, 2003 at 07:16:05PM -0500, Mikhail Teterin wrote: > > It did not crash or anything, but the following is printed on > > console now (addresses ommitted): > > > > lock order reversal > > 1st ... UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 > > 2nd ... system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 > > Thanks, this was reported several times already. How about this one? lock order reversal 1st 0xc37abad4 vm object (vm object) @ /usr/src/sys/vm/swap_pager.c:1323 2nd 0xc098d020 swap_pager swhash (swap_pager swhash) @ /usr/src/sys/vm/swap_pager.c:1838 3rd 0xc1036948 vm object (vm object) @ /usr/src/sys/vm/uma_core.c:876 Sorry, backtrace was not logged, so I can not recreate it. -mi ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
lock order reversal is 5.2-BETA Nov 26
It did not crash or anything, but the following is printed on console now (addresses ommitted): lock order reversal 1st ... UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 2nd ... system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 Stack backtrace: backtrace() at backtrace+0x17 witness_lock() at wintess_lock+0x672 _mtx_lock_flags() at _mtx_lock_flags+0xba _vm_map_lock() at _vm_map_lock+0x36 vm_map_remove() at vm_map_remove+0x30 kmem_free() at kmem_free+0x32 page_free() at page_free+0x3b zone_drain() at zone_drain+0x2cf zone_foreach() at zone_foreach+0x45 uma_reclaim() at uma_reclaim+0x17 vm_pageout_scan() at vm_pageout_scan+0x148 vm_pageout() at vm_pageout+0x31b fork_exit() at fork_exit+0xb4 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = ., ebp = 0 --- Hope, this is usefull. A new kernel -- from today's sources -- is being compiled now. -mi ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: lock order reversal
On Tue, Nov 25, 2003 at 07:05:36PM -0600, John wrote: > i was just looking through my daily reports from my new 5.2 beta box and > found this in dmesg. > lock order reversal > 1st 0xc08f7ce0 UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 > 2nd 0xc1031100 system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 Here is the stack trace for the first one: lock order reversal 1st 0xc098e840 UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 2nd 0xc1031100 system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 Stack backtrace: backtrace(c089c5dc,c1031100,c08afe80,c08afe80,c08afef6) at backtrace+0x17 witness_lock(c1031100,8,c08afef6,8a2,c10310a0) at witness_lock+0x672 _mtx_lock_flags(c1031100,0,c08afef6,8a2,c55ae000) at _mtx_lock_flags+0xba _vm_map_lock(c10310a0,c08afef6,8a2,c5394bd0,0) at _vm_map_lock+0x36 vm_map_remove(c10310a0,c55ae000,c55af000,d77e5bf8,c07eacab) at vm_map_remove+0x30 kmem_free(c10310a0,c55ae000,1000,d77e5c28,c07ea6bf) at kmem_free+0x32 page_free(c55ae000,1000,2,0,c55ae000) at page_free+0x3b zone_drain(c101e000,0,c08b16a1,4b1,0) at zone_drain+0x2cf zone_foreach(c07ea3f0,d77e5cf0,c07e7b98,c08b154f,0) at zone_foreach+0x45 uma_reclaim(c08b154f,0,c08b14bc,29e,c095bf80) at uma_reclaim+0x17 vm_pageout_scan(0,0,c08b14bc,5a9,1f4) at vm_pageout_scan+0x148 vm_pageout(0,d77e5d48,c0896d18,311,0) at vm_pageout+0x31b fork_exit(c07e8990,0,d77e5d48) at fork_exit+0xb4 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xd77e5d7c, ebp = 0 --- pgp0.pgp Description: PGP signature
lock order reversal
i was just looking through my daily reports from my new 5.2 beta box and found this in dmesg. lock order reversal 1st 0xc08f7ce0 UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 2nd 0xc1031100 system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 Stack backtrace: lock order reversal 1st 0xc214c948 vm object (vm object) @ /usr/src/sys/vm/swap_pager.c:1323 2nd 0xc08f7160 swap_pager swhash (swap_pager swhash) @ /usr/src/sys/vm/swap_pager.c:1838 3rd 0xc10358c4 vm object (vm object) @ /usr/src/sys/vm/uma_core.c:876 Stack backtrace: I went back through /var/log/messages and found more, looks like it started around nov12 This box isn't really being used for much. It was a test box for mysql 4.0.15 and is a nfs server (no rpc.lockd running) Nov 12 01:36:40 nfs kernel: lock order reversal Nov 12 01:36:40 nfs kernel: 1st 0xc211f818 vm object (vm object) @ /usr/src/sys/vm/swap_pager.c:1323 Nov 12 01:36:40 nfs kernel: 2nd 0xc08ed180 swap_pager swhash (swap_pager swhash) @ /usr/src/sys/vm/swap_pager.c:1838 Nov 12 01:36:40 nfs kernel: 3rd 0xc103440c vm object (vm object) @ /usr/src/sys/vm/uma_core.c:876 Nov 12 01:36:40 nfs kernel: Stack backtrace: Nov 23 21:48:19 nfs kernel: lock order reversal Nov 23 21:48:19 nfs kernel: 1st 0xc08f7ce0 UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 Nov 23 21:48:19 nfs kernel: 2nd 0xc1031100 system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 Nov 23 21:48:19 nfs kernel: Stack backtrace: Nov 23 21:51:19 nfs kernel: lock order reversal Nov 23 21:51:19 nfs kernel: 1st 0xc2154294 vm object (vm object) @ /usr/src/sys/vm/swap_pager.c:1323 Nov 23 21:51:19 nfs kernel: 2nd 0xc08f7160 swap_pager swhash (swap_pager swhash) @ /usr/src/sys/vm/swap_pager.c:1838 Nov 23 21:51:19 nfs kernel: 3rd 0xc10358c4 vm object (vm object) @ /usr/src/sys/vm/uma_core.c:876 Nov 23 21:51:19 nfs kernel: Stack backtrace: Nov 24 03:03:52 nfs kernel: lock order reversal Nov 24 03:03:52 nfs kernel: 1st 0xc08f7ce0 UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1201 Nov 24 03:03:52 nfs kernel: 2nd 0xc1031100 system map (system map) @ /usr/src/sys/vm/vm_map.c:2210 Nov 24 03:03:52 nfs kernel: Stack backtrace: here is my whole dmesg. Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.2-BETA #0: Sun Nov 23 19:35:06 CST 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC Preloaded elf kernel "/boot/kernel/kernel" at 0xc09e. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Pentium/P55C (232.67-MHz 586-class CPU) Origin = "GenuineIntel" Id = 0x543 Stepping = 3 Features=0x8001bf real memory = 134217728 (128 MB) avail memory = 120754176 (115 MB) Intel Pentium detected, installing workaround for F00F bug ACPI-0159: *** Error: AcpiLoadTables: Could not get RSDP, AE_NO_ACPI_TABLES ACPI-0213: *** Error: AcpiLoadTables: Could not load tables: AE_NO_ACPI_TABLES ACPI: table load failed: AE_NO_ACPI_TABLES npx0: [FAST] npx0: on motherboard npx0: INT 16 interface pcibios: BIOS version 2.10 pcib0: at pcibus 0 on motherboard pci0: on pcib0 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0xffa0-0xffaf at device 7.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata0: [MPSAFE] ata1: at 0x170 irq 15 on atapci0 ata1: [MPSAFE] pci0: at device 8.0 (no driver attached) bt0: port 0xfff4-0xfff7 mem 0xfff7f000-0xfff7 irq 10 at device 17.0 on pci0 bt0: BT-958 FW Rev. 5.07B Ultra Wide SCSI Host Adapter, SCSI ID 7, 192 CCBs de0: port 0xf880-0xf8ff mem 0xfff7ec00-0xfff7ec7f irq 9 at device 19.0 on pci0 de0: SMC 9332BDT 21140A [10-100Mb/s] pass 2.2 de0: address 00:e0:29:00:b1:c2 orm0: at iomem 0xea000-0xebfff,0xe9000-0xe9fff,0xc8000-0xcbfff,0xc-0xc7fff on isa0 pmtimer0 on isa0 atkbdc0: at port 0x64,0x60 on isa0 atkbd0: flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 fdc0: at port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa-0xb on isa0 unknown: can't assign resources (port) psmcpnp0: irq resource info is missing; assuming irq 12 unknown: can't assign resources (port) ppc1: parallel port not found. unknown: can't assign resources (port) unknown: can't assign resources (port) Timecounter "TSC" frequency 232671577 Hz quality 800 Timecounters tick every 10.000 msec Waiting 15 seconds for SCSI devices to settle de0: enabling Full Duplex 100baseTX port GEOM: create disk da0 dp=0xc2096c50 da0 at bt0 bus 0 target 4 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 10.000MB/s transfers (10.000MHz
netgraph/ng_eiface double panic & turnstile/sio lock order reversal in 5.2-BETA
I've been experiencing a repeatable panic using ng_eiface(4) on -CURRENT of the last few days. Environment: FreeBSD twiddle.local 5.2-BETA FreeBSD 5.2-BETA #0: Tue Nov 25 19:28:22 UTC 2003 [EMAIL PROTECTED]:/home/data/work/usr/src/sys/TWIDDLE i386 Description: Shutting down an ng_eiface node which has a non-zero lladdr causes a panic. How-To-Repeat: Configure an ng_eiface(4) node, set its lladdr with ifconfig(8), shut it down: # cat >/tmp/ngctl mkpeer . eiface hook ether name .:hook vif0 rmhook . hook # ngctl -f /tmp/ngctl # ifconfig ngeth0 link '00:bd:03:11:25:01' # ngctl shutdown vif0: lock order reversal 1st 0xc0765d8c turnstile chain (turnstile chain) @ /usr/src/sys/kern/subr_turnstile.c:370 2nd 0xc079a500 sio (sio) @ /usr/src/sys/dev/sio/sio.c:3203 Stack backtrace: backtrace(c070794e,c079a500,c07502c0,c07502c0,c0719757) at backtrace+0x17 witness_lock(c079a500,8,c0719757,c83,3f8) at witness_lock+0x672 _mtx_lock_spin_flags(c079a500,0,c0719757,c83,1) at _mtx_lock_spin_flags+0xda siocnputc(c0750440,6b,5,ddcb08b4,6b) at siocnputc+0x81 cnputc(6b,c076e780,1,c4a40500,c071d675) at cnputc+0x7a putchar(6b,ddcb08b4,c053a72d,c076e800,1) at putchar+0x6c kvprintf(c071d674,c0564b80,ddcb08b4,a,ddcb08d4) at kvprintf+0x8d printf(c071d674,c,c056ced2,c078ac40,38) at printf+0x57 trap(c0760018,c0760010,c0760010,0,c4a40500) at trap+0xd7 calltrap() at calltrap+0x5 --- trap 0xc, eip = 0xc056a146, esp = 0xddcb0958, ebp = 0xddcb0978 --- turnstile_wait(0,c479b9c8,1103bd00,1cc,c479b9c8) at turnstile_wait+0x86 _mtx_lock_sleep(c479b9c8,0,c070c7ca,250,c49efe7c) at _mtx_lock_sleep+0x115 _mtx_lock_flags(c479b9c8,0,c070c7ca,250,c0538e1c) at _mtx_lock_flags+0x97 if_detach(c479b808,c4d64300,ddcb0a58,c4dbfa51,c479b808) at if_detach+0x394 ether_ifdetach(c479b808,c070d32d,820,c4d64300,c4d64300) at ether_ifdetach+0x30 ng_eiface_rmnode(c4d64300,0,0,c4d64300,c4d64300) at ng_eiface_rmnode+0x61 ng_rmnode(c4d64300,0,0,0,0) at ng_rmnode+0xc7 ng_generic_msg(c4d64300,c48b7780,0,c0768598,c07acfd4) at ng_generic_msg+0x11f ng_apply_item(c4d64300,c48b7780,c070d32d,7d6,c48b7780) at ng_apply_item+0x365 ng_snd_item(c48b7780,0,c47a4360,0,0) at ng_snd_item+0x7b6 ngc_send(c4ab4780,0,c1d12800,c47a4820,0) at ngc_send+0x146 sosend(c4ab4780,c47a4820,ddcb0c4c,c1d12800,0) at sosend+0x4cd kern_sendit(c4a40500,3,ddcb0cc4,0,0) at kern_sendit+0x17c sendit(c4a40500,3,ddcb0cc4,0,804f034) at sendit+0x16e sendto(c4a40500,ddcb0d14,c071d6f6,3ee,6) at sendto+0x5b syscall(2f,2f,2f,bfbfe9c0,bfbfe9c2) at syscall+0x2c0 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (133), eip = 0x280c58cf, esp = 0xbfbfe96c, ebp = 0xbfbfebe8 --- kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode fault virtual address = 0x1103bd00 fault code = supervisor read, page not present instruction pointer = 0x8:0xc056a146 stack pointer = 0x10:0xddcb0958 frame pointer = 0x10:0xddcb0978 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= resume, IOPL = 0 current process = 582 (ngctl) kernel: type 12 trap, code=0 Stopped at turnstile_wait+0x86:movl0(%edx),%eax db> panic panic: from debugger Debugger("panic") Fatal trap 3: breakpoint instruction fault while in kernel mode instruction pointer = 0x8:0xc06a9254 stack pointer = 0x10:0xddcb0720 frame pointer = 0x10:0xddcb072c code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= IOPL = 0 current process = 582 (ngctl) Stopped at turnstile_wait+0x86:movl0(%edx),%eax db> where turnstile_wait(0,c479b9c8,1103bd00,1cc,c479b9c8) at turnstile_wait+0x86 _mtx_lock_sleep(c479b9c8,0,c070c7ca,250,c49efe7c) at _mtx_lock_sleep+0x115 _mtx_lock_flags(c479b9c8,0,c070c7ca,250,c0538e1c) at _mtx_lock_flags+0x97 if_detach(c479b808,c4d64300,ddcb0a58,c4dbfa51,c479b808) at if_detach+0x394 ether_ifdetach(c479b808,c070d32d,820,c4d64300,c4d64300) at ether_ifdetach+0x30 ng_eiface_rmnode(c4d64300,0,0,c4d64300,c4d64300) at ng_eiface_rmnode+0x61 ng_rmnode(c4d64300,0,0,0,0) at ng_rmnode+0xc7 ng_generic_msg(c4d64300,c48b7780,0,c0768598,c07acfd4) at ng_generic_msg+0x11f ng_apply_item(c4d64300,c48b7780,c070d32d,7d6,c48b7780) at ng_apply_item+0x365 ng_snd_item(c48b7780,0,c47a4360,0,0) at ng_snd_item+0x7b6 ngc_send(c4ab4780,0,c1d12800,c47a4820,0) at ngc_send+0x146 sosend(c4ab4780,c47a4820,ddcb0c4c,c1d12800,0) at sosend+0x4cd kern_sendit(c4a40500,3,ddcb0cc4,0,0) at kern_sendit+0x17c sendit(c4a40500,3,ddcb0cc4,0,804f034) at sendit+0x16e sendto(c4a40500,ddcb0d14,c071d6f6,3ee,6) at sendto+0x5b syscall(2f,2f,2f,bfbfe9c0,bfbfe9c2) at syscall+0x2c0 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (133, FreeBSD ELF32, sendto), eip = 0x280c58cf, esp = 0xbfbfe96c, ebp = 0xbfbfebe8 --- db> panic pa
lock order reversal on 5.1
Got this earlier tonight on a box of mine... was testing a possible upgrade from 4.8 to 5.1 on a box to act as my router... dc0: promiscuous mode enabled Oct 26 20:32:37 router kernel: dc0: promiscuous mode enabled lock order reversal 1st 0xc05a4600 bpf global lock (bpf global lock) @ /usr/src/sys/net/bpf.c:375 2nd 0xc1f427bc dc0 (network driver) @ /usr/src/sys/pci/if_dc.c:3543 dc0: promiscuous mode disabled Oct 26 20:32:45 router kernel: dc0: promiscuous mode disabled I'll cvsup again here in a bit and rebuild yet again. I am seeing a version difference on this box and a box I have that builds rather frequently. Im thinking that it's fixed, but figured a heads up wouldn't hurt. I was attempting to go into tcpdump (with nnxXv set as flags). If the new sources dont fix it, I'll email an update, else just assume that it's all fixed. Regards, Nick H. Network Operations Center [EMAIL PROTECTED] Please rate my performance! http://www.supportteam.net/rate.php3 Please submit all new support requests to http://ticketmonster.hostingsupport.com/ --- Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer do not consent to Internet email for messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of my firm shall be understood as neither given nor endorsed by it. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: ip_divert.c lock order reversal
Hi, I am seeing an occasional kernel panic. I think it is related to natd and ip_divert Fatal trap 3: breakpoint instruction fault while in kernel mode instruction pointer = 0x8:0xc07e6c24 stack pointer = 0x10:0xce7026c4 frame pointer = 0x10:0xce7026d0 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= IOPL = 0 current process = 273 (natd) Reading symbols from /usr/obj/usr/src/sys/MYKERNEL1/modules/usr/src/sys/modules/acpi/acpi.ko.debug...done. Loaded symbols for /usr/obj/usr/src/sys/MYKERNEL1/modules/usr/src/sys/modules/acpi/acpi.ko.debug #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240 240 dumping++; (kgdb) where #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240 #1 0xc065c29c in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:372 #2 0xc065c627 in panic () at /usr/src/sys/kern/kern_shutdown.c:550 #3 0xc0467692 in db_panic () at /usr/src/sys/ddb/db_command.c:450 #4 0xc04675f2 in db_command (last_cmdp=0xc08f7d80, cmd_table=0x0, aux_cmd_tablep=0xc0882788, aux_cmd_tablep_end=0xc08827a0) at /usr/src/sys/ddb/db_command.c:346 #5 0xc0467735 in db_command_loop () at /usr/src/sys/ddb/db_command.c:472 #6 0xc046a735 in db_trap (type=12, code=0) at /usr/src/sys/ddb/db_trap.c:73 #7 0xc07e696c in kdb_trap (type=12, code=0, regs=0xce702904) at /usr/src/sys/i386/i386/db_interface.c:171 #8 0xc07f7d96 in trap_fatal (frame=0xce702904, eva=0) at /usr/src/sys/i386/i386/trap.c:815 #9 0xc07f7a62 in trap_pfault (frame=0xce702904, usermode=0, eva=3735929054) at /usr/src/sys/i386/i386/trap.c:734 #10 0xc07f761d in trap (frame= {tf_fs = 24, tf_es = 16, tf_ds = -831520752, tf_edi = -1064957451, tf_esi = -559038242, tf_ebp = -831510204, tf_isp = -831510224, tf_ebx = -831509976, tf_edx = -559038242, tf_ecx = 0, tf_eax = -559038242, tf_trapno = 12, tf_err = 0, tf_eip = -1066647656, tf_cs = 8, tf_eflags = 66118, tf_esp = -831510004, tf_ss = -1066938255}) at /usr/src/sys/i386/i386/trap.c:419 #11 0xc07e8358 in calltrap () at {standard input}:102 #12 0xc067d071 in kvprintf (fmt=0xc08609f5 " @ %s:%d", func=0xc067ca10 , arg=0xce702a28, radix=10, ap=0xce702a74 "\004É\206À\n\001") at /usr/src/sys/kern/subr_prf.c:669 #13 0xc067c98e in vsnprintf (str=0xc09214e0 "mtx_lock() of spin mutex ", size=0, format=0x0, ap=0x0) at /usr/src/sys/kern/subr_prf.c:414 #14 0xc065c541 in panic (fmt=0xc08609da "mtx_lock() of spin mutex %s @ %s:%d") at /usr/src/sys/kern/kern_shutdown.c:511 #15 0xc0652646 in _mtx_lock_flags (m=0xc2f37d90, opts=0, file=0xc086c904 "/usr/src/sys/netinet/ip_output.c", line=266) at /usr/src/sys/kern/kern_mutex.c:332 #16 0xc06f50c7 in ip_output (m0=0xc2f37d90, opt=0x10a, ro=0xc086c904, flags=34, imo=0x0, inp=0x0) at /usr/src/sys/netinet/ip_output.c:266 #17 0xc06e9021 in div_output (so=0xc2ee2000, m=0xc16e2f00, sin=0xc2ecb240, control=0x0) at /usr/src/sys/netinet/ip_divert.c:320 #18 0xc06e94fd in div_send (so=0x0, flags=0, m=0x0, nam=0x0, control=0x0, td=0xc2d40720) at /usr/src/sys/netinet/ip_divert.c:476 #19 0xc0699ecd in sosend (so=0xc2ee2000, addr=0xc2ecb240, uio=0xce702c48, top=0xc16e2f00, control=0x0, flags=0, td=0xc2d40720) at /usr/src/sys/kern/uipc_socket.c:714 #20 0xc069e48c in kern_sendit (td=0xc2d40720, s=3, mp=0xce702cc0, flags=0, control=0x0) at /usr/src/sys/kern/uipc_syscalls.c:723 #21 0xc069e2de in sendit (td=0x0, s=0, mp=0xce702cc0, flags=0) at /usr/src/sys/kern/uipc_syscalls.c:663 #22 0xc069e61b in sendto (td=0x0, uap=0x0) at /usr/src/sys/kern/uipc_syscalls.c:784 #23 0xc07f8100 in syscall (frame= {tf_fs = 47, tf_es = 47, tf_ds = -1078001617, tf_edi = -1078002688, tf_esi = 2, tf_ebp = -1077937128, tf_isp = -831509132, tf_ebx = 482, tf_edx = 26852, tf_ecx = 1148159575, tf_eax = 133, tf_trapno = 0, tf_err = 2, tf_eip = 134558627, tf_cs = 31, tf_eflags = 582, tf_esp = -1078002836, tf_ss = 47}) at /usr/src/sys/i386/i386/trap.c:1009 #24 0xc07e83ad in Xint0x80_syscall () at {standard input}:144 ---Can't read userspace from dump, or kernel process--- (kgdb) quit -- Craig Rodrigues http://crodrigues.org [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
ip_divert.c lock order reversal
Hi, I just cvsup'd and noticed the following: Starting sshd. lock order reversal 1st 0xc2ee3998 inp (inp) @ /usr/src/sys/netinet/udp_usrreq.c:1042 2nd 0xc094876c div (div) @ /usr/src/sys/netinet/ip_divert.c:225 Stack backtrace: backtrace(c08643de,c094876c,c087b2de,c087b2de,c086b1c6) at backtrace+0x17 witness_lock(c094876c,8,c086b1c6,e1,0) at witness_lock+0x672 _mtx_lock_flags(c094876c,0,c086b1c6,e1,c16db200) at _mtx_lock_flags+0xba divert_packet(c16db200,0,21dc,32,c06832f3) at divert_packet+0x133 ip_output(c16db200,0,c2ee3924,0,0) at ip_output+0x851 udp_output(c2ee38e8,c16db200,0,0,c16cc4c0) at udp_output+0x403 udp_send(c2f1c000,0,c16db200,0,0) at udp_send+0xb7 sosend(c2f1c000,0,cdd10c48,c16db200,0) at sosend+0x44d kern_sendit(c16cc4c0,4,cdd10cc0,0,0) at kern_sendit+0x17c sendit(c16cc4c0,4,cdd10cc0,0,bfbfd2f8) at sendit+0x16e sendto(c16cc4c0,cdd10d10,c087d03a,3ed,6) at sendto+0x5b syscall(2f,2f,2f,8106000,28) at syscall+0x2c0 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (133), eip = 0x282a810f, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Initial i386 initialization:. -- Craig Rodrigues http://crodrigues.org [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
lock order reversal
I've got a lock order reversal. #uname -a FreeBSD barleycoren.oikumene.gcd.org 5.1-CURRENT FreeBSD 5.1-CURRENT #1: Tue Sep 23 21:37:42 JST 2003 [EMAIL PROTECTED]:/build/usr/src/sys/BARLEYCOREN i386 Oct 11 18:14:53 barleycoren kernel: 1st 0xc082f060 system map (system map) @ /usr/src/sys/vm/vm_map.c:2236 Oct 11 18:14:53 barleycoren kernel: 2nd 0xc048e560 Giant (Giant) @ /usr/src/sys/vm/vm_map.c:2188 backtrace(c0418e40,c048e560,c0416343,c0416343,c0426bdd) at backtrace+0x17 witness_lock(c048e560,8,c0426bdd,88c,0) at witness_lock+0x5b6 _mtx_lock_flags(c048e560,0,c0426bdd,88c,c026cf6a) at _mtx_lock_flags+0x6a vm_map_delete(c082f000,dd839000,dd84a000,c456eb6c,c456eb6c) at vm_map_delete+0x1d6 vm_map_remove(c082f000,dd839000,dd84a000,dd547bd4,c029c970) at vm_map_remove+0x55 kmem_free(c082f000,dd839000,11000,c456eb6c,c456eb6c) at kmem_free+0x32 pipe_destroy_write_buffer(c456eb6c,0,c0419360,360,0) at pipe_destroy_write_buffer+0x60 pipe_direct_write(c456eb6c,dd547c80,c0419360,39c,c04b5be8) at pipe_direct_write+0x4e4 pipe_write(c456a000,dd547c80,c444e280,0,c424b5f0) at pipe_write+0x284 dofilewrite(c424b5f0,c456a000,1,80afb40,8000) at dofilewrite+0xe9 write(c424b5f0,dd547d14,c042b83f,3ec,3) at write+0x6e syscall(2f,2f,2f,80afb40,8000) at syscall+0x233 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (4), eip = 0x8050ec3, esp = 0xbfbfee8c, ebp = 0xbfbfeea8 --- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Lock order reversal
Has anyone seen this before? Sep 5 15:06:02 rdaver kernel: lock order reversal Sep 5 15:06:02 rdaver kernel: 1st 0xcf33ba34 filedesc structure (filedesc structure) @ kern/sys_generic.c:895 Sep 5 15:06:02 rdaver kernel: 2nd 0xc054c640 Giant (Giant) @ fs/specfs/spec_vnops.c:372 I've been getting this regularly. The above error was generated on a CVSUP from last night, but has occured on everything I've tried in 5.1. I'm running about 3T of disk, split into roughly 600G chunks for the bulk, and smaller volumes for /, /usr, etc. -- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: lock order reversal
On Thu, Jul 10, 2003 at 11:58:14AM +0200, Gordon Bergling wrote: > Hi, > > getting this on a -current from 'Jul 6 23:16:01 CEST 2003'. > I'll recieve this error a few minutes ago directly after booting the > system. No user where logged in this time. FAQ..it's harmless Kris pgp0.pgp Description: PGP signature
lock order reversal
Hi, getting this on a -current from 'Jul 6 23:16:01 CEST 2003'. I'll recieve this error a few minutes ago directly after booting the system. No user where logged in this time. lock order reversal 1st 0xc2ea9094 vm object (vm object) @ /usr/src/sys/vm/vm_object.c:1516 2nd 0xc082f110 system map (system map) @ /usr/src/sys/vm/vm_kern.c:325 Stack backtrace: backtrace(c03cb376,c082f110,c03d9b16,c03d9b16,c03d99b1) at backtrace+0x17 witness_lock(c082f110,8,c03d99b1,145,1) at witness_lock+0x697 _mtx_lock_flags(c082f110,0,c03d99b1,145,60e) at _mtx_lock_flags+0xb1 _vm_map_lock(c082f0b0,c03d99b1,145,c03c7a6a,1bc) at _vm_map_lock+0x36 kmem_malloc(c082f0b0,1000,101,d27e8a8c,c0357f5a) at kmem_malloc+0x39 page_alloc(c083a1c0,1000,d27e8a7f,101,c041210c) at page_alloc+0x27 slab_zalloc(c083a1c0,101,c03db375,664,c268ba94) at slab_zalloc+0x14a uma_zone_slab(c083a1c0,101,c03db375,664,0) at uma_zone_slab+0xd8 uma_zalloc_internal(c083a1c0,0,101,6e8,0) at uma_zalloc_internal+0x55 uma_zfree_arg(c268ba80,d1c36114,0,1,0) at uma_zfree_arg+0x2e7 swp_pager_meta_free(c2ea9094,1a,0,1,0) at swp_pager_meta_free+0x1b2 swap_pager_freespace(c2ea9094,1a,0,1,0) at swap_pager_freespace+0x57 vm_object_backing_scan(c2c725c8,4,c03da44e,5ec,c03da44e) at vm_object_backing_scan+0x36a vm_object_collapse(c2c725c8,0,c03da44e,1ef,c02390d0) at vm_object_collapse+0xdd vm_object_deallocate(c2e6b940,c285b03c,c2e6b940,c285b03c,d27e8c64) at vm_object_deallocate+0x28e vm_map_entry_delete(c2c73200,c285b03c,c03d9b84,8bc,c03c6b3c) at vm_map_entry_delete+0x3b vm_map_delete(c2c73200,0,bfc0,c2c73200,c2690380) at vm_map_delete+0x3e3 vm_map_remove(c2c73200,0,bfc0,11d,c03c60ed) at vm_map_remove+0x58 exit1(c2dd8980,0,c03c60ed,65,d27e8d40) at exit1+0x696 sys_exit(c2dd8980,d27e8d10,c03def62,3fd,1) at sys_exit+0x41 syscall(2f,2f,2f,0,) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (1), eip = 0x2829f29f, esp = 0xbfbfc3ec, ebp = 0xbfbfc408 --- best regards, Gordon -- There is no place like 127.0.0.1/8! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Lock order reversal
On Mon, Jul 07, 2003 at 02:26:43PM +0400, Andrew Kolchoogin wrote: > lock order reversal > 1st 0xc2f4b128 vm object (vm object) @ vm/vm_object.c:432 > 2nd 0xc082f110 system map (system map) @ vm/vm_kern.c:325 This is known to be harmless. Kris pgp0.pgp Description: PGP signature
Lock order reversal
Dear colleagues, === lock order reversal 1st 0xc2f4b128 vm object (vm object) @ vm/vm_object.c:432 2nd 0xc082f110 system map (system map) @ vm/vm_kern.c:325 Stack backtrace: backtrace(c04eacec,c082f110,c04fb2fa,c04fb2fa,c04fb1a2) at backtrace+0x17 witness_lock(c082f110,8,c04fb1a2,145,c082f0b0) at witness_lock+0x692 _mtx_lock_flags(c082f110,0,c04fb199,145,101) at _mtx_lock_flags+0xb1 _vm_map_lock(c082f0b0,c04fb199,145,0,c05b4600) at _vm_map_lock+0x36 kmem_malloc(c082f0b0,1000,101,d258fac4,c044c7df) at kmem_malloc+0x3a page_alloc(c083a1c0,1000,d258fab7,101,c05ae0e0) at page_alloc+0x27 slab_zalloc(c083a1c0,101,8,c04fcb37,664) at slab_zalloc+0x14f uma_zone_slab(c083a1c0,101,c04fcb2e,664,0) at uma_zone_slab+0xcb uma_zalloc_internal(c083a1c0,0,101,6e8,0) at uma_zalloc_internal+0x55 uma_zfree_arg(c268aa80,d1c77e04,0,1,0) at uma_zfree_arg+0x2bf swp_pager_meta_free_all(c2f4b128,c04faaf9,c04faa91,1b2) at swp_pager_meta_free_all+0x18f swap_pager_dealloc(c2f4b128,1,c04fca3d,10c,0) at swap_pager_dealloc+0x113 vm_pager_deallocate(c2f4b128,0,c04fbc2b,25f,1b0) at vm_pager_deallocate+0x3d vm_object_terminate(c2f4b128,0,c04fbc2b,1b0,d258fc14) at vm_object_terminate+0x1e8 vm_object_deallocate(c2f4b128,c2b9bd20,c2f4b128,c2b9bd20,d258fc68) at vm_object_deallocate+0x35f vm_map_entry_delete(c2ab2a00,c2b9bd20,c04fb368,8bc,0) at vm_map_entry_delete+000,0,bfc0,c2ab2a00,c26fc700) at vm_map_delete+0x3d3 vm_map_remove(c2ab2a00,0,bfc0,11d,65) at vm_map_remove+0x55 exit1(c2aa15f0,0,c04e5b6a,65,d258fd40) at exit1+0x60d sys_exit(c2aa15f0,d258fd14,c0500c6f,3fd,1) at sys_exit+0x41 syscall(2f,2f,2f,0,) at syscall+0x251 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (1), eip = 0x280d159f, esp = 0xbfbff42c, ebp = 0xbfbff448 --- === 5.1-CURRENT from July, 6, 2003. GENERIC kernel. --- Yours Andrew Kolchoogin. [DREW-RIPE, AKOL-RIPN] ... Contrary to popular belief, UNIX is user-friendly. It just happens to be very selective about who it decides to make friends with. A. Haiut. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
lock order reversal on recent (06/30) CURRENT
Is the following a known problem? (occured while running python2.3 using kse) lock order reversal 1st 0xc03c3c40 smp rendezvous (smp rendezvous) @ /usr/src/sys/kern/subr_smp.c:3 13 2nd 0xc03c00c0 sched lock (sched lock) @ /usr/src/sys/i386/i386/sys_machdep.c:2 92 Stack backtrace: backtrace(c0343555,c03c00c0,c033fd89,c033fd89,c0357a74) at backtrace+0x17 witness_lock(c03c00c0,8,c0357a74,124,ffc00034) at witness_lock+0x697 _mtx_lock_spin_flags(c03c00c0,0,c0357a74,124,f5ec0c54) at _mtx_lock_spin_flags+0 xd1 set_user_ldt_rv(c7eed390,f5ec0c78,c01d9bbb,9a,0) at set_user_ldt_rv+0x3d smp_rendezvous_action(9a,0,c0342f33,139,f5ec0d10) at smp_rendezvous_action+0x57 smp_rendezvous(0,c0312a10,0,c7eed390,c7eb97c0) at smp_rendezvous+0xab i386_set_ldt(c7eed390,bfbff96c,c0357a74,5f,c8228da8) at i386_set_ldt+0x16d sysarch(c7eed390,f5ec0d10,c0357cf5,3fd,2) at sysarch+0x64 syscall(2f,2f,2f,11,681518e4) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (165), eip = 0x6826dda3, esp = 0xbfbff958, ebp = 0xbfbff984 --- ===output of dmesg -a=== Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.1-CURRENT #12: Thu Jul 3 13:46:40 JST 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/MYKERNEL Preloaded elf kernel "/boot/kernel/kernel" at 0xc04e5000. Preloaded elf module "/boot/kernel/acpi.ko" at 0xc04e5294. Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 2392048864 Hz CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2392.05-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7 Features=0xbfebfbff Hyperthreading: 2 logical CPUs real memory = 2146893824 (2047 MB) avail memory = 2084458496 (1987 MB) Programming 24 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 24 pins in IOAPIC #1 Programming 24 pins in IOAPIC #2 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): apic id: 0, version: 0x00050014, at 0xfee0 cpu1 (AP): apic id: 1, version: 0x00050014, at 0xfee0 cpu2 (AP): apic id: 6, version: 0x00050014, at 0xfee0 cpu3 (AP): apic id: 7, version: 0x00050014, at 0xfee0 io0 (APIC): apic id: 2, version: 0x00178020, at 0xfec0 io1 (APIC): apic id: 3, version: 0x00178020, at 0xfec8 io2 (APIC): apic id: 4, version: 0x00178020, at 0xfec80100 Pentium Pro MTRR support enabled npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard pcibios: BIOS version 2.10 acpi0: power button is handled as a fixed feature programming model. Timecounter "ACPI-fast" frequency 3579545 Hz can't fetch resources for \\_SB_.PCI0.LPC0.SIO_.LPT_ - AE_AML_INVALID_RESOURCE_TYPE acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 acpi_cpu0: on acpi0 acpi_cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 IOAPIC #0 intpin 16 -> irq 2 IOAPIC #0 intpin 19 -> irq 5 IOAPIC #0 intpin 18 -> irq 10 IOAPIC #0 intpin 23 -> irq 11 agp0: mem 0xe000-0xefff at device 0.0 on pci0 pci0: at device 0.1 (no driver attached) pcib1: mem 0xd400-0xd7ff at device 1.0 on pci0 pci1: on pcib1 pcib2: at device 2.0 on pci0 pcib2: could not get PCI interrupt routing table for \\_SB_.PCI0.HLB_ - AE_NOT_FOUND pci2: on pcib2 pci2: at device 28.0 (no driver attached) pcib3: at device 29.0 on pci2 pci3: on pcib3 pci2: at device 30.0 (no driver attached) pcib4: at device 31.0 on pci2 pci4: on pcib4 IOAPIC #2 intpin 0 -> irq 16 IOAPIC #2 intpin 4 -> irq 17 ti0: mem 0xd021-0xd0213fff irq 16 at device 3.0 on pci4 ti0: Ethernet address: 00:a0:cc:73:49:65 bge0: mem 0xd020-0xd020 irq 17 at device 4.0 on pci4 bge0: Ethernet address: 00:50:45:00:96:f7 miibus0: on bge0 brgphy0: on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto pci0: at device 29.0 (no driver attached) pci0: at device 29.1 (no driver attached) pci0: at device 29.2 (no driver attached) pci0: at device 29.7 (no driver attached) pcib5: at device 30.0 on pci0 pci5: on pcib5 IOAPIC #0 intpin 21 -> irq 18 IOAPIC #0 intpin 22 -> irq 19 pci5: at device 0.0 (no driver attached) pci5: at device 1.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1460-0x146f,0-0x3,0-0x7,0-0x3,0-0x7 irq 0 at device 31.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 fdc0: port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0 port 0x378-0x37f on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lp
lock order reversal under recent -CURRENT
Hi, under a recent -CURRENT I get this: <---> lock order reversal 1st 0xc2e475c8 vm object (vm object) @ /usr/src/sys/vm/vm_object.c:432 2nd 0xc082f110 system map (system map) @ /usr/src/sys/vm/vm_kern.c:328 Stack backtrace: backtrace(c03c9d09,c082f110,c03d8851,c03d8851,c03d86ec) at backtrace+0x17 witness_lock(c082f110,8,c03d86ec,148,0) at witness_lock+0x697 _mtx_lock_flags(c082f110,0,c03d86ec,148,3) at _mtx_lock_flags+0xb1 _vm_map_lock(c082f0b0,c03d86ec,148,d26e3a54,c0247ce4) at _vm_map_lock+0x36 kmem_malloc(c082f0b0,1000,101,d26e3ac0,c03586e0) at kmem_malloc+0x66 page_alloc(c083a240,1000,d26e3ab3,101,c041066c) at page_alloc+0x27 slab_zalloc(c083a240,101,c03da0b0,66e,c26c56e4) at slab_zalloc+0x150 uma_zone_slab(c083a240,101,c03da0b0,66e,0) at uma_zone_slab+0xd8 uma_zalloc_internal(c083a240,0,101,6ee,0) at uma_zalloc_internal+0x55 uma_zfree_arg(c26c56c0,d1db6bdc,0,1,0) at uma_zfree_arg+0x2cb swp_pager_meta_free_all(c2e475c8,c03d8044,c03d7fd8,1b2) at swp_pager_meta_free_all+0x1b0 swap_pager_dealloc(c2e475c8,1,c03d9fb3,10c,0) at swap_pager_dealloc+0x113 vm_pager_deallocate(c2e475c8,0,c03d9189,25f,0) at vm_pager_deallocate+0x3d vm_object_terminate(c2e475c8,0,c03d9189,1b0,c02388e0) at vm_object_terminate+0x1f4 vm_object_deallocate(c2e475c8,c2da999c,c2e475c8,c2da999c,d26e3c64) at vm_object_deallocate+0x377 vm_map_entry_delete(c2dad800,c2da999c,c03d88bf,86d,c03c54da) at vm_map_entry_delete+0x3b vm_map_delete(c2dad800,0,bfc0,c2dad800,c26b7400) at vm_map_delete+0x413 vm_map_remove(c2dad800,0,bfc0,11d,c03c4a8b) at vm_map_remove+0x58 exit1(c2c8c980,0,c03c4a8b,65,d26e3d40) at exit1+0x696 sys_exit(c2c8c980,d26e3d10,c03dd56d,3fd,1) at sys_exit+0x41 syscall(2f,2f,2f,0,e34) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (1), eip = 0x2851d44f, esp = 0xbfbff840, ebp = 0xbfbff86c --- <-> `uname -a` says: <-> FreeBSD nemesis.bsd-network.org 5.1-CURRENT FreeBSD 5.1-CURRENT #3: Mon Jun 23 00:30:36 CEST 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/NEMESIS i386 <-> The LOR happens at normal work under X. A few xterm, mozilla, gimp and gqview. Not very wild things. best regards, Gordon -- There is no place like 127.0.0.1/8! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
lock order reversal (2 different items)
Has anyone come across this before? dc0: promiscuous mode enabled Jun 20 09:56:15 router kernel: dc0: promiscuous mode enabled lock order reversal 1st 0xc05a0100 bpf global lock (bpf global lock) @ /usr/src/sys/net/bpf.c:375 2nd 0xc1f5d7bc dc0 (network driver) @ /usr/src/sys/pci/if_dc.c:3543 dc0: promiscuous mode disabled --this happened when I did a tcpdump -nnxXv for my dc0 interface. I also got this one on boot today (gg cable company): Entropy harvesting: interrupts ethernet point_to_point . lock order reversal 1st 0xc1fb65c0 process lock (process lock) @ /usr/src/sys/kern/kern_descrip.c:2112 2nd 0xc200ac34 filedesc structure (filedesc structure) @ /usr/src/sys/kern/kern_descrip.c:2119 swapon: adding /dev/ad0s1b as swap device --Noticed it on boot, figured it'd be worth a mention It only happens one time per boot (which isnt often). Here's some stats on it: [EMAIL PROTECTED] harm $ uname -a FreeBSD router.XX.com 5.0-RELEASE-p7 FreeBSD 5.0-RELEASE-p7 #6: Sun Jun 15 01:09:07 CDT 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/Router i386 CPU: Pentium III/Pentium III Xeon/Celeron (448.80-MHz 686-class CPU) FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs real memory = 201261056 (191 MB) avail memory = 188223488 (179 MB) As you can see, the sources are fairly new (Jun 14th is the last time I grabbed and built them). If anyone has any ideas/suggestions or if you need anything more off of me (Im sorry, Im fresh out of money to give out) ;) I'll be more than happy to provide with what I can. Thanks! Regards, Nick H. [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
RE: lock order reversal? current with tl ethernet
On Wed, 12 Mar 2003, John Baldwin wrote: > It's holding the lock across bus_setup_intr(). You can try the > following patch: > > Index: if_tl.c > === > RCS file: /usr/cvs/src/sys/pci/if_tl.c,v > retrieving revision 1.74 > diff -u -r1.74 if_tl.c > --- if_tl.c 19 Feb 2003 05:47:41 - 1.74 > +++ if_tl.c 12 Mar 2003 15:20:47 - > @@ -1138,12 +1138,11 @@ > > if (t->tl_name == NULL) { > device_printf(dev, "unknown device!?\n"); > - goto fail; > device_printf(dev, "unknown device!?\n"); > - goto fail; > RCS file: /usr/cvs/src/sys/pci/if_tl.c,v > retrieving revision 1.74 > diff -u -r1.74 if_tl.c > --- if_tl.c 19 Feb 2003 05:47:41 - 1.74 > +++ if_tl.c 12 Mar 2003 15:20:47 - > @@ -1138,12 +1138,11 @@ > > if (t->tl_name == NULL) { > device_printf(dev, "unknown device!?\n"); > - goto fail; > + return (ENXIO); > } > > mtx_init(&sc->tl_mtx, device_get_nameunit(dev), MTX_NETWORK_LOCK, > MTX_DEF | MTX_RECURSE); > - TL_LOCK(sc); > > /* > * Map control/status registers. > @@ -1348,12 +1347,12 @@ > /* > * Call MI attach routine. > */ Thanks John -- This patch looks a little bit mangled to me. It has two sections talking about line 1138 of if_tl.c (with two different changes) and a section talking about line 1348 (with no changes). I assumed cut and paste error and proceeded along the same lines with this patch instead: Index: if_tl.c === RCS file: /usr/src/cvs-repo/src/sys/pci/if_tl.c,v retrieving revision 1.74 diff -u -r1.74 if_tl.c --- if_tl.c 19 Feb 2003 05:47:41 - 1.74 +++ if_tl.c 13 Mar 2003 00:26:20 - @@ -1138,12 +1138,11 @@ if (t->tl_name == NULL) { device_printf(dev, "unknown device!?\n"); - goto fail; + return (ENXIO); } mtx_init(&sc->tl_mtx, device_get_nameunit(dev), MTX_NETWORK_LOCK, MTX_DEF | MTX_RECURSE); - TL_LOCK(sc); /* * Map control/status registers. @@ -1349,11 +1348,9 @@ * Call MI attach routine. */ ether_ifattach(ifp, sc->arpcom.ac_enaddr); - TL_UNLOCK(sc); return(0); fail: - TL_UNLOCK(sc); mtx_destroy(&sc->tl_mtx); return(error); } This has made the messages go away -- thanks for that! If this is a correct fix, should I submit a PR to have it committed? -- Tod McQuillin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
RE: lock order reversal? current with tl ethernet
On 12-Mar-2003 Tod McQuillin wrote: > > Running -current from March 11 on a dual cpu compaq 5100, there are some > warnings in the dmesg about the tl ethernet interface. > > Here are the warnings: > > malloc() of "128" with the following non-sleepablelocks held: > exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ > /usr/src/5-current/src/sys/pci/if_tl.c:1146 It's holding the lock across bus_setup_intr(). You can try the following patch: Index: if_tl.c === RCS file: /usr/cvs/src/sys/pci/if_tl.c,v retrieving revision 1.74 diff -u -r1.74 if_tl.c --- if_tl.c 19 Feb 2003 05:47:41 - 1.74 +++ if_tl.c 12 Mar 2003 15:20:47 - @@ -1138,12 +1138,11 @@ if (t->tl_name == NULL) { device_printf(dev, "unknown device!?\n"); - goto fail; device_printf(dev, "unknown device!?\n"); - goto fail; RCS file: /usr/cvs/src/sys/pci/if_tl.c,v retrieving revision 1.74 diff -u -r1.74 if_tl.c --- if_tl.c 19 Feb 2003 05:47:41 - 1.74 +++ if_tl.c 12 Mar 2003 15:20:47 - @@ -1138,12 +1138,11 @@ if (t->tl_name == NULL) { device_printf(dev, "unknown device!?\n"); - goto fail; + return (ENXIO); } mtx_init(&sc->tl_mtx, device_get_nameunit(dev), MTX_NETWORK_LOCK, MTX_DEF | MTX_RECURSE); - TL_LOCK(sc); /* * Map control/status registers. @@ -1348,12 +1347,12 @@ /* * Call MI attach routine. */ -- John Baldwin <[EMAIL PROTECTED]> <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
lock order reversal? current with tl ethernet
Running -current from March 11 on a dual cpu compaq 5100, there are some warnings in the dmesg about the tl ethernet interface. Here are the warnings: malloc() of "128" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 malloc() of "PROC" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 lock order reversal 1st 0xc4017aa8 tl0 (network driver) @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 2nd 0xc043f8c0 allproc (allproc) @ /usr/src/5-current/src/sys/kern/kern_fork.c:328 Stack backtrace: malloc() of "64" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 malloc() of "256" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 malloc() of "64" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 malloc() of "512" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:658 I'm willing to work on this myself if someone can give me a pointer to technical docs describing how things are supposed to work. I have not yet attempted to use the tl0 interface since I also have an fxp in the system, but I do plan on using it later. Here's the complete dmesg with warnings intact: Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.0-CURRENT #0: Tue Mar 11 18:56:10 JST 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/5-current/src/sys/BOROSILICATE Preloaded elf kernel "/boot/kernel/kernel" at 0xc0565000. Timecounter "i8254" frequency 1193182 Hz CPU: Pentium II/Pentium II Xeon/Celeron (299.94-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x634 Stepping = 4 Features=0x80fbff real memory = 536870912 (512 MB) avail memory = 515575808 (491 MB) APIC_IO: MP table broken: 8259->APIC entry missing! Programming 24 pins in IOAPIC #0 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee0 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee0 io0 (APIC): apic id: 8, version: 0x00170011, at 0xfec0 Allocating major#253 to "net" Allocating major#252 to "pci" Pentium Pro MTRR support enabled npx0: on motherboard npx0: INT 16 interface pcibios: BIOS version 2.10 pcib0: at pcibus 0 on motherboard pci0: on pcib0 IOAPIC #0 intpin 19 -> irq 2 IOAPIC #0 intpin 18 -> irq 11 IOAPIC #0 intpin 17 -> irq 15 pci0: at device 3.0 (no driver attached) pci0: at device 4.0 (no driver attached) fxp0: port 0x6020-0x603f mem 0xe020-0xe02f,0xe048-0xe0480fff irq 15 at device 5.0 on pci0 fxp0: Ethernet address 00:a0:c9:c8:b6:2f inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto atapci0: port 0x6010-0x601f,0x6054-0x6057,0x6048-0x604f,0x6050-0x6053,0x6040-0x6047 mem 0xe040-0xe0403fff irq 16 at device 6.0 on pci0 ata2: at 0x6040 on atapci0 ata3: at 0x6048 on atapci0 isab0: at device 15.0 on pci0 isa0: on isab0 atapci1: port 0x6000-0x600f,0-0x3,0-0x7,0-0x3,0-0x7 irq 15 at device 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci1 ata1: simplex device, DMA on primary only ata1: at 0x170 irq 15 on atapci1 pcib1: at pcibus 1 on motherboard pci1: on pcib1 IOAPIC #0 intpin 23 -> irq 17 IOAPIC #0 intpin 20 -> irq 18 IOAPIC #0 intpin 21 -> irq 19 ohci0: mem 0xe000-0xefff irq 17 at device 10.0 on pci1 usb0: OHCI version 1.0, legacy support usb0: on ohci0 usb0: USB revision 1.0 uhub0: (0x0e11) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered tl0: port 0x5400-0x540f mem 0xe018-0xe018000f irq 18 at device 11.0 on pci1 malloc() of "128" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 malloc() of "PROC" with the following non-sleepablelocks held: exclusive sleep mutex tl0 (network driver) r = 0 (0xc4017aa8) locked @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 lock order reversal 1st 0xc4017aa8 tl0 (network driver) @ /usr/src/5-current/src/sys/pci/if_tl.c:1146 2nd 0xc043f8c0 allproc (allproc) @ /usr/src/5-current/src/sys/kern/kern_fork.c:328 Stack backtrace: malloc() of "64" with the following non-sleepablelocks held: exclu
lock order reversal
Good day ladies and gents Cvsupping and rebuilding a 5.0 current box to release Monday resulted in the following curiousness: lock order reversal 1st 0xc1190068 process lock (process lock) @ /usr/src/sys/kern/kern_descrip.c:2 112 2nd 0xc11ac934 filedesc structure (filedesc structure) @ /usr/src/sys/kern/kern _descrip.c:2119 Uname -a: FreeBSD xxx.xxx.xxx 5.0-RELEASE-p3 FreeBSD 5.0-RELEASE-p3 #2: Tue Feb 25 14:45:28 EST 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/BINKY4 i386 I did try grabbing a more recent copy of the file in question yesterday, but diff produced no diff. This appears to prevent Postgres postmaster from starting. /src and /obj were cleaned out prior to the cvsup and build so sources were indeed new (yes, I realise recent recommendations oppose this sort of thing, but it is an old 586 box with limited space). Haven't seen any new commits regarding this either (the last that appeared to touch it seems to be from the 19th - between my last cvsup on the 18th and now). Suggestions, recommendations? Cheers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: fix: lock order reversal proc/filedesc.
On Sat, Feb 15, 2003 at 11:30:09AM +1030, Greg 'groggy' Lehey wrote: > > It is becoming increasingly clear to me that the majority of FreeBSD > > developers don't really care if their code works, as long as they get > > the credit (and / or paycheck) for committing it. > > I think that's unnecessarily rude. I don't see what is so rude about DES's last paragraph. He is simply expressing his opinion about quality and why it happens. Have we really become this PC??? Geez! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: fix: lock order reversal proc/filedesc.
On Friday, 14 February 2003 at 12:23:48 +0100, Dag-Erling Smorgrav wrote: > Alfred Perlstein <[EMAIL PROTECTED]> writes: >> What exactly is broken about dumps for you on ata? > > Well, after you told me that "call dumpsys" is no longer kosher (when > did that happen, and where was it documented?), I tried 'call doadump': > > # Debugger("manual escape to debugger") > Stopped at Debugger+0x50: xchgl %ebx,in_Debugger.0 > db> call doadump > Dumping 511 MB > ata1: resetting devices .. > Context switches not allowed in the debugger. > db> call cpu_reset > > Makes me want to get my Norwegian Sword [tm] and make a short trip to > Denmark. > > It is becoming increasingly clear to me that the majority of FreeBSD > developers don't really care if their code works, as long as they get > the credit (and / or paycheck) for committing it. I think that's unnecessarily rude. Greg -- See complete headers for address and phone numbers msg52386/pgp0.pgp Description: PGP signature
Re: fix: lock order reversal proc/filedesc.
In message <[EMAIL PROTECTED]>, Dag-Erling Smorgrav writes: >Makes me want to get my Norwegian Sword [tm] and make a short trip to >Denmark. I my be reading my email out of order here, but I guess you changed your mind and tried to bury use with boatloads of useless asterix ('*') instead ? :-) >It is becoming increasingly clear to me that the majority of FreeBSD >developers don't really care if their code works, as long as they get >the credit (and / or paycheck) for committing it. I realize that you live in Norway and therefore have a bigger supply or rocks than most of us. You should still be aware that you live in a glass house yourself. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message