Re: Panic after USB deadlock followed by kldunload umass.ko
Fabian Keil freebsd-lis...@fabiankeil.de wrote: Hans Petter Selasky h...@selasky.org wrote: On 10/23/14 16:28, Fabian Keil wrote: kldunload umass.ko lead to a panic, dumping didn't work. Screenshots are available at: http://www.fabiankeil.de/bilder/freebsd/kernel-panic-r273434-usb/ I've seen locked-up usbconfig processes in the past, usually after executing a shell function that does: | usbconfig_output=$(sudo usbconfig -d ${device} add_quirk UQ_MSC_NO_INQUIRY) | [... error handling snipped ] | usbconfig_output=$(sudo usbconfig -d ${device} reset) Sometimes the second command seems to mess up the USB system. USB detach is synchronous. If for example umass fails to detach, due to reference counts not reaching zero, that might be the root cause. Try to get a backtrace from the usb_process() processes using kgdb, and you'll see right away what is going on. Thanks for the quick response. So far I haven't been able to intentionally reproduce the issue, but I'll try the above the next time I unintentionally run into this again. A couple of days ago usbconfig got stuck again: fk@r500 ~ $sudo procstat -kk $(pgrep usbconfig) PIDTID COMM TDNAME KSTACK 2444 100971 usbconfig-mi_switch+0xe1 sleepq_wait+0x3a _sx_xlock_hard+0x522 _sx_xlock+0x5d usbd_enum_lock+0x3a usb_ref_device+0x157 usb_open+0xbf devfs_open+0x122 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x310 kern_openat+0x26f amd64_syscall+0x3fd Xfast_syscall+0xfb 2425 100970 usbconfig-mi_switch+0xe1 sleepq_timedwait+0x3a _sleep+0x294 pause_sbt+0xd0 usb_pause_mtx+0x85 usb_ioctl+0x3e7 devfs_ioctl_f+0x13b kern_ioctl+0x3ce sys_ioctl+0x140 amd64_syscall+0x3fd Xfast_syscall+0xfb The USB-related threads: (kgdb) thread 520 [Switching to thread 520 (Thread 100970)]#0 sched_switch (td=0xf8001d2044a0, newtd=value optimized out, flags=value optimized out) at /usr/src/sys/kern/sched_ule.c:1940 1940cpuid = PCPU_GET(cpuid); (kgdb) where #0 sched_switch (td=0xf8001d2044a0, newtd=value optimized out, flags=value optimized out) at /usr/src/sys/kern/sched_ule.c:1940 #1 0x805bbf91 in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:492 #2 0x80601a7a in sleepq_timedwait (wchan=0x0, pri=0) at /usr/src/sys/kern/subr_sleepqueue.c:666 #3 0x805bb984 in _sleep (ident=value optimized out, lock=value optimized out, priority=value optimized out, wmesg=value optimized out, sbt=value optimized out, pr=value optimized out, flags=value optimized out) at /usr/src/sys/kern/kern_synch.c:250 #4 0x805bbe10 in pause_sbt (wmesg=value optimized out, sbt=value optimized out, pr=0, flags=value optimized out) at /usr/src/sys/kern/kern_synch.c:379 #5 0x81e2f175 in usb_pause_mtx (mtx=value optimized out, timo=value optimized out) at /usr/src/sys/modules/usb/usb/../../../dev/usb/usb_util.c:143 #6 0x81e16ed7 in usb_ioctl (dev=value optimized out, cmd=value optimized out, addr=value optimized out, fflag=0, td=value optimized out) at /usr/src/sys/modules/usb/usb/../../../dev/usb/usb_dev.c:1128 #7 0x8047cffb in devfs_ioctl_f (fp=0xf8001d3d1b40, com=2147767558, data=0xfe009453aa20, cred=value optimized out, td=0xf8001d2044a0) at /usr/src/sys/fs/devfs/devfs_vnops.c:775 #8 0x8061041e in kern_ioctl (td=0xf8001d2044a0, fd=value optimized out, com=0) at file.h:318 #9 0x8060ffa0 in sys_ioctl (td=0xf8001d2044a0, uap=0xfe009453ab80) at /usr/src/sys/kern/sys_generic.c:718 #10 0x80877d5d in amd64_syscall (td=0xf8001d2044a0, traced=0) at subr_syscall.c:133 #11 0x8085ac9b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:390 #12 0x000800b7caea in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) thread 522 [Switching to thread 522 (Thread 100971)]#0 sched_switch (td=0xf8001d3a5940, newtd=value optimized out, flags=value optimized out) at /usr/src/sys/kern/sched_ule.c:1940 1940cpuid = PCPU_GET(cpuid); (kgdb) where #0 sched_switch (td=0xf8001d3a5940, newtd=value optimized out, flags=value optimized out) at /usr/src/sys/kern/sched_ule.c:1940 #1 0x805bbf91 in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:492 #2 0x8060126a in sleepq_wait (wchan=0x0, pri=0) at /usr/src/sys/kern/subr_sleepqueue.c:631 #3 0x805bae82 in _sx_xlock_hard (sx=0xf80015e82050, tid=18446735278106892608, opts=value optimized out, file=0x0, line=0) at /usr/src/sys/kern/kern_sx.c:688 #4 0x805ba6ad in _sx_xlock (sx=0x0, opts=0, file=value optimized out, line=0) at sx.h:154 #5 0x81e19baa in usbd_enum_lock (udev=0xf80015e82000) at /usr/src/sys/modules/usb/usb/../../../dev/usb/usb_device.c:2755 #6 0x81e18967 in usb_ref_device
Re: Panic after USB deadlock followed by kldunload umass.ko
On 12/21/14 14:54, Fabian Keil wrote: #0 sched_switch (td=0xf80015ea0940, newtd=value optimized out, flags=value optimized out) at /usr/src/sys/kern/sched_ule.c:1940 #1 0x805bbf91 in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:492 #2 0x8060126a in sleepq_wait (wchan=0x0, pri=0) at /usr/src/sys/kern/subr_sleepqueue.c:631 #3 0x805bb99d in _sleep (ident=value optimized out, lock=value optimized out, priority=value optimized out, wmesg=value optimized out, sbt=value optimized out, pr=value optimized out, flags=value optimized out) at /usr/src/sys/kern/kern_synch.c:254 #4 0x802a2548 in cam_sim_free (sim=0xf8004f9fa800, free_devq=1) at /usr/src/sys/cam/cam_sim.c:109 Because cam_sim_free() doesn't return, the USB subsystem belonging to that controller is somewhat blocked. --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Panic after USB deadlock followed by kldunload umass.ko
Hans Petter Selasky h...@selasky.org wrote: On 10/23/14 16:28, Fabian Keil wrote: kldunload umass.ko lead to a panic, dumping didn't work. Screenshots are available at: http://www.fabiankeil.de/bilder/freebsd/kernel-panic-r273434-usb/ I've seen locked-up usbconfig processes in the past, usually after executing a shell function that does: | usbconfig_output=$(sudo usbconfig -d ${device} add_quirk UQ_MSC_NO_INQUIRY) | [... error handling snipped ] | usbconfig_output=$(sudo usbconfig -d ${device} reset) Sometimes the second command seems to mess up the USB system. USB detach is synchronous. If for example umass fails to detach, due to reference counts not reaching zero, that might be the root cause. Try to get a backtrace from the usb_process() processes using kgdb, and you'll see right away what is going on. Thanks for the quick response. So far I haven't been able to intentionally reproduce the issue, but I'll try the above the next time I unintentionally run into this again. Fabian signature.asc Description: PGP signature
Panic after USB deadlock followed by kldunload umass.ko
A few days ago a couple of usbconfig processed did not exit: fk@r500 ~ $sudo procstat -kk $(pgrep usbconfig) PIDTID COMM TDNAME KSTACK 1624 100781 usbconfig-mi_switch+0xe1 sleepq_wait+0x3a _sx_xlock_hard+0x522 _sx_xlock+0x5d usbd_enum_lock+0x3a usb_ref_device+0x157 usb_open+0xbf devfs_open+0x122 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x351 kern_openat+0x26f amd64_syscall+0x3fb Xfast_syscall+0xfb 1617 100779 usbconfig-mi_switch+0xe1 sleepq_wait+0x3a _sx_xlock_hard+0x522 _sx_xlock+0x5d usbd_enum_lock+0x3a usb_ref_device+0x157 usb_open+0xbf devfs_open+0x122 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x351 kern_openat+0x26f amd64_syscall+0x3fb Xfast_syscall+0xfb 1615 100777 usbconfig-mi_switch+0xe1 sleepq_wait+0x3a _sx_xlock_hard+0x522 _sx_xlock+0x5d usbd_enum_lock+0x3a usb_ref_device+0x157 usb_open+0xbf devfs_open+0x122 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x351 kern_openat+0x26f amd64_syscall+0x3fb Xfast_syscall+0xfb 1601 100774 usbconfig-mi_switch+0xe1 sleepq_timedwait+0x3a _sleep+0x294 pause_sbt+0xd0 usb_pause_mtx+0x85 usb_ioctl+0x3e7 devfs_ioctl_f+0x13b kern_ioctl+0x3cd sys_ioctl+0x13c amd64_syscall+0x3fb Xfast_syscall+0xfb kldunload umass.ko lead to a panic, dumping didn't work. Screenshots are available at: http://www.fabiankeil.de/bilder/freebsd/kernel-panic-r273434-usb/ I've seen locked-up usbconfig processes in the past, usually after executing a shell function that does: | usbconfig_output=$(sudo usbconfig -d ${device} add_quirk UQ_MSC_NO_INQUIRY) | [... error handling snipped ] | usbconfig_output=$(sudo usbconfig -d ${device} reset) Sometimes the second command seems to mess up the USB system. Fabian signature.asc Description: PGP signature
Re: Panic after USB deadlock followed by kldunload umass.ko
On 10/23/14 16:28, Fabian Keil wrote: A few days ago a couple of usbconfig processed did not exit: fk@r500 ~ $sudo procstat -kk $(pgrep usbconfig) PIDTID COMM TDNAME KSTACK 1624 100781 usbconfig-mi_switch+0xe1 sleepq_wait+0x3a _sx_xlock_hard+0x522 _sx_xlock+0x5d usbd_enum_lock+0x3a usb_ref_device+0x157 usb_open+0xbf devfs_open+0x122 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x351 kern_openat+0x26f amd64_syscall+0x3fb Xfast_syscall+0xfb 1617 100779 usbconfig-mi_switch+0xe1 sleepq_wait+0x3a _sx_xlock_hard+0x522 _sx_xlock+0x5d usbd_enum_lock+0x3a usb_ref_device+0x157 usb_open+0xbf devfs_open+0x122 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x351 kern_openat+0x26f amd64_syscall+0x3fb Xfast_syscall+0xfb 1615 100777 usbconfig-mi_switch+0xe1 sleepq_wait+0x3a _sx_xlock_hard+0x522 _sx_xlock+0x5d usbd_enum_lock+0x3a usb_ref_device+0x157 usb_open+0xbf devfs_open+0x122 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x351 kern_openat+0x26f amd64_syscall+0x3fb Xfast_syscall+0xfb 1601 100774 usbconfig-mi_switch+0xe1 sleepq_timedwait+0x3a _sleep+0x294 pause_sbt+0xd0 usb_pause_mtx+0x85 usb_ioctl+0x3e7 devfs_ioctl_f+0x13b kern_ioctl+0x3cd sys_ioctl+0x13c amd64_syscall+0x3fb Xfast_syscall+0xfb kldunload umass.ko lead to a panic, dumping didn't work. Screenshots are available at: http://www.fabiankeil.de/bilder/freebsd/kernel-panic-r273434-usb/ I've seen locked-up usbconfig processes in the past, usually after executing a shell function that does: | usbconfig_output=$(sudo usbconfig -d ${device} add_quirk UQ_MSC_NO_INQUIRY) | [... error handling snipped ] | usbconfig_output=$(sudo usbconfig -d ${device} reset) Sometimes the second command seems to mess up the USB system. Fabian Hi, USB detach is synchronous. If for example umass fails to detach, due to reference counts not reaching zero, that might be the root cause. Try to get a backtrace from the usb_process() processes using kgdb, and you'll see right away what is going on. Thank you! --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org