Re: zfs hang
On 10/10/2014 04:27, Steve Wills wrote: Dying drives shouldn't cause panic, right? There can be different shades of dying. Returning errors is one thing, hanging is a different thing. There is a good reason why systems panics in that case. I am surprised though that the hang was not detected in the lower layers like ahci driver or CAM. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfs hang
On Wed, Oct 08, 2014 at 08:55:26AM +0300, Andriy Gapon wrote: On 08/10/2014 03:40, Steve Wills wrote: Hi, Not sure which thread this belongs to, but I have a zfs hang on one of my boxes running r272152. Running procstat -kka looks like: http://pastebin.com/szZZP8Tf My zpool commands seem to be hung in spa_errlog_lock while others are hung in zfs_lookup. Suggestions? There are several threads in zio_wait. If this is their permanent state then there is some problem with I/O somewhere below ZFS. Thanks for the feedback. It seems one of my disks is dying, I rebooted and it came up OK, but today I got: panic: I/O to pool 'rpool' appears to be hung on vdev guid . at '/dev/ada0p3' I have screenshots and backtrace if anyone is interested. Dying drives shouldn't cause panic, right? Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfs hang
- Original Message - From: Steve Wills swi...@freebsd.org To: Andriy Gapon a...@freebsd.org Cc: curr...@freebsd.org; f...@freebsd.org Sent: Friday, October 10, 2014 2:27 AM Subject: Re: zfs hang On Wed, Oct 08, 2014 at 08:55:26AM +0300, Andriy Gapon wrote: On 08/10/2014 03:40, Steve Wills wrote: Hi, Not sure which thread this belongs to, but I have a zfs hang on one of my boxes running r272152. Running procstat -kka looks like: http://pastebin.com/szZZP8Tf My zpool commands seem to be hung in spa_errlog_lock while others are hung in zfs_lookup. Suggestions? There are several threads in zio_wait. If this is their permanent state then there is some problem with I/O somewhere below ZFS. Thanks for the feedback. It seems one of my disks is dying, I rebooted and it came up OK, but today I got: panic: I/O to pool 'rpool' appears to be hung on vdev guid . at '/dev/ada0p3' I have screenshots and backtrace if anyone is interested. Dying drives shouldn't cause panic, right? Its the deadman timer kicking in so yes, thats expected. The following sysctls control this behaviour if you want to try and recover: vfs.zfs.deadman_synctime_ms: 100 vfs.zfs.deadman_checktime_ms: 5000 vfs.zfs.deadman_enabled: 1 Regards Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfs hang
On Fri, Oct 10, 2014 at 02:35:14AM +0100, Steven Hartland wrote: - Original Message - From: Steve Wills swi...@freebsd.org To: Andriy Gapon a...@freebsd.org Cc: curr...@freebsd.org; f...@freebsd.org Sent: Friday, October 10, 2014 2:27 AM Subject: Re: zfs hang On Wed, Oct 08, 2014 at 08:55:26AM +0300, Andriy Gapon wrote: On 08/10/2014 03:40, Steve Wills wrote: Hi, Not sure which thread this belongs to, but I have a zfs hang on one of my boxes running r272152. Running procstat -kka looks like: http://pastebin.com/szZZP8Tf My zpool commands seem to be hung in spa_errlog_lock while others are hung in zfs_lookup. Suggestions? There are several threads in zio_wait. If this is their permanent state then there is some problem with I/O somewhere below ZFS. Thanks for the feedback. It seems one of my disks is dying, I rebooted and it came up OK, but today I got: panic: I/O to pool 'rpool' appears to be hung on vdev guid . at '/dev/ada0p3' I have screenshots and backtrace if anyone is interested. Dying drives shouldn't cause panic, right? Its the deadman timer kicking in so yes, thats expected. The following sysctls control this behaviour if you want to try and recover: vfs.zfs.deadman_synctime_ms: 100 vfs.zfs.deadman_checktime_ms: 5000 vfs.zfs.deadman_enabled: 1 Ah, ok. This pool has two disks, mirrored. I think one of them is dying, the BIOS gives a SMART error on startup, but it still uses the disk fine. From what I read of the zfs deadman design, it's for when the controller is acting up. So I'm confused. Maybe this means both disks are dying? Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
zfs hang
Hi, Not sure which thread this belongs to, but I have a zfs hang on one of my boxes running r272152. Running procstat -kka looks like: http://pastebin.com/szZZP8Tf My zpool commands seem to be hung in spa_errlog_lock while others are hung in zfs_lookup. Suggestions? Thanks, Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfs hang
On 08/10/2014 03:40, Steve Wills wrote: Hi, Not sure which thread this belongs to, but I have a zfs hang on one of my boxes running r272152. Running procstat -kka looks like: http://pastebin.com/szZZP8Tf My zpool commands seem to be hung in spa_errlog_lock while others are hung in zfs_lookup. Suggestions? There are several threads in zio_wait. If this is their permanent state then there is some problem with I/O somewhere below ZFS. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
