Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 15:28, Matthew Seaman wrote: On 05/07/15 14:32, Steven Hartland wrote: I wouldn't have thought so, I would expect that to only have an effect on removal media such as CDROM drives, but no harm in trying ;-) zpool offline -t zroot da19 That might work but it also might just we

Re: panic: pmap active 0xfffff8001b7154b8

2015-05-07 Thread Johan Schuijt-Li
> > What we really need is a full core dump (minidump) and backtrace. This > will let us inspect the pmap state. > > https://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug.html > > https://www.freebsd.org/d

Re: panic: pmap active 0xfffff8001b7154b8

2015-05-07 Thread Bryan Drewery
On 5/7/2015 10:06 AM, Bryan Drewery wrote: > On 5/7/2015 7:08 AM, Johan Schuijt-Li wrote: >> Hi, >> >> We’ve been seeing (seemingly) random reboots on 10.1-RELEASE virtual >> machines (KVM virtualisation) on our production servers. In an attempt to >> determine what was causing this we’ve switche

Re: panic: pmap active 0xfffff8001b7154b8

2015-05-07 Thread Bryan Drewery
On 5/7/2015 7:08 AM, Johan Schuijt-Li wrote: > Hi, > > We’ve been seeing (seemingly) random reboots on 10.1-RELEASE virtual machines > (KVM virtualisation) on our production servers. In an attempt to determine > what was causing this we’ve switched to running a kernel with INVARIANTS > enabled.

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Matthew Seaman
On 05/07/15 14:32, Steven Hartland wrote: > > > On 07/05/2015 14:29, Ronald Klop wrote: >> On Thu, 07 May 2015 15:23:58 +0200, Steven Hartland >> wrote: >> >>> >>> >>> On 07/05/2015 14:10, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 02:05:11PM +0100, Steven Hartland wrote: >

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Paul Mather
On May 7, 2015, at 8:00 AM, Steven Hartland wrote: > On 07/05/2015 11:46, Slawa Olhovchenkov wrote: >> On Thu, May 07, 2015 at 11:38:46AM +0100, Steven Hartland wrote: >> >> How I can cancel this 24 requst? >> Why this requests don't timeout (3 hours already)? >> How I can forced det

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Ronald Klop
On Thu, 07 May 2015 15:23:58 +0200, Steven Hartland wrote: On 07/05/2015 14:10, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 02:05:11PM +0100, Steven Hartland wrote: On 07/05/2015 13:51, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 01:46:40PM +0100, Steven Hartland wrote: Ye

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
On Thu, May 07, 2015 at 03:29:20PM +0200, Ronald Klop wrote: > On Thu, 07 May 2015 15:23:58 +0200, Steven Hartland > wrote: > > > > > > > On 07/05/2015 14:10, Slawa Olhovchenkov wrote: > >> On Thu, May 07, 2015 at 02:05:11PM +0100, Steven Hartland wrote: > >> > >>> > >>> On 07/05/2015 13:51, S

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 14:29, Ronald Klop wrote: On Thu, 07 May 2015 15:23:58 +0200, Steven Hartland wrote: On 07/05/2015 14:10, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 02:05:11PM +0100, Steven Hartland wrote: On 07/05/2015 13:51, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 01

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 14:10, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 02:05:11PM +0100, Steven Hartland wrote: On 07/05/2015 13:51, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 01:46:40PM +0100, Steven Hartland wrote: Yes in theory new requests should go to the other vdev, but there

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
On Thu, May 07, 2015 at 02:05:11PM +0100, Steven Hartland wrote: > > > On 07/05/2015 13:51, Slawa Olhovchenkov wrote: > > On Thu, May 07, 2015 at 01:46:40PM +0100, Steven Hartland wrote: > > > Yes in theory new requests should go to the other vdev, but there could > be some dependency

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 13:51, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 01:46:40PM +0100, Steven Hartland wrote: Yes in theory new requests should go to the other vdev, but there could be some dependency issues preventing that such as a syncing TXG. Currenly this pool must not have write acti

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
On Thu, May 07, 2015 at 01:46:40PM +0100, Steven Hartland wrote: > >> Yes in theory new requests should go to the other vdev, but there could > >> be some dependency issues preventing that such as a syncing TXG. > > Currenly this pool must not have write activity (from application). > > What about

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 13:44, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 01:35:05PM +0100, Steven Hartland wrote: On 07/05/2015 13:05, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 01:00:40PM +0100, Steven Hartland wrote: On 07/05/2015 11:46, Slawa Olhovchenkov wrote: On Thu, May 07, 2

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
On Thu, May 07, 2015 at 01:35:05PM +0100, Steven Hartland wrote: > > > On 07/05/2015 13:05, Slawa Olhovchenkov wrote: > > On Thu, May 07, 2015 at 01:00:40PM +0100, Steven Hartland wrote: > > > >> > >> On 07/05/2015 11:46, Slawa Olhovchenkov wrote: > >>> On Thu, May 07, 2015 at 11:38:46AM +0100,

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 13:05, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 01:00:40PM +0100, Steven Hartland wrote: On 07/05/2015 11:46, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 11:38:46AM +0100, Steven Hartland wrote: How I can cancel this 24 requst? Why this requests don't timeout

panic: pmap active 0xfffff8001b7154b8

2015-05-07 Thread Johan Schuijt-Li
Hi, We’ve been seeing (seemingly) random reboots on 10.1-RELEASE virtual machines (KVM virtualisation) on our production servers. In an attempt to determine what was causing this we’ve switched to running a kernel with INVARIANTS enabled. This resulted for us in the following panic: Unread por

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
On Thu, May 07, 2015 at 01:00:40PM +0100, Steven Hartland wrote: > > > On 07/05/2015 11:46, Slawa Olhovchenkov wrote: > > On Thu, May 07, 2015 at 11:38:46AM +0100, Steven Hartland wrote: > > > > How I can cancel this 24 requst? > > Why this requests don't timeout (3 hours already)? > >>>

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 11:46, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 11:38:46AM +0100, Steven Hartland wrote: How I can cancel this 24 requst? Why this requests don't timeout (3 hours already)? How I can forced detach this disk? (I am lready try `camcontrol reset`, `camconrol rescan`). Why

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
On Thu, May 07, 2015 at 11:38:46AM +0100, Steven Hartland wrote: > >>> How I can cancel this 24 requst? > >>> Why this requests don't timeout (3 hours already)? > >>> How I can forced detach this disk? (I am lready try `camcontrol reset`, > >>> `camconrol rescan`). > >>> Why ZFS (or geom) don't t

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 10:50, Slawa Olhovchenkov wrote: On Thu, May 07, 2015 at 09:41:43AM +0100, Steven Hartland wrote: On 07/05/2015 09:07, Slawa Olhovchenkov wrote: I have zpool of 12 vdev (zmirrors). One disk in one vdev out of service and stop serving reuquest: dT: 1.036s w: 1.000s L(q) op

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
On Thu, May 07, 2015 at 09:41:43AM +0100, Steven Hartland wrote: > On 07/05/2015 09:07, Slawa Olhovchenkov wrote: > > I have zpool of 12 vdev (zmirrors). > > One disk in one vdev out of service and stop serving reuquest: > > > > dT: 1.036s w: 1.000s > > L(q) ops/sr/s kBps ms/rw/s

Re: zfs, cam sticking on failed disk

2015-05-07 Thread Steven Hartland
On 07/05/2015 09:07, Slawa Olhovchenkov wrote: I have zpool of 12 vdev (zmirrors). One disk in one vdev out of service and stop serving reuquest: dT: 1.036s w: 1.000s L(q) ops/sr/s kBps ms/rw/s kBps ms/w %busy Name 0 0 0 00.0 0 00.0

zfs, cam sticking on failed disk

2015-05-07 Thread Slawa Olhovchenkov
I have zpool of 12 vdev (zmirrors). One disk in one vdev out of service and stop serving reuquest: dT: 1.036s w: 1.000s L(q) ops/sr/s kBps ms/rw/s kBps ms/w %busy Name 0 0 0 00.0 0 00.00.0| ada0 0 0 0 00.0 0