Re: Fwd: ZFS Crash/Pool in unhealthy state
On 17/06/2019 17:32, Larry Rosenman wrote: > Forwarding to -Current as well, since it's a -current box > > Original Message > Subject: ZFS Crash/Pool in unhealthy state > Date: 06/17/2019 8:30 am > From: Larry Rosenman > To: Freebsd fs > > I had a power failure today and my big server no longer boots > > If I try to import the pool using the memstick image and -F -f I get a panic > (see link). > > https://www.lerctr.org/~ler/ZFS_CRASH.png <--- Panic > > https://www.lerctr.org/~ler/ZFS_IMPORT.png <--- import tries. > > Ideas? Help? My reading of it is that somehow you got a situation where a block that ZFS considers as free is also queued for deferred freeing, which makes no sense. So, as soon as ZFS starts processing the deferred frees it sees the inconsistency and panics. Usually, I tend to suspect the software first, but in this case I see that the data has got other inconsistencies in the last TXG(s) before the crash. That's why import -F is required. So, I suspect that either the storage controller is misconfigured with respect to caches (its own and disks) or it otherwise misbehaves in that respect. As to the recovery. The proper way would be to import the pool read-only and transfer the data to a new pool. The more risky way would be to import the pool on a system where that consistency check is simply disabled. One way to do that is to clear bit 6 (mask 0x40) from ZFS debug flags. Or to use a STABLE FreeBSD image for an import. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Fwd: ZFS Crash/Pool in unhealthy state
On 06/17/2019 10:33 am, Larry Rosenman wrote: On 06/17/2019 10:31 am, Kurt Jaeger wrote: Hi! the drives are fine. If I import the pool readonly, zpool status is also fine. https://www.lerctr.org/~ler/ZFS_STATUS.png Interesting. Which version of FreeBSD is this box running ? -CURRENT. From Sunday. And the recovery is running from the 6/14 Snap Memstick image. to be specific the system is at R349093 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Fwd: ZFS Crash/Pool in unhealthy state
On 06/17/2019 10:31 am, Kurt Jaeger wrote: Hi! the drives are fine. If I import the pool readonly, zpool status is also fine. https://www.lerctr.org/~ler/ZFS_STATUS.png Interesting. Which version of FreeBSD is this box running ? -CURRENT. From Sunday. And the recovery is running from the 6/14 Snap Memstick image. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Fwd: ZFS Crash/Pool in unhealthy state
Hi! > the drives are fine. > > If I import the pool readonly, zpool status is also fine. > > https://www.lerctr.org/~ler/ZFS_STATUS.png Interesting. Which version of FreeBSD is this box running ? -- p...@opsec.eu+49 171 3101372One year to go ! ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Fwd: ZFS Crash/Pool in unhealthy state
On 06/17/2019 9:53 am, Kurt Jaeger wrote: Hi! Forwarding to -Current as well, since it's a -current box What is the status of the drives ? zpool status ? There's a crash recovery tool, if all else fails: https://www.klennet.com/zfs-recovery/default.aspx the drives are fine. If I import the pool readonly, zpool status is also fine. https://www.lerctr.org/~ler/ZFS_STATUS.png -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Fwd: ZFS Crash/Pool in unhealthy state
Hi! > Forwarding to -Current as well, since it's a -current box What is the status of the drives ? zpool status ? There's a crash recovery tool, if all else fails: https://www.klennet.com/zfs-recovery/default.aspx -- p...@opsec.eu+49 171 3101372One year to go ! ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Fwd: ZFS Crash/Pool in unhealthy state
Forwarding to -Current as well, since it's a -current box Original Message Subject: ZFS Crash/Pool in unhealthy state Date: 06/17/2019 8:30 am From: Larry Rosenman To: Freebsd fs I had a power failure today and my big server no longer boots If I try to import the pool using the memstick image and -F -f I get a panic (see link). https://www.lerctr.org/~ler/ZFS_CRASH.png <--- Panic https://www.lerctr.org/~ler/ZFS_IMPORT.png <--- import tries. Ideas? Help? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
