Re: [zfs-discuss] Recovering lost labels on raidz member
On 08/13/2012 03:02 AM, Scott wrote: Hi all, I have a 5 disk raidz array in a state of disrepair. Suffice to say three disks are ok, while two are missing all their labels. (Both ends of the disks were overwritten). The data is still intact. There are 4 labels on a zfs-labeled disk, two at the start and two at the end. Have all been overwritten? Unfortunately I don't have a zpool.cache either. Is there a way to reconstruct the labels using the infomration from the 3 valid disks? The simple answer is: no, without labels those disks are toast. The more complicated answer: the labels contain the root block pointer to the root of the filesystem tree on the given device. Without it you don't know where to start looking for many of the top-level structures needed to begin making sense of the data on the disk. -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering lost labels on raidz member
On 08/13/2012 10:00 AM, Sašo Kiselkov wrote: On 08/13/2012 03:02 AM, Scott wrote: Hi all, I have a 5 disk raidz array in a state of disrepair. Suffice to say three disks are ok, while two are missing all their labels. (Both ends of the disks were overwritten). The data is still intact. There are 4 labels on a zfs-labeled disk, two at the start and two at the end. Have all been overwritten? Just re-read your post again, and I realized my question here is redundant. Without the labels your data is toast. -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering lost labels on raidz member
Hi Saso, thanks for your reply. If all disks are the same, is the root pointer the same? Also, is there a signature or something unique to the root block that I can search for on the disk? I'm going through the On-disk specification at the moment. Scott On Mon, Aug 13, 2012 at 10:02:58AM +0200, Sa?o Kiselkov wrote: On 08/13/2012 10:00 AM, Sa?o Kiselkov wrote: On 08/13/2012 03:02 AM, Scott wrote: Hi all, I have a 5 disk raidz array in a state of disrepair. Suffice to say three disks are ok, while two are missing all their labels. (Both ends of the disks were overwritten). The data is still intact. There are 4 labels on a zfs-labeled disk, two at the start and two at the end. Have all been overwritten? Just re-read your post again, and I realized my question here is redundant. Without the labels your data is toast. -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering lost labels on raidz member
On 08/13/2012 10:45 AM, Scott wrote: Hi Saso, thanks for your reply. If all disks are the same, is the root pointer the same? No. Also, is there a signature or something unique to the root block that I can search for on the disk? I'm going through the On-disk specification at the moment. Nope. The checksums are part of the blockpointer, and the root blockpointer is in the uberblock, which itself resides in the label. By overwriting the label you've essentially erased all hope of practically finding the root of the filesystem tree - not even checksumming all possible block combinations (of which there are quite a few) will help you here, because you have no checksums to compare them against. I'd love to be wrong, and I might be (I don't have as intimate a knowledge of ZFS' on-disk structure as I'd like), but from where I'm standing, your raidz vdev is essentially lost. -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering lost labels on raidz member
Thanks again Saso, at least I have closure :) Scott On Mon, Aug 13, 2012 at 11:24:55AM +0200, Sa?o Kiselkov wrote: On 08/13/2012 10:45 AM, Scott wrote: Hi Saso, thanks for your reply. If all disks are the same, is the root pointer the same? No. Also, is there a signature or something unique to the root block that I can search for on the disk? I'm going through the On-disk specification at the moment. Nope. The checksums are part of the blockpointer, and the root blockpointer is in the uberblock, which itself resides in the label. By overwriting the label you've essentially erased all hope of practically finding the root of the filesystem tree - not even checksumming all possible block combinations (of which there are quite a few) will help you here, because you have no checksums to compare them against. I'd love to be wrong, and I might be (I don't have as intimate a knowledge of ZFS' on-disk structure as I'd like), but from where I'm standing, your raidz vdev is essentially lost. -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering lost labels on raidz member
On Aug 13, 2012, at 2:24 AM, Sašo Kiselkov wrote: On 08/13/2012 10:45 AM, Scott wrote: Hi Saso, thanks for your reply. If all disks are the same, is the root pointer the same? No. Also, is there a signature or something unique to the root block that I can search for on the disk? I'm going through the On-disk specification at the moment. Nope. The checksums are part of the blockpointer, and the root blockpointer is in the uberblock, which itself resides in the label. By overwriting the label you've essentially erased all hope of practically finding the root of the filesystem tree - not even checksumming all possible block combinations (of which there are quite a few) will help you here, because you have no checksums to compare them against. I'd love to be wrong, and I might be (I don't have as intimate a knowledge of ZFS' on-disk structure as I'd like), but from where I'm standing, your raidz vdev is essentially lost. The labels are not identical, because each contains the guid for the device. It is possible, though nontrivial, to recreate. That said, I've never seen a failure that just takes out only the ZFS labels. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering lost labels on raidz member
On Mon, Aug 13, 2012 at 10:40:45AM -0700, Richard Elling wrote: On Aug 13, 2012, at 2:24 AM, Sa?o Kiselkov wrote: On 08/13/2012 10:45 AM, Scott wrote: Hi Saso, thanks for your reply. If all disks are the same, is the root pointer the same? No. Also, is there a signature or something unique to the root block that I can search for on the disk? I'm going through the On-disk specification at the moment. Nope. The checksums are part of the blockpointer, and the root blockpointer is in the uberblock, which itself resides in the label. By overwriting the label you've essentially erased all hope of practically finding the root of the filesystem tree - not even checksumming all possible block combinations (of which there are quite a few) will help you here, because you have no checksums to compare them against. I'd love to be wrong, and I might be (I don't have as intimate a knowledge of ZFS' on-disk structure as I'd like), but from where I'm standing, your raidz vdev is essentially lost. The labels are not identical, because each contains the guid for the device. It is possible, though nontrivial, to recreate. That said, I've never seen a failure that just takes out only the ZFS labels. You'd have to go out of your way to take out the labels. Which is just what I did (imagine: moving drives over to USB external enclosures, then putting them onto a HP Raid controller (which overwrites the end of the disk) - which also assumed that two disks should be automatically mirrored (if you miss the 5 second prompt where you can tell it not to). Then try and recover the labels without really knowing what you're doing (my bad). Suffice to say I have no confidence in the labels of two drives. On OI I can forcefully import the pool but with any file that lives on multiple disks (ie, over a certain size), all I get is an I/O error. Some of datasets also fail to mount. Thanks everyone for your input. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering lost labels on raidz member
On Aug 13, 2012, at 8:59 PM, Scott wrote: On Mon, Aug 13, 2012 at 10:40:45AM -0700, Richard Elling wrote: On Aug 13, 2012, at 2:24 AM, Sa?o Kiselkov wrote: On 08/13/2012 10:45 AM, Scott wrote: Hi Saso, thanks for your reply. If all disks are the same, is the root pointer the same? No. Also, is there a signature or something unique to the root block that I can search for on the disk? I'm going through the On-disk specification at the moment. Nope. The checksums are part of the blockpointer, and the root blockpointer is in the uberblock, which itself resides in the label. By overwriting the label you've essentially erased all hope of practically finding the root of the filesystem tree - not even checksumming all possible block combinations (of which there are quite a few) will help you here, because you have no checksums to compare them against. I'd love to be wrong, and I might be (I don't have as intimate a knowledge of ZFS' on-disk structure as I'd like), but from where I'm standing, your raidz vdev is essentially lost. The labels are not identical, because each contains the guid for the device. It is possible, though nontrivial, to recreate. That said, I've never seen a failure that just takes out only the ZFS labels. You'd have to go out of your way to take out the labels. Which is just what I did (imagine: moving drives over to USB external enclosures, then putting them onto a HP Raid controller (which overwrites the end of the disk) - which also assumed that two disks should be automatically mirrored (if you miss the 5 second prompt where you can tell it not to). ouch. But that shouldn't be enough. Then try and recover the labels without really knowing what you're doing (my bad). d'oh! Suffice to say I have no confidence in the labels of two drives. On OI I can forcefully import the pool but with any file that lives on multiple disks (ie, over a certain size), all I get is an I/O error. Some of datasets also fail to mount. please tell me you imported readonly? -- richard Thanks everyone for your input. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Recovering lost labels on raidz member
Hi all, I have a 5 disk raidz array in a state of disrepair. Suffice to say three disks are ok, while two are missing all their labels. (Both ends of the disks were overwritten). The data is still intact. Unfortunately I don't have a zpool.cache either. Is there a way to reconstruct the labels using the infomration from the 3 valid disks? Thanks Scott ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
