Hi Jeff, Somone from the ZFS team might want to comment, but I've been down this road a bit so I thought I'd add my two cents.
ZFS doesn't let you create a mirror of RAID-Z vdevs on the command line, but it is possible if you're willing to construct such a configuration programmatically (in fact, this is what's done for the ztest test program in usr/src/cmd/ztest/ztest.c). An important constraint on mirrors in ZFS is that all elements have to have identical configurations. That is, you can't have one half be RAID-Z and the other be a raw defive or file. Unfortunately, that's exactly what you'd have if you exported a ZVOL backed by RAID-Z over iSCSI. What you'd need to do is export the individual disks via iSCSI and then build the RAID-Z vdev on the data manipulating machine (the "central site that would logically host the data"). This has some obvious deficiencies -- you're shipping parity over the wire -- but it works. I don't know the current state of remote replication in ZFS, but that might provide a more efficient solution at some point. Adam On Thu, Oct 05, 2006 at 01:50:08PM -0700, Jeff Anderson-Lee wrote: > We're setting up a large data store and and would like to use zfs with > some sort of distributed mirroring for disaster recovery. We have 3 > "sites" in mind: two machine rooms in one building and one in another > across the street, all connected by GbT networking. > > Our thought was to keep 3 mirrored copies of the data, each protected by > Raid-5 (raid-z?). Remote sites would export raw or raid-5/z protected > disk space using iscsi to the central site that would logically host the > data. > > Given the size of the data sets (~100TB), 3-way mirroring alone is (on > paper at least) insufficient. Given the number of blocks involved, the > chances of all three copies being bad for *some* logical block are too > large. On the other hand, 6-way mirroring seems excessive. (Our > original concept was to use 4-of-15 erasure coding, so that any 4 of 15 > related blocks would be enough for disaster recovery, while each site > would normally hold 5 blocks, allowing for Raid-5 style local recovery > of single failed blocks/disks.) > > Questions: > > 1) is it possible to layer mirroring on top of Raid-Z under ZFS? > > 2) Is is possible/practical to export raw disks or raid5/raidZ disk > space using iscsi? > > 3) Are there some other, easier alternatives we haven't thought of? > > Thanks in advance. > > Jeff Anderson-Lee > Petabyte Storage Infrastructure Project > University of California at Berkeley > > > > > _______________________________________________ > storage-discuss mailing list > [email protected] > http://mail.opensolaris.org/mailman/listinfo/storage-discuss -- Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl _______________________________________________ storage-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/storage-discuss
