On Thu, 2006-05-11 at 17:01 -0700, Jeff Bonwick wrote: > > plan A. To mirror on iSCSI devices: > > keep one server with a set of zfs file systems > > with 2 (sub)mirrors each, one of the mirrors use > > devices physically on remote site accessed as > > iSCSI LUNs. > > > > How does ZFS handle remote replication? > > If the Internet link is down for hours or days, > > can the file systems still be written? Will > > the submirrors be resync'ed efficiently? > > This would work. If the link goes down, it's no different than > if someone trips over the cable for a local disk. All writes > (and reads) will go to the local disk until the remote one returns.
By mirroring remotely, you create a dependency between the two hosts and their interconnects. In general, this will be noticeable by performance bottlenecks as the system is only as fast as the slowest component (ala a chain is as strong as its weakest link). >From a practical perspective, this tends to increase the fault detection time. In the local disk case, if you unplug the disk it is either immediately detected (SAS, SATA) or quickly detected (direct FC, parallel SCSI). For IP networks, the detection may take some number of minutes before the various timeouts expire. The timeouts exist to provide stability, so you can't just tune them short without risking instability. However, it is rare that people do such analysis. As to what ZFS does when one side of the mirror starts having very long delays, I'm not sure. For most other LVMs, they just slow down. Architecturally, it is usually not in your best interest to have performance in a primary datacenter depend directly on the performance of the secondary datacenter and the link between. Using a one-way replication such as ZFS send/receive will allow you to break this dependency. > When the remote disk returns, we'll resilver it. The cool thing here > is that ZFS resilvering is logical, not physical, so it'll only copy > the blocks that changed during the outage (i.e., it'll be fast). > For a bit more detail on how ZFS mirroring works, see: > > http://blogs.sun.com/roller/page/bonwick?entry=smokin_mirrors > > The one hesitation I'd have about Plan A is that ZFS doesn't yet > support the notion of two sides of a mirror being very different in > performance. With a local/remote pair, you really want different > semantics than a pair of local disks. You want to send all reads > to the local disk, and you want to consider a write complete when > the local disk is done (and let the remote write be asynchronous). > We're planning to do this soon, but it's not there yet. Yes, preferred policy will be appreciated. This would be useful in the case where I have local devices which are different, such as [NV]RAM on one side and spinning rust on the other. -- richard _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
