> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
> boun...@opensolaris.org] On Behalf Of Jim Klimov
> 2) In particular, if I wanted to make a mirror of raidzN's,
> can it be done in one ZFS pool, or would I have to play
> with iSCSI and ZVOLs?
Although you can probably do something like this (which might give you some
problems in terms of mount order, etc) it's probably not a great idea. One
part of the system will be optimizing for mirrored disks, another optimizing
for a raidz. Block patterns may not (probably won't) align, possibly
resulting in some performance issues... You'll be double-checksumming...
If you want to mirror raidz devices, why don't you just go with a mirror and
no raidz? The mirror performs better. The only reason I can think of to do
what you're saying is if you want more redundancy than a 2-way mirror, but
not as much as a 3-way mirror. When they designed zfs, they weren't
designing for you to have THAT level of redundancy precision. The more you
stray off the beaten path, the more you discover the things other people
My advice would be to stick with the 2-way or 3-way mirror. Don't trick the
system into using multi-layered vdev's when it's not intended to be used
that way. Not saying you can't - just that it's not intended, and hence,
not well tested and not recommended.
> 3) If I have two separate ZFS nodes with local storage,
> intended for failure-tolerant HA, which approach is better -
> to create a mirror over ZVOLs (importing a remote ZVOL over
> iSCSI and loopback), or to replicate using frequent snapshots
> and zfs send/zfs recv?
Each way has some pros & cons.
If you attach the remote zvol to a mirror, it will start resilvering
(latency intensive) and suddenly all your present disk activities will be
bottlenecked to remote system speeds. Maybe not an issue if you're
connecting over infiniband or something super fast like it. The pro of
doing things this way is - if the local side of the mirror dies for some
reason, the storage doesn't go down. So you're giving up performance in
order to gain reliability.
If you use zfs send/receive, your local system will still be able to perform
at local system speeds while it's sending an optimized network data stream
over to the other side. The downside is if the local storage system fails,
your storage system goes down, and you will only have the most recently sent
snapshot over on the remote side, so you could potentially lose some
granularity in the most recent data. You're gaining performance at the
expense of risk to your most recent data.
Actually - There's a situation where the zfs send is actually MORE reliable
too. Although not very common, it certainly CAN happen, and HAS happened,
that live replicating systems (such as remotely attached mirror) both get
destroyed at the same time. Whatever destroys the primary storage pool
(such as a sysadmin accidentally destroying the wrong zpool) also
simultaneously destroys the backup data pool. So the added layer of
separation by using the zfs send might be desired.
Also, if you mirror the remote zvol, you'll be constrained to using the same
size/geometry/configuration of storage on the local and remote sides.
Usually, when I build systems, I use a better class of hardware on the
primary server, and a cheaper class of hardware on the backup server.
Say... Use SAS on the primary and SATA on the backup. And use the SSD ZIL
on the primary, but simply disable ZIL on the backup server. And use twice
as many half-the-size disks on the primary, while using half-as-many
twice-the-size disks on the backup. Use mirror on the primary, use raidz2
on the backup. Enable compression on the backup but no compression on the
primary. Etc. (Actually, in most cases, you *gain* performance by adding
compression to both, depending on your data.)
zfs-discuss mailing list