On 4/4/2017 6:59 PM, Konstantin Olchanski wrote:
Moving to ZFS...
ZFS is also scary...
Heh - another soon to be victim of ZFS on linux :)
No kidding. Former victim of XLV+XFS (remember XLV?), former
victim of LV+EFS, former victim of ext2, ext3, reiserfs, former
victim of LVM, current victim of mdadm/raid5/6/ext4/xfs.
You'll quickly realise that the majority of major features you'd expect
to work - don't.
I am not big on "features". For me the main features is
open()/read()/write()/close(),
mkdir()/rmdir()/readdir() and those seem to work on all filesystems. Next
features are:
a) non-scary raid rebuild after a crash or disk failure,
b) "online" fsck
You can't grow a ZFS 'raid'. You're stuck with the number of disks you first
start with.
(I know this is 2 quotes back) That's kind of unfair since here you're
talking about a feature that was never offered, it's just an incorrect
assumption. It took me a while to understand what ZFS does and does not
offer as well - I missed many things from (ancient history) Digital's
advfs - but ZFS does lots of things quite well. There really is no such
thing as "a ZFS raid", that's probably most analogous to a zfs pool made
of a single raidz vdev, but that's a very simple case. What other system
lets you make large reliable storage pools from hundreds of drives on a
single server? I built some with 200+ 4TB drives some years back.
We only have a few hardware configurations, all with fixed number of disks, so
not a problem:
a) single 120GB ssd for OS (/home on NFS)
b) single SSD for OS, dual 4-6-8 TB HDD for data, RAID1 configuration to
protect against single disk failure
c) dual SSD for OS and /home, dual HDD for data, both RAID1 configuration to
protect against single disk failure
d) single SSD for OS, multiple (usually 8) 6-8 TB HDDs for data, mdadm
raid6+xfs and now raidz2 ZFS (protection against single disk failure + failure
of second disk during raid rebuild).
For case (b) for your data storage, you can expand a ZFS mirror
reasonably easily.
For case (c) I don't know how hard it is to use ZFS for the OS drive on
linux; I only used it on BSD. But mdadm on linux is ok for that role.
For case (d) it is true that you cannot expand a ZFS RAIDZ(2) vdev, but
that's ok if you know that going in.
BTRFS is billed as "open source replacement for ZFS", but after testing it,
my impression is that it is only used by a couple of enthusiasts
in single-disk laptop configurations. In a single-disk system, it is not
clear how btrfs/zfs is better than old-plain ext4/xfs.
I've never seen any good success stories for btrfs but to be fair I have
not followed it closely.
zfs can still give you some useful things on a single drive system: you
get the data checksums, useful snapshots (as opposed to LVM), volume
manager features, etc.
By default the checksums would only warn you of problems with a single
drive, but you can tell zfs to keep multiple copies of your data ("zfs
set copies=n") so that might well also let it recover from bad blocks.
G.
--
Graham Allan
Minnesota Supercomputing Institute - g...@umn.edu
