Re: quotas and reservations (was Re: Looking at RAID)
On Sun, Mar 11, 2018 at 07:56:37PM +1100, Russell Coker wrote: > On Sunday, 11 March 2018 6:37:56 PM AEDT Craig Sanders via luv-main wrote: > > On Fri, Mar 09, 2018 at 07:49:48PM +1100, Russell Coker wrote: > > > Unlike BTRFS, you can expect every feature of ZFS to just work. It may be > > > a total PITA to get it working, it may not be something you even want to > > > > ZFS just works, and I've never found it to be a PITA at all. I've been > > using it since around 2010. > > BTRFS snapshots just appear ready to use. ZFS snapshots have to have extra > stuff done to expose them. Snapshots of ZFS block devices have similar levels > of pain. What, you mean `zfs set snapdir=visible pool/dataset` or `zfs set snapdev=visible pool/zvol` is difficult? (like most other attributes, these are inherited by default by child datasets, and can be enabled/disabled individually) The only thing difficult about visible snapshots in ZFS is realising that tools like rsync will recurse into the .zfs/ snapshot directory unless you tell them not to. e.g. for rsync you can use a .rsync-filter file like this: $ cat /.rsync-filter -r .zfs/*** -s .zfs/*** - /tmp/*** See http://blog.taz.net.au/2012/04/01/rsync-and-zfs-snapshot-directories/ > > * non-rootfs storage just works - no hassle, no fuss, no drama. Couldn't > > be easier...certainly a lot less work/hassle than LVM and/or mdam + ext4 > > or xfs. > > If the installer supported it then that would be the case. As an aside does > the Ubuntu installer support ZFS? no idea. the last time I used an ubuntu installer was long before they started supporting ZFS. > But given that you can just install straight to mdadm+LVM+Ext4 or BTRFS, in > Debian at least ZFS is much harder to use. We obviously have different definitions of 'difficult to use'. I'd agree with "a bit more difficult to set up". but definitely not with "much harder to use". There are enough howtos and blog posts and answers on sites like stackexchange detailing how to get your rootfs on ZFS that it doesn't even qualify as "much harder to setup", only "a bit harder". When it comes to actually using either ZFS or LVM to administer your systems and organise your filesystems/datasets & partitions/zvols, ZFS wins for ease of use - it's not even a contest. LVM was good in it's day, but that day was over long ago. > > * getting linux to boot from ZFS requires some work (unless your distro > > supports it natively, like Ubuntu). A few hours of reading and about the > > same of careful, methodical work the first time you do it. > > Or you decide that something that requires a few hours of reading might be > too fragile for your root filesystem. "requires research and work to set up" does not equal "fragile". Personally, I think anyone who doesn't take the time to research how to best use the tools at their disposal, and which of the many tools to use, is negligent. ZFS doesn't require research because it's extremely difficult to use or understand. It requires research because it can do a lot, and there are many different ways to configure and use it. To make good choices you need some understanding of the concepts and you need some familiarity with the tools. Any complex/flexible tool will require a similar level of research effort. > >IMO it's worth the effort just for snapshots & zfs send backups of the > >rootfs. All the other ZFS features are just gravy. > > You can have a cron job that rsyncs the root on Ext4 to ZFS and then uses > ZFS snapshots etc for managing backups. Yes, i still do that on my systems that don't have zfs for the root fs. rsync's a great tool, but it doesn't come close to `zfs snapshot ...` followed by `zfs send`. `zfs send` completes in a tiny fraction of the time that an rsync takes, with much lower disk I/O. Incremental daily backups that used to take tens of minutes or even hours to complete - even with hardly any changes - now complete in a just few minutes. I barely even notice when the backup jobs start at ~1am - they used to be a sign that I should go to bed because my system will be running at a crawl, thrashing the disks for the next few hours. This is entirely due to rsync having to examine at least the metadata for each file in the directory tree being backed up (and possibly compare the block checksums on both sides of the rsync). With `zfs send`, ZFS knows **exactly** which blocks have changed in that snapshot, no lookup or scan required - just send the changed blocks. Even the startup time of rsync, while it's doing its initial scan of the backup path is several times longer than an entire zfs send backup. As I said, this alone makes it worth using ZFS for the rootfs. Like LVM, rsync was a great tool for its day. Unlike LVM, it's still worth using for some things. IMO the only reason to use LVM these days is because you have legacy systems that can't be taken down long enough to convert to ZFS (but you should be planning to switch them to ZFS
Re: quotas and reservations (was Re: Looking at RAID)
On Sunday, 11 March 2018 6:37:56 PM AEDT Craig Sanders via luv-main wrote: > On Fri, Mar 09, 2018 at 07:49:48PM +1100, Russell Coker wrote: > > Unlike BTRFS, you can expect every feature of ZFS to just work. It may be > > a total PITA to get it working, it may not be something you even want to > > ZFS just works, and I've never found it to be a PITA at all. I've been > using it since around 2010. BTRFS snapshots just appear ready to use. ZFS snapshots have to have extra stuff done to expose them. Snapshots of ZFS block devices have similar levels of pain. > * `apt-get install zfs-dkms zfsutils-linux` (and optionally zfs-zed and >zfsnap) is no big deal. It's no more trouble than installing anything >else. Well the extra compilation step when updating the kernel and the requirement to have GCC installed on systems that otherwise don't need it is annoying. But an Ubuntu kernel solves that. > * non-rootfs storage just works - no hassle, no fuss, no drama. Couldn't be > easier...certainly a lot less work/hassle than LVM and/or mdam + ext4 or > xfs. If the installer supported it then that would be the case. As an aside does the Ubuntu installer support ZFS? But given that you can just install straight to mdadm+LVM+Ext4 or BTRFS, in Debian at least ZFS is much harder to use. > * getting linux to boot from ZFS requires some work (unless your distro >supports it natively, like Ubuntu). A few hours of reading and about the > same of careful, methodical work the first time you do it. Or you decide that something that requires a few hours of reading might be too fragile for your root filesystem. >IMO it's worth the effort just for snapshots & zfs send backups of the >rootfs. All the other ZFS features are just gravy. You can have a cron job that rsyncs the root on Ext4 to ZFS and then uses ZFS snapshots etc for managing backups. > I don't really care about samba, but one of these days I should look into > doing NFS exports just by setting an attribute on a dataset in zfs. For > now, editing /etc/exports is easy enough. Good luck with that, it didn't work well for me. I probably could have got it working without too much effort, but the alternative of just editing /etc/exports and running "exportfs -r" is very easy so I gave up. -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
Re: quotas and reservations (was Re: Looking at RAID)
On Fri, Mar 09, 2018 at 07:49:48PM +1100, Russell Coker wrote: > Unlike BTRFS, you can expect every feature of ZFS to just work. It may be > a total PITA to get it working, it may not be something you even want to ZFS just works, and I've never found it to be a PITA at all. I've been using it since around 2010. * `apt-get install zfs-dkms zfsutils-linux` (and optionally zfs-zed and zfsnap) is no big deal. It's no more trouble than installing anything else. * non-rootfs storage just works - no hassle, no fuss, no drama. Couldn't be easier...certainly a lot less work/hassle than LVM and/or mdam + ext4 or xfs. * getting linux to boot from ZFS requires some work (unless your distro supports it natively, like Ubuntu). A few hours of reading and about the same of careful, methodical work the first time you do it. IMO it's worth the effort just for snapshots & zfs send backups of the rootfs. All the other ZFS features are just gravy. > use (EG ZFS control of Samba configuration), but it will work if you do the > right thing. I've never used that. Or the auto-NFS stuff. I Never saw any need to try it since I was already running NFS and Samba - continuing to use what I already had was no trouble at all :) I don't really care about samba, but one of these days I should look into doing NFS exports just by setting an attribute on a dataset in zfs. For now, editing /etc/exports is easy enough. craig -- craig sanders ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
Re: quotas and reservations (was Re: Looking at RAID)
On Friday, 9 March 2018 1:04:13 PM AEDT Craig Sanders via luv-main wrote: > I forgot to mention one very useful difference between 1. partitions & LVM > Logical Volumes (LV) and 2. btrfs sub-volumes & ZFS datasets. > > Partitions & LVs are created with a fixed size. The size can be changed > later, but if you create a 50G /data partittion and a 100G /home partition > it's easily possible to have one or the other partition being full while the > other has plenty of space. > > Changing the size of a partition/LV requires changing both the size of the > partition or LV AND changing the size of the filesystem that's on > it. This isn't too difficult but does require some caution and attention to > detail. It's recommended to make sure you have a recent backup in case you > make a mistake. Some filesystems like XFS don't support reducing the size at all. So LVM in that case is good for extending but can never reduce the size. Older filesystems like XFS and ext[234] have a fixed table of Inodes which will be increased linearly with increases in overall size which may or may not be what you want. > btrfs has btrfs-quota and btrfs-qgroup commands to control subvolume size > limits. I've never used them, so can't say any more than that they exist. > > BTW, The btrfs-quota man page says "When the quotas are turned on, they > affect all extent processing, taking a performance hit. It is not > recommended to turn on qgroups unless the user intends to actually use > them." They probably don't work well due to lack of testing. Last time I played with them I found a bug that made them unusable on my system in 5 minutes. > With ZFS, you can set a maximum size (quota) for a dataset (e.g. you might > limit Δ‘ata/videos to using no more than 200G). You can also reserve space in > the pool for a dataset to prevent any other dataset from using the reserved > space - i.e. to guarantee that a dataset will have a minimum amount of > space available to it, no matter what else is stored on the pool. Unlike BTRFS, you can expect every feature of ZFS to just work. It may be a total PITA to get it working, it may not be something you even want to use (EG ZFS control of Samba configuration), but it will work if you do the right thing. -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main