Re: ZFS vs btfrs
I've been happy running a home NAS since 2001ish on Solaris, Opensolaris, Linux. The best thing I did was switch from mdadm/LVM to ZFS so I could change "partitions" on the fly. Auto snapshotting every hour/day/week/month was a nice addition I missed from Netapp. The ECC & self correction of ZFS is very important to me. ZFS has survived power hits, losing a core on a dual core CPU (!) and bad ZoL upgrades (early CentOS versions). I used to make the OS use its own RAID1 (not ZFS), but I don't need the uptime vs power. I can easily reinstall the OS again. I had RAIDZ and upgraded my disks from gigabytes up to terabytes a few times. I now use 2 disk RAID1 blocks of 4TB or 6TB drives. When I want to upgrade, I only need to buy 2 drives at a time. 4TB has been a sweet spot for *me*. $/GB, availability of non-SMR drives and needing only 1 parity to keep 4TB safe for the data. Initially, I put drives into the system. I found SATA cages that put 4 drives into where the floppies would go are better. When I ran out of space, you can run regular SATA cables outside the box to a drive. No special eSATA needed. I used an old PC chassis w/ its own power supply to power the drives. I've since found cages that have fans and use them w/ an external power supply. There are SATA cards w/ the single connector to 4 sata ports that cut down clutter. I share filesystems as NFS, SMB and a web server. My chromebook or android can use those or SFTP mount to get to things. I have KVM to run a music server, jellyfin, search engine and ssh gateway. I can move those VMs to a different system and NFS/SMB mount the NAS. jellyfin replaces plex & does DLNA/uPNP. I'm planning on paperless-ng and a photo organizer in VMs on another system. I have a 3 GHz sandy lake quad cpu with 24 GB RAM which was an upgrade from an athlon dual core w/ 8GB and a bad core :-) that worked well for years with only the SSH gateway VM. I have a UPS that will auto shutdown after a 5 minute power loss. Because that's what a UPS is for: to ensure a clean shutdown if a generator or the power grid isn't supplying power. btrfs looks good (only with RAID1 IMO) but I'll stick with ZFS. When I installed Fedora on my 10yr old i5 with 8GB RAM, it chose btrfs. It was much slower than the previous Fedora with ext4 so I reinstalled it with ext4. I don't see much point in using btrfs/zfs on a single drive. On Wed, Feb 23, 2022 at 11:26 AM Ben Scott wrote: > Hi all, > > We haven't had a really good flamewar ^W discussion on here in far too > long... > > SUMMARY > > Btfrs vs ZFS. I was wondering if others would like to share their > opinions on either or both? Or something else entirely? (Maybe you > just don't feel alive if you're not compiling your kernel from > patches?) Especially cool would be recent comparisons of two or more. > > I'll provide an info dump of my plans below, but I do so mainly as > discussion-fodder. Don't feel obligated to address my scenario in > particular. Of course, commentary on anything in particular that > seems like a good/bad/cool idea is still welcome. > > RECEIVED WISDOM > > This is the stuff every article says. I rarely find anything that goes > deeper. > > - ZFS has been around/stable/whatever longer > - btfrs has been on Linux longer > - btfrs is GPL, ZFS is CDDL or whatever > - Licensing kept ZFS off Linux for a while > - ZFS is available on major Linux distros now > - People say one is faster, but disagree on which one > - Oracle is a bag of dicks > - ZFS is easier to pronounce > > For both, by coupling the filesystem layer and the block layer, we get > a lot of advantages, especially for things like snapshots and > deduplication. The newcomers also get you things like checksums for > every block, fault-tolerance over heterogenous physical devices, more > encryption and compression options. Faster, bigger, longer, lower, > wider, etc., etc. More superlatives than any other filesystem. > > MY SCENARIO > > I'm going to be building a new home server soon. Historically I've > used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are > using smarter filesystems these days. I should really get with the > times. They do seem to confer a lot of advantages, at least on paper. > > USE CASES > > User community is me and my girlfriend and a motley collection of > computing devices from multiple millenia. Administrator community is > me. > > Mostly plain old network file storage. Mixed use within that. I'm a > data hoarder. > > All sorts of stuff I've downloaded over the years, some not even from > the Internet (ZMODEM baby!). So large numbers of large write-once > files. "Large" has changed over the years, from something that fills > a floppy diskette to something that fills a DVD, but they don't change > once written. ISO images, tarballs, music and photo collections > (FLAC, MP3, JPEG). > > Also large numbers of small write-once files. I've got 20 GB of mail > archives in maildir format,
Re: ZFS vs btfrs
On 2022-02-24 12:42, Ian Kelling wrote: >> So what I do: >> >> * Create a copy on the destination host. >> * Snapshot it. >> * Mount the snapshot as my rsync backup destination. >> * And make a snapshot of _that_. > I'm confused by those bullets, I understand the general idea though. Sorry. It's been a Hell of a week. What I _should_ have written: * Create a copy, via rsync, on the destination host. This is my "origin" * Create a CoW snapshot of the origin -- giving snapshots datestamps * rsync to the new snapshot * create a (datestamped) snapshot of the newly-rsync'd-to snapshot * rsync to *that* * Rinse and repeat daily And, finally, delete -- or, rather, "btrfs sub del" -- whatever snapshots are outside of my retention period. -K ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
Ken D'Ambrosio writes: > On 2022-02-24 11:31, Ian Kelling wrote: >> Chuck McAndrew writes: >> >>> I would add one feature about ZFS that is super useful and that is >>> the ability to replicate datasets to a remote server. I don't know if >>> btrfs has a >>> similar feature, but the ability to have a backup server offsite and >>> just setup a cron job to replicate it was awesome for DR. It makes >>> backing up >>> your snapshots very very easy. >>> >> Yes, btrfs has this. I use it mostly through this tool: >> https://github.com/digint/btrbk . I recommend it. > > I use the btrfs-send (which, of course, is modeled after > zfs-send)... except, I kinda don't. And this isn't a dig at btrfs (or > ZFS), but just paranoia: I'm afraid that, if there were corruption on > the source FS, using a FS-specific/replicating tool to do the data > transfer might bring over whatever corruption was on the source in the > first place. > So what I do: > > * Create a copy on the destination host. > * Snapshot it. > * Mount the snapshot as my rsync backup destination. > * And make a snapshot of _that_. > > That way, I have two essentially identical CoW hierarchies, but that > have "left" the source FS, and gone to the destination one. Not as > efficient as sending CoW deltas, but it gives me a little more peace of > mind. Yes, my scenario seems awfully unlikely, buut... I'm confused by those bullets, I understand the general idea though. I think that it is a valid concern. I don't worry about btrfs send deleting data, so much as a filesystem having some metadata error propagated that you need a btrfs expert to help you correct before you can mount it read-write. The same concern applies to ZFS. It reminds me of the 2 media types recommendation of 3-2-1 backup strategy https://www.backblaze.com/blog/the-3-2-1-backup-strategy/ An rsync-like solution is simpler and more flexible in a bunch of ways, and even more performant in some ways. For FSF servers, we use a restic wrapper we wrote, https://github.com/leaf-node/kaya. btrfs-send and wrappers like btrbk are not my first recommendation for backups unless someone really wants or needs their specific benefits. -- Ian Kelling | Senior Systems Administrator, Free Software Foundation GPG Key: B125 F60B 7B28 7FF6 A2B7 DF8F 170A F0E2 9542 95DF https://fsf.org | https://gnu.org ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
On Thu, Feb 24, 2022, 11:55 Ken D'Ambrosio wrote: I use the btrfs-send (which, of course, is modeled after zfs-send)... except, I kinda don't. And this isn't a dig at btrfs (or ZFS), but just paranoia... On 2022-02-24 13:24, Bill Ricker wrote: SAN dutifully copied the block level writes to alternate site, so that panicked also. Oopsie. They had to restore Prod last backup onto UAT system (and recreate all logged transactions... a day of market!) to return to service. It was a bad week. I much prefer semantic (vs block/bit) replication. "Just because you're paranoid doesn't mean they're not out to get you." Dam. OK! I feel better about my belt-and-suspenders measures, now. ;-) -Ken___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
On Thu, Feb 24, 2022, 11:55 Ken D'Ambrosio wrote: > I use the btrfs-send (which, of course, is modeled after zfs-send)... > except, I kinda don't. And this isn't a dig at btrfs (or ZFS), but just > paranoia: I'm afraid that, if there were corruption on the source FS, > using a FS-specific/replicating tool to do the data transfer might bring > over whatever corruption was on the source in the first place. Not a merely theoretical concern. I saw this happen. Our British cousins fielded the same application we did, but since their geographically dispersed data centers were within the radius supported for syncronous SAN replication , they opted for that from primary cluster to Disaster cluster. We were replicating much further so used semantic replication stateside $BigCorp (log forwarding for DB, file level for normal volumes), even though it had a chance of losing the last transaction in flight. The DB driver made an error and wrote garbage 🗑, which corrupted DB indices, DB panicked. SAN dutifully copied the block level writes to alternate site, so that panicked also. Oopsie. They had to restore Prod last backup onto UAT system (and recreate all logged transactions... a day of market!) to return to service. It was a bad week. I much prefer semantic (vs block/bit) replication. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
On 2022-02-24 11:31, Ian Kelling wrote: > Chuck McAndrew writes: > >> I would add one feature about ZFS that is super useful and that is the >> ability to replicate datasets to a remote server. I don't know if >> btrfs has a >> similar feature, but the ability to have a backup server offsite and >> just setup a cron job to replicate it was awesome for DR. It makes >> backing up >> your snapshots very very easy. >> > > Yes, btrfs has this. I use it mostly through this tool: > https://github.com/digint/btrbk . I recommend it. I use the btrfs-send (which, of course, is modeled after zfs-send)... except, I kinda don't. And this isn't a dig at btrfs (or ZFS), but just paranoia: I'm afraid that, if there were corruption on the source FS, using a FS-specific/replicating tool to do the data transfer might bring over whatever corruption was on the source in the first place. So what I do: * Create a copy on the destination host. * Snapshot it. * Mount the snapshot as my rsync backup destination. * And make a snapshot of _that_. That way, I have two essentially identical CoW hierarchies, but that have "left" the source FS, and gone to the destination one. Not as efficient as sending CoW deltas, but it gives me a little more peace of mind. Yes, my scenario seems awfully unlikely, buut... ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
Chuck McAndrew writes: > I would add one feature about ZFS that is super useful and that is the > ability to replicate datasets to a remote server. I don't know if btrfs has a > similar feature, but the ability to have a backup server offsite and just > setup a cron job to replicate it was awesome for DR. It makes backing up > your snapshots very very easy. > Yes, btrfs has this. I use it mostly through this tool: https://github.com/digint/btrbk . I recommend it. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
Ben Scott writes: > Hi all, > > We haven't had a really good flamewar ^W discussion on here in far too long... > > SUMMARY > > Btfrs vs ZFS. I was wondering if others would like to share their > opinions on either or both? Or something else entirely? (Maybe you > just don't feel alive if you're not compiling your kernel from > patches?) Especially cool would be recent comparisons of two or more. > > I'll provide an info dump of my plans below, but I do so mainly as > discussion-fodder. Don't feel obligated to address my scenario in > particular. Of course, commentary on anything in particular that > seems like a good/bad/cool idea is still welcome. > > RECEIVED WISDOM > > This is the stuff every article says. I rarely find anything that goes > deeper. > > - ZFS has been around/stable/whatever longer > - btfrs has been on Linux longer > - btfrs is GPL, ZFS is CDDL or whatever > - Licensing kept ZFS off Linux for a while > - ZFS is available on major Linux distros now Those points aren't quite right. Nothing has changed regarding licensing. https://www.fsf.org/licensing/zfs-and-linux https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/ . Tldr: cddl is gpl incompatible. The only thing that changed is that Ubuntu decided to ship the zfs kernel module, they are ignoring the license, and there hasn't been any public license enforcement yet, but it could come from ZFS or linux copyright holders. Other than ubuntu, zfs for linux comes without cooperation from the major distros or upstream linux, and thus, if you are using your distro's kernel, it will sometimes break when you upgrade. ZFS doesn't benefit from the linux's code review, continuous integration, collaboration, etc. So, a good reason to use BTRFS over ZFS on linux is simply to support copyleft licensing in general and support upstream linux. For all your technical goals, I think BTRFS will do fine, I've been using it for many years now. -- Ian Kelling | Senior Systems Administrator, Free Software Foundation GPG Key: B125 F60B 7B28 7FF6 A2B7 DF8F 170A F0E2 9542 95DF https://fsf.org | https://gnu.org ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
(glances over at stack of full external USB drives...) (shifts uncomfortably) ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
On 2022-02-23 11:25, Ben Scott wrote: > Hi all, > > Btfrs vs ZFS. I was wondering if others would like to share their > opinions on either or both? So... really, the two filesystems have a lot in in common. ZFS is absolutely more mature, especially WRT RAID (more below). But btrfs has some really nifty features, and with its arrival in Fedora, is getting the support it so badly needed. So, for me, the big win for btrfs is: alias clone='cp --reflink=always' Hey, presto! You just cloned your base 5 GB virtual image in under a second. You now have tow CoW "copies" of the exact same file, and unlike hard links, you are now free to munge them to your heart's content. NOW: the last time I checked for this on ZFS was sometime around the Sauron's revealing himself as a Dark Power, so maybe ZFS supports it now. And I totally know that ZFS supports lightweight snapshots (as does btrfs), but being able to clone a file -- or an entire hierarchy, such as all of my company's repos -- just so I can have a "play" hierarchy, and a not-play one is handy. Likewise, when editing video files, you can have the original and the tweaked one, with only the delta as additionally used storage. tl;dr: it's handy, especially for lots and lots of files in a hierarchy, or really big, related files. The bad: DO NOT DO RAID =~ /[56]/ ON btrfs. What I do is a ye-olde mdm RAID, and lay btrfs on top of that. Works the bomb. Doesn't give all the bells and whistles that a RAID-savvy FS would (e.g., only rebuilding places with data, instead of the entire volume), but I've had no problems. Last thing: Timeshift is really cool. I wrote my own scripts, but I admit, Timeshift gets 'er done. I assume, but do not know, that there's a similar utility that can make use of ZFS. -Ken ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
I'm nowhere near as familiar with FreeBSD as I am with Debian so there's a bit more comfort to at least debug or know where to look when things go wrong. That being said, if all you're looking for is a box to host SMB/NFS/iSCSI then either will work fine. Now on to concrete things that I see SCALE doing better: I can easily get a letsencrypt SSL cert set up bhyve is hot garbage. guests will randomly die, lose network connection, can see the rest of the network but not resources on the host system, and if you use UEFI and Linux you're in for a world of pain if you ever upgrade (upgrade what? yes). SCALE uses qemu and so far seems much more stable *knocks on head* ditched jails for docker containers, much improved 'app store' I haven't used it yet, but apparently you can use glusterfs to share storage across SCALE instances To be fair, these are things that you wouldn't normally use a professional-grade file server for. It would serve files and that's it. My use case is my basement where I have a few TB of movie and audio files, a need for a few VMs to round out other services, want to run a few containers for small tasks, make sure my mom's computer 300 miles away backs up nightly, and don't want to have 15 boxes in my basement to do it all. -Mark On 2/23/22 15:11, Jason T. Nelson wrote: > In a previous email, Mark Komarinski (mkomarin...@wayga.org) said: >> For everyone else, TrueNAS SCALE was released yesterday. Debian+ZFS >> makes this a lot more useful than when it was FreeBSD based. > I know this is a LUG list, but out of (perhaps morbid) curiosity, why "a lot > more useful"? > ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
In a previous email, Mark Komarinski (mkomarin...@wayga.org) said: > For everyone else, TrueNAS SCALE was released yesterday. Debian+ZFS > makes this a lot more useful than when it was FreeBSD based. I know this is a LUG list, but out of (perhaps morbid) curiosity, why "a lot more useful"? -- Jason T. Nelson GPG key 0xFF676C9E ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
With LVM (and it looks like btrfs) you can pool mirrored drives together into what is effectively a RAID10 and you can remove individual mirrors to shrink or grow the pool. ZFS does not allow you to do that. Once you expand a pool there's no going back. You can replace individual drives in a mirror with like or larger, but you can't go smaller and the only way for that pool to be bigger is to replace all the drives one at a time. This is less of an issue now that I just build out a RAIDZ2 with 5 6xTB drives and make that the entire pool. Given your use case, I'd say go with btrfs. For everyone else, TrueNAS SCALE was released yesterday. Debian+ZFS makes this a lot more useful than when it was FreeBSD based. -Mark On 2/23/2022 11:25 AM, Ben Scott wrote: > Hi all, > > We haven't had a really good flamewar ^W discussion on here in far too long... > > SUMMARY > > Btfrs vs ZFS. I was wondering if others would like to share their > opinions on either or both? Or something else entirely? (Maybe you > just don't feel alive if you're not compiling your kernel from > patches?) Especially cool would be recent comparisons of two or more. > > I'll provide an info dump of my plans below, but I do so mainly as > discussion-fodder. Don't feel obligated to address my scenario in > particular. Of course, commentary on anything in particular that > seems like a good/bad/cool idea is still welcome. > > RECEIVED WISDOM > > This is the stuff every article says. I rarely find anything that goes > deeper. > > - ZFS has been around/stable/whatever longer > - btfrs has been on Linux longer > - btfrs is GPL, ZFS is CDDL or whatever > - Licensing kept ZFS off Linux for a while > - ZFS is available on major Linux distros now > - People say one is faster, but disagree on which one > - Oracle is a bag of dicks > - ZFS is easier to pronounce > > For both, by coupling the filesystem layer and the block layer, we get > a lot of advantages, especially for things like snapshots and > deduplication. The newcomers also get you things like checksums for > every block, fault-tolerance over heterogenous physical devices, more > encryption and compression options. Faster, bigger, longer, lower, > wider, etc., etc. More superlatives than any other filesystem. > > MY SCENARIO > > I'm going to be building a new home server soon. Historically I've > used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are > using smarter filesystems these days. I should really get with the > times. They do seem to confer a lot of advantages, at least on paper. > > USE CASES > > User community is me and my girlfriend and a motley collection of > computing devices from multiple millenia. Administrator community is > me. > > Mostly plain old network file storage. Mixed use within that. I'm a > data hoarder. > > All sorts of stuff I've downloaded over the years, some not even from > the Internet (ZMODEM baby!). So large numbers of large write-once > files. "Large" has changed over the years, from something that fills > a floppy diskette to something that fills a DVD, but they don't change > once written. ISO images, tarballs, music and photo collections > (FLAC, MP3, JPEG). > > Also large numbers of small write-once files. I've got 20 GB of mail > archives in maildir format, one file per message, less than 4K per > file for the old stuff (modern HTML mail is rather bloated). These > generally don't change once written either, but there are lots of > them. Some single directories have over 200 K files. > > Backups of my user systems. Currently accomplished via rsnapshot and > rsync (or ROBOCOPY for 'doze). So small to medium-small files, but > changing and updating and hardlinking and moving a lot. With a > smarter filesystem I can likely dispense with rsnapshot, but I doubt > I'm going to move away from plain-old-files-as-backup-storage any time > soon. (rsync might conceivably be replaced with a smarter network > filesystem someday, but likely not soon.) > > ANTI USE CASES > > Not a lot of mass-market videos -- the boob tube is one area where I > let others do it for me. (Roku, Netflix, Blu-ray, etc.) > > No plans to network mount home directories for my daily-driver PCs. > For laptops especially that's problematic (and sorting apps > (particularly browsers) that can copy with a distributed filesystem > seems unlikely to pay off). > > Not planning on any serious hosting of VMs or containers or complex > application software on this box. I can't rule it out entirely for > (especially as an experiment), but this is mainly intended to be a > NAS-type server. It will run NFS, Samba, SSH, rsync. It might run > some mail daemons (SMTP, IMAP) just to make accessing archives easier, > but it won't be the public-facing MX for anything. > > It's unlikely to run any point-and-drool administration (web) GUIs. I > have a set of config files I've been carrying around with me since I > kept them on floppy diskette, and they've
Re: ZFS vs btfrs
On Wed, Feb 23, 2022 at 11:44 AM Bruce Dawson wrote: > Well, you're more concerned with files than large blocks of data, so I > don't think either matter - other than standard filesystem performance. I wouldn't go that far. In particular, snapshots at the block layer are generally less efficient compared to filesystem integrated snapshots. With all those static files, built-in checksums (vs external MD5/SHA1) is rather nice. EXT4 does handle large directories better than its predecessors, but others are still better, or so I'm told. Tuning for file sizes and inodes gets old, too. Mutt disk collections (i.e., different ages and specs) is suboptimal for traditional RAID; allegedly the new guys can handle that better. And so on. Granted, I've never actually tried any of these things (or I wouldn't be asking), but the brochure makes them sound really cool. > ZFS is nice, but resource intensive. Care to expound? Even if your use case isn't easily analogous to mine, it'd be interesting reading for me and others, I expect. > How about choosing btrfs so we (the community) can learn more about it?! Well, whichever road I end up going down, I'll be happy to share notes on the route once I get there. :-) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: ZFS vs btfrs
Well, you're more concerned with files than large blocks of data, so I don't think either matter - other than standard filesystem performance. I've had some experience with ZFS, and practically none with btrfs. ZFS is nice, but resource intensive. I suspect btrfs is similar, but probably not in standard Linux development uses. How about choosing btrfs so we (the community) can learn more about it?! --Bruce On 2/23/22 11:25, Ben Scott wrote: > Hi all, > > We haven't had a really good flamewar ^W discussion on here in far too long... > > SUMMARY > > Btfrs vs ZFS. I was wondering if others would like to share their > opinions on either or both? Or something else entirely? (Maybe you > just don't feel alive if you're not compiling your kernel from > patches?) Especially cool would be recent comparisons of two or more. > > I'll provide an info dump of my plans below, but I do so mainly as > discussion-fodder. Don't feel obligated to address my scenario in > particular. Of course, commentary on anything in particular that > seems like a good/bad/cool idea is still welcome. > > RECEIVED WISDOM > > This is the stuff every article says. I rarely find anything that goes > deeper. > > - ZFS has been around/stable/whatever longer > - btfrs has been on Linux longer > - btfrs is GPL, ZFS is CDDL or whatever > - Licensing kept ZFS off Linux for a while > - ZFS is available on major Linux distros now > - People say one is faster, but disagree on which one > - Oracle is a bag of dicks > - ZFS is easier to pronounce > > For both, by coupling the filesystem layer and the block layer, we get > a lot of advantages, especially for things like snapshots and > deduplication. The newcomers also get you things like checksums for > every block, fault-tolerance over heterogenous physical devices, more > encryption and compression options. Faster, bigger, longer, lower, > wider, etc., etc. More superlatives than any other filesystem. > > MY SCENARIO > > I'm going to be building a new home server soon. Historically I've > used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are > using smarter filesystems these days. I should really get with the > times. They do seem to confer a lot of advantages, at least on paper. > > USE CASES > > User community is me and my girlfriend and a motley collection of > computing devices from multiple millenia. Administrator community is > me. > > Mostly plain old network file storage. Mixed use within that. I'm a > data hoarder. > > All sorts of stuff I've downloaded over the years, some not even from > the Internet (ZMODEM baby!). So large numbers of large write-once > files. "Large" has changed over the years, from something that fills > a floppy diskette to something that fills a DVD, but they don't change > once written. ISO images, tarballs, music and photo collections > (FLAC, MP3, JPEG). > > Also large numbers of small write-once files. I've got 20 GB of mail > archives in maildir format, one file per message, less than 4K per > file for the old stuff (modern HTML mail is rather bloated). These > generally don't change once written either, but there are lots of > them. Some single directories have over 200 K files. > > Backups of my user systems. Currently accomplished via rsnapshot and > rsync (or ROBOCOPY for 'doze). So small to medium-small files, but > changing and updating and hardlinking and moving a lot. With a > smarter filesystem I can likely dispense with rsnapshot, but I doubt > I'm going to move away from plain-old-files-as-backup-storage any time > soon. (rsync might conceivably be replaced with a smarter network > filesystem someday, but likely not soon.) > > ANTI USE CASES > > Not a lot of mass-market videos -- the boob tube is one area where I > let others do it for me. (Roku, Netflix, Blu-ray, etc.) > > No plans to network mount home directories for my daily-driver PCs. > For laptops especially that's problematic (and sorting apps > (particularly browsers) that can copy with a distributed filesystem > seems unlikely to pay off). > > Not planning on any serious hosting of VMs or containers or complex > application software on this box. I can't rule it out entirely for > (especially as an experiment), but this is mainly intended to be a > NAS-type server. It will run NFS, Samba, SSH, rsync. It might run > some mail daemons (SMTP, IMAP) just to make accessing archives easier, > but it won't be the public-facing MX for anything. > > It's unlikely to run any point-and-drool administration (web) GUIs. I > have a set of config files I've been carrying around with me since I > kept them on floppy diskette, and they've served me well. Those that > like them, more power to you, but they're not for me. I inevitably > bump into their limitations and have to go outside them anyway. > > I've tried a few consumer NAS appliances and have generally been > disappointed. It's the same as the GUI thing above, except I hit the > limits sooner and i
ZFS vs btfrs
Hi all, We haven't had a really good flamewar ^W discussion on here in far too long... SUMMARY Btfrs vs ZFS. I was wondering if others would like to share their opinions on either or both? Or something else entirely? (Maybe you just don't feel alive if you're not compiling your kernel from patches?) Especially cool would be recent comparisons of two or more. I'll provide an info dump of my plans below, but I do so mainly as discussion-fodder. Don't feel obligated to address my scenario in particular. Of course, commentary on anything in particular that seems like a good/bad/cool idea is still welcome. RECEIVED WISDOM This is the stuff every article says. I rarely find anything that goes deeper. - ZFS has been around/stable/whatever longer - btfrs has been on Linux longer - btfrs is GPL, ZFS is CDDL or whatever - Licensing kept ZFS off Linux for a while - ZFS is available on major Linux distros now - People say one is faster, but disagree on which one - Oracle is a bag of dicks - ZFS is easier to pronounce For both, by coupling the filesystem layer and the block layer, we get a lot of advantages, especially for things like snapshots and deduplication. The newcomers also get you things like checksums for every block, fault-tolerance over heterogenous physical devices, more encryption and compression options. Faster, bigger, longer, lower, wider, etc., etc. More superlatives than any other filesystem. MY SCENARIO I'm going to be building a new home server soon. Historically I've used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are using smarter filesystems these days. I should really get with the times. They do seem to confer a lot of advantages, at least on paper. USE CASES User community is me and my girlfriend and a motley collection of computing devices from multiple millenia. Administrator community is me. Mostly plain old network file storage. Mixed use within that. I'm a data hoarder. All sorts of stuff I've downloaded over the years, some not even from the Internet (ZMODEM baby!). So large numbers of large write-once files. "Large" has changed over the years, from something that fills a floppy diskette to something that fills a DVD, but they don't change once written. ISO images, tarballs, music and photo collections (FLAC, MP3, JPEG). Also large numbers of small write-once files. I've got 20 GB of mail archives in maildir format, one file per message, less than 4K per file for the old stuff (modern HTML mail is rather bloated). These generally don't change once written either, but there are lots of them. Some single directories have over 200 K files. Backups of my user systems. Currently accomplished via rsnapshot and rsync (or ROBOCOPY for 'doze). So small to medium-small files, but changing and updating and hardlinking and moving a lot. With a smarter filesystem I can likely dispense with rsnapshot, but I doubt I'm going to move away from plain-old-files-as-backup-storage any time soon. (rsync might conceivably be replaced with a smarter network filesystem someday, but likely not soon.) ANTI USE CASES Not a lot of mass-market videos -- the boob tube is one area where I let others do it for me. (Roku, Netflix, Blu-ray, etc.) No plans to network mount home directories for my daily-driver PCs. For laptops especially that's problematic (and sorting apps (particularly browsers) that can copy with a distributed filesystem seems unlikely to pay off). Not planning on any serious hosting of VMs or containers or complex application software on this box. I can't rule it out entirely for (especially as an experiment), but this is mainly intended to be a NAS-type server. It will run NFS, Samba, SSH, rsync. It might run some mail daemons (SMTP, IMAP) just to make accessing archives easier, but it won't be the public-facing MX for anything. It's unlikely to run any point-and-drool administration (web) GUIs. I have a set of config files I've been carrying around with me since I kept them on floppy diskette, and they've served me well. Those that like them, more power to you, but they're not for me. I inevitably bump into their limitations and have to go outside them anyway. I've tried a few consumer NAS appliances and have generally been disappointed. It's the same as the GUI thing above, except I hit the limits sooner and in more ways. Some of them have really disgusting software internals. (A shame, because some of the hardware is appealing, especially in terms of watts and price.) I don't want to put this on somebody else's computer. HARDWARE I'm shooting for a super compact PC chassis, mini-ITX mainboard, 4 x 3.5-inch hot swap bays, SATA interfaces, x86-64 processor. Initially it will be two spinning disks. Somewhere in the neighborhood of 3 to 6 TB effective. The disks will be relatively slow, favoring lower price-per-GB and less heat over performance. This is bulk data storage. The user PCs have SSDs. If fancy filesystems were
