Re: ZFS vs btfrs

2022-03-02 Thread Tom Buskey
I've been happy running a home NAS since 2001ish on Solaris, Opensolaris,
Linux.

The best thing I did was switch from mdadm/LVM to ZFS so I could change
"partitions" on the fly.  Auto snapshotting every hour/day/week/month was a
nice addition I missed from Netapp.  The ECC & self correction of ZFS is
very important to me.  ZFS has survived power hits, losing a core on a dual
core CPU (!) and bad ZoL upgrades (early CentOS versions).  I used to make
the OS use its own RAID1 (not ZFS), but I don't need the uptime vs power.
I can easily reinstall the OS again.

I had RAIDZ and upgraded my disks from gigabytes up to terabytes a few
times.  I now use 2 disk RAID1 blocks of 4TB or 6TB drives.  When I want to
upgrade, I only need to buy 2 drives at a time.  4TB has been a sweet
spot for *me*.  $/GB, availability of non-SMR drives and needing only 1
parity to keep 4TB safe for the data.

Initially, I put drives into the system.  I found SATA cages that put 4
drives into where the floppies would go are better.

When I ran out of space, you can run regular SATA cables outside the box to
a drive.  No special eSATA needed.  I used an old PC chassis w/ its own
power supply to power the drives.  I've since found cages that have fans
and use them w/ an external power supply.  There are SATA cards w/ the
single connector to 4 sata ports that cut down clutter.

I share filesystems as NFS, SMB and a web server.  My chromebook or android
can use those or SFTP mount to get to things.
I have KVM to run a music server, jellyfin, search engine and ssh gateway.
I can move those VMs to a different system and NFS/SMB mount the NAS.

jellyfin replaces plex & does DLNA/uPNP.  I'm planning on paperless-ng and
a photo organizer in VMs on another system.

I have a 3 GHz sandy lake quad cpu with 24 GB RAM which was an upgrade from
an athlon dual core w/ 8GB and a bad core :-) that worked well for years
with only the SSH gateway VM.  I have a UPS that will auto shutdown after a
5 minute power loss.  Because that's what a UPS is for: to ensure a clean
shutdown if a generator or the power grid isn't supplying power.

btrfs looks good (only with RAID1 IMO) but I'll stick with ZFS.  When I
installed Fedora on my 10yr old i5 with 8GB RAM, it chose btrfs.  It was
much slower than the previous Fedora with ext4 so I reinstalled it with
ext4.  I don't see much point in using btrfs/zfs on a single drive.

On Wed, Feb 23, 2022 at 11:26 AM Ben Scott  wrote:

> Hi all,
>
> We haven't had a really good flamewar ^W discussion on here in far too
> long...
>
> SUMMARY
>
> Btfrs vs ZFS. I was wondering if others would like to share their
> opinions on either or both?  Or something else entirely?  (Maybe you
> just don't feel alive if you're not compiling your kernel from
> patches?)  Especially cool would be recent comparisons of two or more.
>
> I'll provide an info dump of my plans below, but I do so mainly as
> discussion-fodder.  Don't feel obligated to address my scenario in
> particular.  Of course, commentary on anything in particular that
> seems like a good/bad/cool idea is still welcome.
>
> RECEIVED WISDOM
>
> This is the stuff every article says.  I rarely find anything that goes
> deeper.
>
> - ZFS has been around/stable/whatever longer
> - btfrs has been on Linux longer
> - btfrs is GPL, ZFS is CDDL or whatever
> - Licensing kept ZFS off Linux for a while
> - ZFS is available on major Linux distros now
> - People say one is faster, but disagree on which one
> - Oracle is a bag of dicks
> - ZFS is easier to pronounce
>
> For both, by coupling the filesystem layer and the block layer, we get
> a lot of advantages, especially for things like snapshots and
> deduplication.  The newcomers also get you things like checksums for
> every block, fault-tolerance over heterogenous physical devices, more
> encryption and compression options.  Faster, bigger, longer, lower,
> wider, etc., etc.  More superlatives than any other filesystem.
>
> MY SCENARIO
>
> I'm going to be building a new home server soon.  Historically I've
> used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are
> using smarter filesystems these days.  I should really get with the
> times.  They do seem to confer a lot of advantages, at least on paper.
>
> USE CASES
>
> User community is me and my girlfriend and a motley collection of
> computing devices from multiple millenia.  Administrator community is
> me.
>
> Mostly plain old network file storage.  Mixed use within that.  I'm a
> data hoarder.
>
> All sorts of stuff I've downloaded over the years, some not even from
> the Internet (ZMODEM baby!).  So large numbers of large write-once
> files.  "Large" has changed over the years, from something that fills
> a floppy diskette to something that fills a DVD, but they don't change
> once written.  ISO images, tarballs, music and photo collections
> (FLAC, MP3, JPEG).
>
> Also large numbers of small write-once files.  I've got 20 GB of mail
> archives in maildir format, 

Re: ZFS vs btfrs

2022-02-25 Thread Ken D'Ambrosio
On 2022-02-24 12:42, Ian Kelling wrote:
>> So what I do:
>> 
>> * Create a copy on the destination host.
>> * Snapshot it.
>> * Mount the snapshot as my rsync backup destination.
>> * And make a snapshot of _that_.
> I'm confused by those bullets, I understand the general idea though.

Sorry.  It's been a Hell of a week.  What I _should_ have written:
* Create a copy, via rsync, on the destination host.  This is my 
"origin"
* Create a CoW snapshot of the origin -- giving snapshots datestamps
* rsync to the new snapshot
* create a (datestamped) snapshot of the newly-rsync'd-to snapshot
* rsync to *that*
* Rinse and repeat daily

And, finally, delete -- or, rather, "btrfs sub del" -- whatever 
snapshots are outside of my retention period.

-K
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-24 Thread Ian Kelling


Ken D'Ambrosio  writes:

> On 2022-02-24 11:31, Ian Kelling wrote:
>> Chuck McAndrew  writes:
>> 
>>> I would add one feature about ZFS that is super useful and that is
>>> the ability to replicate datasets to a remote server. I don't know if
>>> btrfs has a
>>> similar feature, but the ability to have a backup server offsite and
>>> just setup a cron job to replicate it was awesome for DR. It makes
>>> backing up
>>> your snapshots very very easy.
>>> 
>> Yes, btrfs has this. I use it mostly through this tool:
>> https://github.com/digint/btrbk . I recommend it.
>
> I use the btrfs-send (which, of course, is modeled after
> zfs-send)... except, I kinda don't.  And this isn't a dig at btrfs (or
> ZFS), but just paranoia: I'm afraid that, if there were corruption on
> the source FS, using a FS-specific/replicating tool to do the data
> transfer might bring over whatever corruption was on the source in the
> first place.
> So what I do:
>
> * Create a copy on the destination host.
> * Snapshot it.
> * Mount the snapshot as my rsync backup destination.
> * And make a snapshot of _that_.
>
> That way, I have two essentially identical CoW hierarchies, but that
> have "left" the source FS, and gone to the destination one.  Not as
> efficient as sending CoW deltas, but it gives me a little more peace of
> mind.  Yes, my scenario seems awfully unlikely, buut...

I'm confused by those bullets, I understand the general idea though.

I think that it is a valid concern. I don't worry about btrfs send
deleting data, so much as a filesystem having some metadata error
propagated that you need a btrfs expert to help you correct before you
can mount it read-write. The same concern applies to ZFS. It reminds me
of the 2 media types recommendation of 3-2-1 backup strategy
https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

An rsync-like solution is simpler and more flexible in a bunch of ways,
and even more performant in some ways. For FSF servers, we use a restic
wrapper we wrote, https://github.com/leaf-node/kaya.  btrfs-send and
wrappers like btrbk are not my first recommendation for backups unless
someone really wants or needs their specific benefits.

-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-24 Thread Ken D'Ambrosio

On Thu, Feb 24, 2022, 11:55 Ken D'Ambrosio  wrote:


I use the btrfs-send (which, of course, is modeled after zfs-send)...
except, I kinda don't.  And this isn't a dig at btrfs (or ZFS), but 
just

paranoia...

On 2022-02-24 13:24, Bill Ricker wrote:
SAN dutifully copied the block level writes to alternate site, so that 
panicked also. Oopsie. They had to restore Prod last backup onto UAT 
system (and recreate all logged transactions... a day of market!) to 
return to service. It was a bad week.


I much prefer semantic (vs block/bit) replication.


"Just because you're paranoid doesn't mean they're not out to get you."  
Dam.  OK!  I feel better about my belt-and-suspenders measures, now. 
;-)


-Ken___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-24 Thread Bill Ricker
On Thu, Feb 24, 2022, 11:55 Ken D'Ambrosio  wrote:

> I use the btrfs-send (which, of course, is modeled after zfs-send)...
> except, I kinda don't.  And this isn't a dig at btrfs (or ZFS), but just
> paranoia: I'm afraid that, if there were corruption on the source FS,
> using a FS-specific/replicating tool to do the data transfer might bring
> over whatever corruption was on the source in the first place.


Not a merely theoretical concern. I saw this happen.

Our British cousins fielded the same application we did, but since their
geographically dispersed data centers were within the radius supported for
syncronous SAN replication , they opted for that from primary cluster to
Disaster cluster. We were replicating much further so used semantic
replication stateside $BigCorp (log forwarding for DB, file level for
normal volumes), even though it had a chance of losing the last transaction
in flight. The DB driver made an error and wrote garbage ,  which
corrupted DB indices, DB panicked. SAN dutifully copied the block level
writes to alternate site, so that panicked also. Oopsie. They had to
restore Prod last backup onto UAT system (and recreate all logged
transactions... a day of market!) to return to service. It was a bad week.

I much prefer semantic (vs block/bit) replication.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-24 Thread Ken D'Ambrosio
On 2022-02-24 11:31, Ian Kelling wrote:
> Chuck McAndrew  writes:
> 
>> I would add one feature about ZFS that is super useful and that is the 
>> ability to replicate datasets to a remote server. I don't know if 
>> btrfs has a
>> similar feature, but the ability to have a backup server offsite and 
>> just setup a cron job to replicate it was awesome for DR. It makes 
>> backing up
>> your snapshots very very easy.
>> 
> 
> Yes, btrfs has this. I use it mostly through this tool:
> https://github.com/digint/btrbk . I recommend it.

I use the btrfs-send (which, of course, is modeled after zfs-send)... 
except, I kinda don't.  And this isn't a dig at btrfs (or ZFS), but just 
paranoia: I'm afraid that, if there were corruption on the source FS, 
using a FS-specific/replicating tool to do the data transfer might bring 
over whatever corruption was on the source in the first place.  So what 
I do:

* Create a copy on the destination host.
* Snapshot it.
* Mount the snapshot as my rsync backup destination.
* And make a snapshot of _that_.

That way, I have two essentially identical CoW hierarchies, but that 
have "left" the source FS, and gone to the destination one.  Not as 
efficient as sending CoW deltas, but it gives me a little more peace of 
mind.  Yes, my scenario seems awfully unlikely, buut...
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-24 Thread Ian Kelling


Chuck McAndrew  writes:

> I would add one feature about ZFS that is super useful and that is the 
> ability to replicate datasets to a remote server. I don't know if btrfs has a
> similar feature, but the ability to have a backup server offsite and just 
> setup a cron job to replicate it was awesome for DR. It makes backing up
> your snapshots very very easy.
>

Yes, btrfs has this. I use it mostly through this tool:
https://github.com/digint/btrbk . I recommend it.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-23 Thread Ian Kelling


Ben Scott  writes:

> Hi all,
>
> We haven't had a really good flamewar ^W discussion on here in far too long...
>
> SUMMARY
>
> Btfrs vs ZFS. I was wondering if others would like to share their
> opinions on either or both?  Or something else entirely?  (Maybe you
> just don't feel alive if you're not compiling your kernel from
> patches?)  Especially cool would be recent comparisons of two or more.
>
> I'll provide an info dump of my plans below, but I do so mainly as
> discussion-fodder.  Don't feel obligated to address my scenario in
> particular.  Of course, commentary on anything in particular that
> seems like a good/bad/cool idea is still welcome.
>
> RECEIVED WISDOM
>
> This is the stuff every article says.  I rarely find anything that goes 
> deeper.
>
> - ZFS has been around/stable/whatever longer
> - btfrs has been on Linux longer


> - btfrs is GPL, ZFS is CDDL or whatever
> - Licensing kept ZFS off Linux for a while
> - ZFS is available on major Linux distros now

Those points aren't quite right. Nothing has changed regarding
licensing.  https://www.fsf.org/licensing/zfs-and-linux
https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/ . Tldr: cddl
is gpl incompatible. The only thing that changed is that Ubuntu decided
to ship the zfs kernel module, they are ignoring the license, and there
hasn't been any public license enforcement yet, but it could come from
ZFS or linux copyright holders. Other than ubuntu, zfs for linux comes
without cooperation from the major distros or upstream linux, and thus,
if you are using your distro's kernel, it will sometimes break when you
upgrade. ZFS doesn't benefit from the linux's code review, continuous
integration, collaboration, etc. So, a good reason to use BTRFS over ZFS
on linux is simply to support copyleft licensing in general and support
upstream linux. For all your technical goals, I think BTRFS will do
fine, I've been using it for many years now.

-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-23 Thread Curt Howland
(glances over at stack of full external USB drives...)
(shifts uncomfortably)
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-23 Thread Ken D'Ambrosio
On 2022-02-23 11:25, Ben Scott wrote:
> Hi all,
> 
> Btfrs vs ZFS. I was wondering if others would like to share their
> opinions on either or both?

So... really, the two filesystems have a lot in in common.  ZFS is 
absolutely more mature, especially WRT RAID (more below).  But btrfs has 
some really nifty features, and with its arrival in Fedora, is getting 
the support it so badly needed.  So, for me, the big win for btrfs is:
alias clone='cp --reflink=always'
Hey, presto!  You just cloned your base 5 GB virtual image in under a 
second.  You now have tow CoW "copies" of the exact same file, and 
unlike hard links, you are now free to munge them to your heart's 
content.  NOW: the last time I checked for this on ZFS was sometime 
around the Sauron's revealing himself as a Dark Power, so maybe ZFS 
supports it now.  And I totally know that ZFS supports lightweight 
snapshots (as does btrfs), but being able to clone a file -- or an 
entire hierarchy, such as all of my company's repos -- just so I can 
have a "play" hierarchy, and a not-play one is handy.  Likewise, when 
editing video files, you can have the original and the tweaked one, with 
only the delta as additionally used storage.  tl;dr: it's handy, 
especially for lots and lots of files in a hierarchy, or really big, 
related files.

The bad: DO NOT DO RAID =~ /[56]/ ON btrfs.  What I do is a ye-olde mdm 
RAID, and lay btrfs on top of that.  Works the bomb.  Doesn't give all 
the bells and whistles that a RAID-savvy FS would (e.g., only rebuilding 
places with data, instead of the entire volume), but I've had no 
problems.

Last thing: Timeshift is really cool.  I wrote my own scripts, but I 
admit, Timeshift gets 'er done.  I assume, but do not know, that there's 
a similar utility that can make use of ZFS.

-Ken
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-23 Thread Mark Komarinski
I'm nowhere near as familiar with FreeBSD as I am with Debian so there's 
a bit more comfort to at least debug or know where to look when things 
go wrong.  That being said, if all you're looking for is a box to host 
SMB/NFS/iSCSI then either will work fine.

Now on to concrete things that I see SCALE doing better:

I can easily get a letsencrypt SSL cert set up

bhyve is hot garbage.  guests will randomly die, lose network 
connection, can see the rest of the network but not resources on the 
host system, and if you use UEFI and Linux you're in for a world of pain 
if you ever upgrade (upgrade what? yes).  SCALE uses qemu and so far 
seems much more stable *knocks on head*

ditched jails for docker containers, much improved 'app store'

I haven't used it yet, but apparently you can use glusterfs to share 
storage across SCALE instances

To be fair, these are things that you wouldn't normally use a 
professional-grade file server for. It would serve files and that's it.  
My use case is my basement where I have a few TB of movie and audio 
files, a need for a few VMs to round out other services, want to run a 
few containers for small tasks, make sure my mom's computer 300 miles 
away backs up nightly, and don't want to have 15 boxes in my basement to 
do it all.

-Mark

On 2/23/22 15:11, Jason T. Nelson wrote:
> In a previous email, Mark Komarinski (mkomarin...@wayga.org) said:
>> For everyone else, TrueNAS SCALE was released yesterday. Debian+ZFS
>> makes this a lot more useful than when it was FreeBSD based.
> I know this is a LUG list, but out of (perhaps morbid) curiosity, why "a lot 
> more useful"?
>
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-23 Thread Jason T. Nelson
In a previous email, Mark Komarinski (mkomarin...@wayga.org) said:
> For everyone else, TrueNAS SCALE was released yesterday. Debian+ZFS 
> makes this a lot more useful than when it was FreeBSD based.

I know this is a LUG list, but out of (perhaps morbid) curiosity, why "a lot 
more useful"?

-- 
Jason T. Nelson 
GPG key 0xFF676C9E
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-23 Thread Mark Komarinski
With LVM (and it looks like btrfs) you can pool mirrored drives together 
into what is effectively a RAID10 and you can remove individual mirrors 
to shrink or grow the pool.  ZFS does not allow you to do that. Once you 
expand a pool there's no going back.  You can replace individual drives 
in a mirror with like or larger, but you can't go smaller and the only 
way for that pool to be bigger is to replace all the drives one at a time.

This is less of an issue now that I just build out a RAIDZ2 with 5 6xTB 
drives and make that the entire pool.  Given your use case, I'd say go 
with btrfs.

For everyone else, TrueNAS SCALE was released yesterday. Debian+ZFS 
makes this a lot more useful than when it was FreeBSD based.

-Mark

On 2/23/2022 11:25 AM, Ben Scott wrote:
> Hi all,
>
> We haven't had a really good flamewar ^W discussion on here in far too long...
>
> SUMMARY
>
> Btfrs vs ZFS. I was wondering if others would like to share their
> opinions on either or both?  Or something else entirely?  (Maybe you
> just don't feel alive if you're not compiling your kernel from
> patches?)  Especially cool would be recent comparisons of two or more.
>
> I'll provide an info dump of my plans below, but I do so mainly as
> discussion-fodder.  Don't feel obligated to address my scenario in
> particular.  Of course, commentary on anything in particular that
> seems like a good/bad/cool idea is still welcome.
>
> RECEIVED WISDOM
>
> This is the stuff every article says.  I rarely find anything that goes 
> deeper.
>
> - ZFS has been around/stable/whatever longer
> - btfrs has been on Linux longer
> - btfrs is GPL, ZFS is CDDL or whatever
> - Licensing kept ZFS off Linux for a while
> - ZFS is available on major Linux distros now
> - People say one is faster, but disagree on which one
> - Oracle is a bag of dicks
> - ZFS is easier to pronounce
>
> For both, by coupling the filesystem layer and the block layer, we get
> a lot of advantages, especially for things like snapshots and
> deduplication.  The newcomers also get you things like checksums for
> every block, fault-tolerance over heterogenous physical devices, more
> encryption and compression options.  Faster, bigger, longer, lower,
> wider, etc., etc.  More superlatives than any other filesystem.
>
> MY SCENARIO
>
> I'm going to be building a new home server soon.  Historically I've
> used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are
> using smarter filesystems these days.  I should really get with the
> times.  They do seem to confer a lot of advantages, at least on paper.
>
> USE CASES
>
> User community is me and my girlfriend and a motley collection of
> computing devices from multiple millenia.  Administrator community is
> me.
>
> Mostly plain old network file storage.  Mixed use within that.  I'm a
> data hoarder.
>
> All sorts of stuff I've downloaded over the years, some not even from
> the Internet (ZMODEM baby!).  So large numbers of large write-once
> files.  "Large" has changed over the years, from something that fills
> a floppy diskette to something that fills a DVD, but they don't change
> once written.  ISO images, tarballs, music and photo collections
> (FLAC, MP3, JPEG).
>
> Also large numbers of small write-once files.  I've got 20 GB of mail
> archives in maildir format, one file per message, less than 4K per
> file for the old stuff (modern HTML mail is rather bloated).  These
> generally don't change once written either, but there are lots of
> them.  Some single directories have over 200 K files.
>
> Backups of my user systems.  Currently accomplished via rsnapshot and
> rsync (or ROBOCOPY for 'doze).  So small to medium-small files, but
> changing and updating and hardlinking and moving a lot.  With a
> smarter filesystem I can likely dispense with rsnapshot, but I doubt
> I'm going to move away from plain-old-files-as-backup-storage any time
> soon.  (rsync might conceivably be replaced with a smarter network
> filesystem someday, but likely not soon.)
>
> ANTI USE CASES
>
> Not a lot of mass-market videos -- the boob tube is one area where I
> let others do it for me.  (Roku, Netflix, Blu-ray, etc.)
>
> No plans to network mount home directories for my daily-driver PCs.
> For laptops especially that's problematic (and sorting apps
> (particularly browsers) that can copy with a distributed filesystem
> seems unlikely to pay off).
>
> Not planning on any serious hosting of VMs or containers or complex
> application software on this box.  I can't rule it out entirely for
> (especially as an experiment), but this is mainly intended to be a
> NAS-type server.  It will run NFS, Samba, SSH, rsync.  It might run
> some mail daemons (SMTP, IMAP) just to make accessing archives easier,
> but it won't be the public-facing MX for anything.
>
> It's unlikely to run any point-and-drool administration (web) GUIs.  I
> have a set of config files I've been carrying around with me since I
> kept them on floppy diskette, and they've 

Re: ZFS vs btfrs

2022-02-23 Thread Ben Scott
On Wed, Feb 23, 2022 at 11:44 AM Bruce Dawson  wrote:
> Well, you're more concerned with files than large blocks of data, so I
> don't think either matter - other than standard filesystem performance.

I wouldn't go that far.  In particular, snapshots at the block layer
are generally less efficient compared to filesystem integrated
snapshots.  With all those static files, built-in checksums (vs
external MD5/SHA1) is rather nice.  EXT4 does handle large directories
better than its predecessors, but others are still better, or so I'm
told.  Tuning for file sizes and inodes gets old, too.  Mutt disk
collections (i.e., different ages and specs) is suboptimal for
traditional RAID; allegedly the new guys can handle that better.  And
so on.  Granted, I've never actually tried any of these things (or I
wouldn't be asking), but the brochure makes them sound really cool.

> ZFS is nice, but resource intensive.

  Care to expound?  Even if your use case isn't easily analogous to
mine, it'd be interesting reading for me and others, I expect.

> How about choosing btrfs so we (the community) can learn more about it?!

  Well, whichever road I end up going down, I'll be happy to share
notes on the route once I get there.  :-)

-- Ben
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/


Re: ZFS vs btfrs

2022-02-23 Thread Bruce Dawson
Well, you're more concerned with files than large blocks of data, so I 
don't think either matter - other than standard filesystem performance. 
I've had some experience with ZFS, and practically none with btrfs. ZFS 
is nice, but resource intensive. I suspect btrfs is similar, but 
probably not in standard Linux development uses.

How about choosing btrfs so we (the community) can learn more about it?!

--Bruce

On 2/23/22 11:25, Ben Scott wrote:
> Hi all,
>
> We haven't had a really good flamewar ^W discussion on here in far too long...
>
> SUMMARY
>
> Btfrs vs ZFS. I was wondering if others would like to share their
> opinions on either or both?  Or something else entirely?  (Maybe you
> just don't feel alive if you're not compiling your kernel from
> patches?)  Especially cool would be recent comparisons of two or more.
>
> I'll provide an info dump of my plans below, but I do so mainly as
> discussion-fodder.  Don't feel obligated to address my scenario in
> particular.  Of course, commentary on anything in particular that
> seems like a good/bad/cool idea is still welcome.
>
> RECEIVED WISDOM
>
> This is the stuff every article says.  I rarely find anything that goes 
> deeper.
>
> - ZFS has been around/stable/whatever longer
> - btfrs has been on Linux longer
> - btfrs is GPL, ZFS is CDDL or whatever
> - Licensing kept ZFS off Linux for a while
> - ZFS is available on major Linux distros now
> - People say one is faster, but disagree on which one
> - Oracle is a bag of dicks
> - ZFS is easier to pronounce
>
> For both, by coupling the filesystem layer and the block layer, we get
> a lot of advantages, especially for things like snapshots and
> deduplication.  The newcomers also get you things like checksums for
> every block, fault-tolerance over heterogenous physical devices, more
> encryption and compression options.  Faster, bigger, longer, lower,
> wider, etc., etc.  More superlatives than any other filesystem.
>
> MY SCENARIO
>
> I'm going to be building a new home server soon.  Historically I've
> used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are
> using smarter filesystems these days.  I should really get with the
> times.  They do seem to confer a lot of advantages, at least on paper.
>
> USE CASES
>
> User community is me and my girlfriend and a motley collection of
> computing devices from multiple millenia.  Administrator community is
> me.
>
> Mostly plain old network file storage.  Mixed use within that.  I'm a
> data hoarder.
>
> All sorts of stuff I've downloaded over the years, some not even from
> the Internet (ZMODEM baby!).  So large numbers of large write-once
> files.  "Large" has changed over the years, from something that fills
> a floppy diskette to something that fills a DVD, but they don't change
> once written.  ISO images, tarballs, music and photo collections
> (FLAC, MP3, JPEG).
>
> Also large numbers of small write-once files.  I've got 20 GB of mail
> archives in maildir format, one file per message, less than 4K per
> file for the old stuff (modern HTML mail is rather bloated).  These
> generally don't change once written either, but there are lots of
> them.  Some single directories have over 200 K files.
>
> Backups of my user systems.  Currently accomplished via rsnapshot and
> rsync (or ROBOCOPY for 'doze).  So small to medium-small files, but
> changing and updating and hardlinking and moving a lot.  With a
> smarter filesystem I can likely dispense with rsnapshot, but I doubt
> I'm going to move away from plain-old-files-as-backup-storage any time
> soon.  (rsync might conceivably be replaced with a smarter network
> filesystem someday, but likely not soon.)
>
> ANTI USE CASES
>
> Not a lot of mass-market videos -- the boob tube is one area where I
> let others do it for me.  (Roku, Netflix, Blu-ray, etc.)
>
> No plans to network mount home directories for my daily-driver PCs.
> For laptops especially that's problematic (and sorting apps
> (particularly browsers) that can copy with a distributed filesystem
> seems unlikely to pay off).
>
> Not planning on any serious hosting of VMs or containers or complex
> application software on this box.  I can't rule it out entirely for
> (especially as an experiment), but this is mainly intended to be a
> NAS-type server.  It will run NFS, Samba, SSH, rsync.  It might run
> some mail daemons (SMTP, IMAP) just to make accessing archives easier,
> but it won't be the public-facing MX for anything.
>
> It's unlikely to run any point-and-drool administration (web) GUIs.  I
> have a set of config files I've been carrying around with me since I
> kept them on floppy diskette, and they've served me well.  Those that
> like them, more power to you, but they're not for me.  I inevitably
> bump into their limitations and have to go outside them anyway.
>
> I've tried a few consumer NAS appliances and have generally been
> disappointed.  It's the same as the GUI thing above, except I hit the
> limits sooner and 

ZFS vs btfrs

2022-02-23 Thread Ben Scott
Hi all,

We haven't had a really good flamewar ^W discussion on here in far too long...

SUMMARY

Btfrs vs ZFS. I was wondering if others would like to share their
opinions on either or both?  Or something else entirely?  (Maybe you
just don't feel alive if you're not compiling your kernel from
patches?)  Especially cool would be recent comparisons of two or more.

I'll provide an info dump of my plans below, but I do so mainly as
discussion-fodder.  Don't feel obligated to address my scenario in
particular.  Of course, commentary on anything in particular that
seems like a good/bad/cool idea is still welcome.

RECEIVED WISDOM

This is the stuff every article says.  I rarely find anything that goes deeper.

- ZFS has been around/stable/whatever longer
- btfrs has been on Linux longer
- btfrs is GPL, ZFS is CDDL or whatever
- Licensing kept ZFS off Linux for a while
- ZFS is available on major Linux distros now
- People say one is faster, but disagree on which one
- Oracle is a bag of dicks
- ZFS is easier to pronounce

For both, by coupling the filesystem layer and the block layer, we get
a lot of advantages, especially for things like snapshots and
deduplication.  The newcomers also get you things like checksums for
every block, fault-tolerance over heterogenous physical devices, more
encryption and compression options.  Faster, bigger, longer, lower,
wider, etc., etc.  More superlatives than any other filesystem.

MY SCENARIO

I'm going to be building a new home server soon.  Historically I've
used Linux RAID and LVM and EXT2/3/4/5/102, but all the cool kids are
using smarter filesystems these days.  I should really get with the
times.  They do seem to confer a lot of advantages, at least on paper.

USE CASES

User community is me and my girlfriend and a motley collection of
computing devices from multiple millenia.  Administrator community is
me.

Mostly plain old network file storage.  Mixed use within that.  I'm a
data hoarder.

All sorts of stuff I've downloaded over the years, some not even from
the Internet (ZMODEM baby!).  So large numbers of large write-once
files.  "Large" has changed over the years, from something that fills
a floppy diskette to something that fills a DVD, but they don't change
once written.  ISO images, tarballs, music and photo collections
(FLAC, MP3, JPEG).

Also large numbers of small write-once files.  I've got 20 GB of mail
archives in maildir format, one file per message, less than 4K per
file for the old stuff (modern HTML mail is rather bloated).  These
generally don't change once written either, but there are lots of
them.  Some single directories have over 200 K files.

Backups of my user systems.  Currently accomplished via rsnapshot and
rsync (or ROBOCOPY for 'doze).  So small to medium-small files, but
changing and updating and hardlinking and moving a lot.  With a
smarter filesystem I can likely dispense with rsnapshot, but I doubt
I'm going to move away from plain-old-files-as-backup-storage any time
soon.  (rsync might conceivably be replaced with a smarter network
filesystem someday, but likely not soon.)

ANTI USE CASES

Not a lot of mass-market videos -- the boob tube is one area where I
let others do it for me.  (Roku, Netflix, Blu-ray, etc.)

No plans to network mount home directories for my daily-driver PCs.
For laptops especially that's problematic (and sorting apps
(particularly browsers) that can copy with a distributed filesystem
seems unlikely to pay off).

Not planning on any serious hosting of VMs or containers or complex
application software on this box.  I can't rule it out entirely for
(especially as an experiment), but this is mainly intended to be a
NAS-type server.  It will run NFS, Samba, SSH, rsync.  It might run
some mail daemons (SMTP, IMAP) just to make accessing archives easier,
but it won't be the public-facing MX for anything.

It's unlikely to run any point-and-drool administration (web) GUIs.  I
have a set of config files I've been carrying around with me since I
kept them on floppy diskette, and they've served me well.  Those that
like them, more power to you, but they're not for me.  I inevitably
bump into their limitations and have to go outside them anyway.

I've tried a few consumer NAS appliances and have generally been
disappointed.  It's the same as the GUI thing above, except I hit the
limits sooner and in more ways.  Some of them have really disgusting
software internals.  (A shame, because some of the hardware is
appealing, especially in terms of watts and price.)

I don't want to put this on somebody else's computer.

HARDWARE

I'm shooting for a super compact PC chassis, mini-ITX mainboard, 4 x
3.5-inch hot swap bays, SATA interfaces, x86-64 processor.  Initially
it will be two spinning disks.  Somewhere in the neighborhood of 3 to
6 TB effective.  The disks will be relatively slow, favoring lower
price-per-GB and less heat over performance.  This is bulk data
storage.  The user PCs have SSDs.  If fancy filesystems