Gandalf Corvotempesta posted on Wed, 02 May 2018 19:25:41 +0000 as excerpted:
> On 05/02/2018 03:47 AM, Duncan wrote: >> Meanwhile, have you looked at zfs? Perhaps they have something like >> that? > > Yes, i've looked at ZFS and I'm using it on some servers but I don't > like it too much for multiple reasons, in example: > > 1) is not officially in kernel, we have to build a module every time > with DKMS FWIW zfz is excluded from my choice domain as well, due to the well known license issues. Regardless of strict legal implications, because Oracle has copyrights they could easily solve that problem and the fact that they haven't strongly suggests they have no interest in doing so. That in turn means they have no interest in people like me running zfs, which means I have no interest in it either. But because it does remain effectively the nearest to btrfs features and potential features "working now" solution out there, for those who simply _must_ have it and/or find it a more acceptable solution than cobbling together a multi-layer solution out of a standard filesystem on top of device-mapper or whatever, it's what I and others point to when people wonder about missing or unstable btrfs features. > I'm new to BTRFS (if fact, i'm not using it) and I've seen in the status > page that "it's almost ready". > The only real missing part is a stable, secure and properly working > RAID56, > so i'm thinking why most effort aren't directed to fix RAID56 ? Well, they are. But finding and fixing corner-case bugs takes time and early-adopter deployments, and btrfs doesn't have the engineering resources to simply assign to the problem that Sun had with zfs. Despite that, as I stated, current btrfs raid56 is, to the best of my/ list knowledge, the current code is now reasonably ready, tho it'll take another year or two without serious bug reports to actually test that, but it simply has the well known write hole that applies to all parity- raid unless they've taken specific measures such as partial-stripe-write logging (slow), writing a full stripe even if it's partially empty (wastes space and needs periodic maintenance to reclaim it), or variable- stripe-widths (needs periodic maintenance and more complex than always writing full stripes even if they're partially empty) (both of the latter avoiding the problem by avoiding in-place read-modify-write cycle entirely). So to a large degree what's left is simply time for testing to demonstrate stability on the one hand, and a well known problem with parity-raid in general on the other. There's the small detail that said well-known write hole has additional implementation-detail implications on btrfs, but at it's root it's the same problem all parity-raid has, and people choosing parity-raid as a solution are already choosing to either live with it or ameliorate it in some other way (tho some parity-raid solutions have that amelioration built-in). > There are some environments where a RAID1/10 is too expensive and a > RAID6 is mandatory, > but with the current state of RAID56, BTRFS can't be used for valuable > data Not entirely true. Btrfs, even btrfs raid56 mode, _can_ be used for "valuable" data, it simply requires astute /practical/ definitions of "valuable", as opposed to simple claims that don't actually stand up in practice. Here's what I mean: The sysadmin's first rule of backups defines "valuable data" by the number of backups it's worth making of that data. If there's no backups, then by definition the data is worth less than the time/hassle/resources necessary to have that backup, because it's not a question of if, but rather when, something's going to go wrong with the working copy and it won't be available any longer. Additional layers of backup and whether one keeps geographically separated off-site backups as well are simply extensions of the first- level-backup case/rule. The more valuable the data, the more backups it's worth having of it, and the more effort is justified in ensuring that single or even multiple disasters aren't going to leave no working backup. With this view, it's perfectly fine to use btrfs raid56 mode for "valuable" data, because that data is backed up and that backup can be used as a fallback if necessary. True, the "working copy" might not be as reliable as it is in some cases, but statistically, that simply brings the 50% chance of failure rate (or whatever other percentage chance you choose) closer, to say once a year, or once a month, rather than perhaps once or twice a decade. Working copy failure is GOING to happen in any case, it's just a matter of playing the chance game as to when, and using a not yet fully demonstrated reliable filesystem mode simply brings ups the chances a bit. But if the data really *is* defined as "valuable", not simply /claimed/ to be valuable, then by that same definition, it *will* have a backup. In the worst case, when some component (here the filesystem) of the storage platform is purely testing and expected to fail in near real- time, the otherwise working-copy is often not even considered the working copy any longer, but rather, the testing copy, garbage value, because it's /expected/ to be destroyed by the test and therefore can't be considered the working copy. In that case, what would be the first-line backup is actually the working copy, and if the testing copy (that would otherwise be the working copy) actually happens to survive the test, it's often deliberately destroyed anyway, with a mkfs (for the filesystem layer, device replace if it's the hardware layer, etc) destroying the data and setting up for the next test or actual working deployment. And by that view, even if btrfs raid56 mode is defined as entirely unreliable, it can still be used for "valuable" data, because by definition, "valuable" data will be backed up, and should the working copy fail for any reason (remembering that even in the best cases, it *WILL* fail, the only question is when, not if), no problem, because the data *was* defined as valuable enough to have a backup, which can simply be restored, or fallen back to, if the first-line backup is deployable as a fallback and there's further backups to allow that without demoting the value to trivial because the working copy is now the /only/ copy. > Also, i've seen that to fix write hole, a dedicated disk is needed ? Is > this true ? > I cant' create a 6 disks RAID6 with only 6 disks and no write-hole like > with ZFS ? A dedicated disk is not /necessary/, tho depending on the chosen mitigation strategy, it might be /useful/faster/. For partial-stripe-write-logging, a comparatively fast device, say an ssd on an otherwise still legacy spinning-rust raid array, will help alleviate the speed issue. But again, parity-raid isn't normally a go-to solution where performance is paramount in any case, so just using another ordinary spinning-rust device may work, if the performance level is acceptable. For either always-full-stripe writes (writing zeros if the write would otherwise be too small) or variable-stripe widths (smaller stripes if necessary, down to a single data strip plus parity), the tradeoff is different and a dedicated logging device isn't used. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
