The HTML...it hurts my eyes... :) On Thu, Sep 18, 2014 at 4:24 AM, J. Roeleveld <jo...@antarean.org> wrote: > > On Wednesday, September 17, 2014 08:56:28 PM James wrote: > >> Alec Ten Harmsel <alec <at> alectenharmsel.com> writes: > >> > As far as HDFS goes, I would only set that up if you will use it for >> > Hadoop or related tools. It's highly specific, and the performance is >> > not good unless you're doing a massively parallel read (what it was >> > designed for). I can elaborate why if anyone is actually interested. >
FYI - one very big limitation of hdfs is its minimum filesize is something huge like 1MB or something like that. Hadoop was designed to take a REALLY big input file and chunk it up. If you use hdfs to store something like /usr/portage it will turn into the sort of monstrosity that you'd actually need a cluster to store. > > My current understanding is: > > - ZFS is production ready, but due to licensing issues, not included in the > kernel > > - BTRFS is included, but not yet production ready with all planned features > Your understanding of their maturity is fairly accurate. They also aren't 100% moving in the same direction - btrfs aims more to be a general-purpose filesystem replacement especially for smaller systems, and zfs is more focused on the enterprise, so it lacks features like raid reshaping (who needs to add 1 disk to a raid5 when you can just add 5 more disks to your 30 disk storage system). I think btrfs has a bit more hope of being an ext4 replacement some day for both this reason and the licensing issue. That in no way detracts from the usefulness of zfs, especially for larger deployments where the few areas where btrfs is more flexible would probably be looked at as gimmicks (kind of like being able to build your whole OS from source :) ). > For me, Raid6-like functionality is an absolute requirement and latest I > know is that that isn't implemented in BTRFS yet. Does anyone know when that > will be implemented and reliable? Eg. what time-frame are we talking about? > I suspect we're talking months before it is really implemented, and much longer before it is reliable. Right now btrfs can write raid6, but it can't really read it. That is, it operates just fine until you actually lose a disk containing something other than parity, and then it loses access to the data. This code is only in the kernel for development purposes and nobody advocates using it for production. Most of the code in btrfs which is reliable has been around for years, like raid1 support, and obviously it will be years until the raid5/6 code reaches that point. I am using btrfs mainly because once that day comes it will be much easier to migrate to it from btrfs raid1 than from zfs (which has no mechanism for migrating raid levels in-place (that is, within an existing vdev) - you would need to add new drives to the pool, migrate the data, and remove the old drives from the pool, which is nice if you have a big stack of drives and spare sata ports lying around like you would in a SAN). -- Rich
