2011/7/13 Josef Bacik <jo...@toxicpanda.com> > On Wed, Jul 13, 2011 at 4:53 PM, Manuel Escudero <jmlev...@gmail.com> > wrote: > > Today I'll be switching from BTRFS to Ext4 again because of the troubles > > I've been having with > > the New Linux Filesystem. As BTRFS is going to be the Default in F16 I > > wanted the developers to > > know what kind of troubles I've been experiencing with this FS in F15 so > > they can take a look > > at them in order to have a better F16 release: > > The Good: > > Since BTRFS arrived into my computer (Everything in the HDD is formated > with > > BTRFS excluding "/boot") > > I've seen a performance improvement in the data transfer part from and to > > the computer (copying files seem to > > be faster than before) But that's all about the good things I noticed... > > The Bad: > > BTRFS has reduced system's overall performance, at this point, sometimes > it > > is OK, sometimes it is > > VERY BAD, I've noticed "Performance Peaks" in F15 with BTRFS and the Boot > > times are not nice: I mean, > > they are not the slowest ones, but they're not as good as Before in F14 > with > > Ext4 instead of BTRFS. > > The performance Running/Launching apps has been afected too and now the > PC > > freezes sometimes (that never > > happened in F14 unless I forced it a lot with 4 VM's to suck the 4GB of > RAM > > I have); And Now it freezes > > very often when it wants without a lot of effort. > > The Ugly: > > Running VM's when having their virtual HDD's stored in a BTRFS partition > is > > DEATH! > > They're very slow, sometimes they open, sometimes they not, usually they > > freeze, You can't > > work with them. Same thing about Gnome Shell working over a BTRFS > partition: > > it is really slow, > > sometimes it reacts but most of the time is pretty unresponsive. > > Reading in the Web, I found that some users think that the BTRFS poor > > performance is caused by some > > special kind of fragmentation it suffers, others think it's because of > it's > > CopyonWrite attributes and some > > others blame other stuff, God Knows! the only thing I know is that BTRFS > is > > not ready for being > > used in normal production machines (as I tought) and it needs to be fixed > > before the release of F16, because it's > > performance is really far from good... > > Other Stuff I noticed is that with Kernel 2.6.38.8-35 the system seems to > > work better that with the previous one, > > just a little, but is some kind of improvement. > > Here you have all the info I found on the net about BTRFS Performance > > issues noticed by users: > > https://bugzilla.redhat.com/show_bug.cgi?id=689127 > > http://arosenfeld.wordpress.com/2010/12/27/back-to-ext4-from-btrfs/ > > http://www.vyatta4people.org/btrfs-is-a-bad-choice-when-running-kvm/ > > http://lkml.org/lkml/2010/7/13/475 > > http://blog.patshead.com/2011/03/btrfs---six-months-later.html > > I only have a question: > > Why Any Kind of VM is Sooo Slow when being stored on a BTRFS > > partition? Any Way to Solve this? or at least have a BTRFS performance > > improvement? > > Yeah VMs are a particular problem with Btrfs. There are a ton of > reasons for this, for example by default we use fsync. Fsync _sucks_ > for btrfs currently, and it has historically not been a well optimized > piece of code. I'm working on fixing this, but it requires VFS level > changes that are currently sitting in Al's queue. I suspect they will > go into 3.1 and so we can move ahead with our work, but for now, it > sucks. You can use cache=none you get better performance, but still > not that great. And this is all because of one major thing > > Btrfs has threads for _everything_. This works out fantastically when > you have big chunks of reads or writes you want done. This _sucks_ > when you are doing little piddly io's. The reason for all of this is > because we don't want you to get bottlenecked on us > calculating/verifying checksums, so we farm all IO and endio out to > different threads, which as I said works out great if you are trying > to cram gigs of data down your drives throat. > > But with VMs you are doing small scattered IO's, so the IO comes down, > we prepare it, and farm it off to a thread and wait for that thread to > wake up and submit the io. Then the io is completed and that is > farmed off to another thread and we wait on that. This switching > around and waiting for things to wake up is hugely painful when all > you want to do is write a few bytes. If you were to do > > dd if=/dev/zero of=/mnt/btrfs/file bs=4k count=100 oflag=direct > > on a btrfs fs and then do it on an ext4 fs, you would see about a 20% > difference between the 2. But if you do say bs=20M, the gap closes > quite a bit. > > I fixed part of this problem for O_DIRECT (which is cache=none with > qemu), if the IO's are small we don't send it off to a thread but > submit it within our threads context, which is what got us with 20% of > ext4 as opposed to 50%. The other half is doing the completion in the > submitters context, which is going to take some extra work. I'm > fixing this in the fsync case as well, but as I said we need a VFS > patch to do it properly so that will be a little later coming. After > that I can do the endio part of it and hopefully get us within > spitting distance of ext4. > > So there's my long ass explanation of why VMs on Btrfs suck. I'm > sorry, I'm aware of the problem and I'm trying to fix it, but it's a > slow going process. > > As for your other spikes, can you test an upstream kernel? I've done > various other performance things to try and get rid of those problems > and would like to know how it helps. If the spikes last long enough a > sysrq+w would be very helpful in seeing what is going on so we can try > and address the problems you are seeing. Thanks, > > Josef > -- > devel mailing list > devel@lists.fedoraproject.org > https://admin.fedoraproject.org/mailman/listinfo/devel >
@Josef: Obviously you have more technical knowledge than I do, but I understood the threads/few bytes problem. Somewhere near there was my guessing about the VM's problem... I'll be glad of bulding a vanilla kernel and then "mash it up" into my F15 install but: A) This is a Very important production machine and B) I have a lot of work to do and not so much time... At this moment I'm making a Quick Backup because I'm going to reinstall F15 with Ext4 as FileSystem :( part of my job is working hand by hand with VM's, sorry.. Hope this thing get's solved before the F16 launch, thanks for the explanation! -- Manuel Escudero Linux User #509052 Twitter: @Jmlevick <http://twitter.com/Jmlevick> Blogger: Blog Xenode <http://xenodesystems.blogspot.com/> PGP/GnuPG: E2F5 12FA E1C3 FA58 CF15 8481 B77B 00CA C1E1 0FA7 Xenode Systems - xenodesystems.com <http://www.xenodesystems.com/> - "Conéctate a Tu Mundo"
-- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel