Re: Btrfs + compression = slow performance and high cpu usage

Message - From: "Peter Grandi" <p...@btrfs.list.sabi.co.uk> To: "Linux fs Btrfs" <linux-btrfs@vger.kernel.org> Sent: Tuesday, 1 August, 2017 10:09:03 PM Subject: Re: Btrfs + compression = slow performance and high cpu usage >> [ ... ] a "RAID5 wit

Re: Btrfs + compression = slow performance and high cpu usage

[ ... ] > This is the "storage for beginners" version, what happens in > practice however depends a lot on specific workload profile > (typical read/write size and latencies and rates), caching and > queueing algorithms in both Linux and the HA firmware. To add a bit of slightly more advanced

Re: Btrfs + compression = slow performance and high cpu usage

>> [ ... ] a "RAID5 with 128KiB writes and a 768KiB stripe >> size". [ ... ] several back-to-back 128KiB writes [ ... ] get >> merged by the 3ware firmware only if it has a persistent >> cache, and maybe your 3ware does not have one, > KOS: No I don't have persistent cache. Only the 512 Mb cache

Re: Btrfs + compression = slow performance and high cpu usage

- Original Message - From: "Peter Grandi" <p...@btrfs.list.sabi.co.uk> To: "Linux fs Btrfs" <linux-btrfs@vger.kernel.org> Sent: Tuesday, 1 August, 2017 3:14:07 PM Subject: Re: Btrfs + compression = slow performance and high cpu usage > Peter, I

Re: Btrfs + compression = slow performance and high cpu usage

> Peter, I don't think the filefrag is showing the correct > fragmentation status of the file when the compression is used. As reported on a previous message the output of 'filefrag -v' which can be used to see what is going on: filefrag /mnt/sde3/testfile /mnt/sde3/testfile: 49287

RE: Btrfs + compression = slow performance and high cpu usage

linux-btrfs@vger.kernel.org> > Subject: Re: Btrfs + compression = slow performance and high cpu usage > > Peter, I don't think the filefrag is showing the correct fragmentation status > of > the file when the compression is used. > At least the one that is installed by default i

Re: Btrfs + compression = slow performance and high cpu usage

the data on the drive? kos - Original Message - From: "Peter Grandi" <p...@btrfs.list.sabi.co.uk> To: "Linux fs Btrfs" <linux-btrfs@vger.kernel.org> Sent: Monday, 31 July, 2017 1:41:07 PM Subject: Re: Btrfs + compression = slow performance and high cpu us

Re: Btrfs + compression = slow performance and high cpu usage

> [ ... ] It is hard for me to see a speed issue here with > Btrfs: for comparison I have done a simple test with a both a > 3+1 MD RAID5 set with a 256KiB chunk size and a single block > device on "contemporary" 1T/2TB drives, capable of sequential > transfer rates of 150-190MB/s: [ ... ] The

Re: Btrfs + compression = slow performance and high cpu usage

[ ... ] > Also added: Feeling very generous :-) today, adding these too: soft# mkfs.btrfs -mraid10 -draid10 -L test5 /dev/sd{b,c,d,e}3 [ ... ] soft# mount -t btrfs -o commit=10,compress-force=zlib /dev/sdb3 /mnt/test5 soft# rm -f /mnt/test5/testfile soft# /usr/bin/time dd

Re: Btrfs + compression = slow performance and high cpu usage

[ ... ] > grep 'model name' /proc/cpuinfo | sort -u > model name : Intel(R) Xeon(R) CPU E5645 @ 2.40GHz Good, contemporary CPU with all accelerations. > The sda device is a hardware RAID5 consisting of 4x8TB drives. [ ... ] > Strip Size : 256 KB So the full RMW data

Re: Btrfs + compression = slow performance and high cpu usage

i" <p...@btrfs.list.sabi.co.uk> To: "Linux fs Btrfs" <linux-btrfs@vger.kernel.org> Sent: Friday, 28 July, 2017 8:08:47 PM Subject: Re: Btrfs + compression = slow performance and high cpu usage > I am stuck with a problem of btrfs slow performance when using > compression. [

Re: Btrfs + compression = slow performance and high cpu usage

In addition to my previous "it does not happen here" comment, if someone is reading this thread, there are some other interesting details: > When the compression is turned off, I am able to get the > maximum 500-600 mb/s write speed on this disk (raid array) > with minimal cpu usage. No details

Re: Btrfs + compression = slow performance and high cpu usage

On Fri, Jul 28, 2017 at 06:20:14PM +, William Muriithi wrote: > Hi Roman, > > > autodefrag > > This sure sounded like a good thing to enable? on paper? right?... > > The moment you see anything remotely weird about btrfs, this is the first > thing you have to disable and retest without. Oh

RE: Btrfs + compression = slow performance and high cpu usage

Hi Roman, > autodefrag This sure sounded like a good thing to enable? on paper? right?... The moment you see anything remotely weird about btrfs, this is the first thing you have to disable and retest without. Oh wait, the first would be qgroups, this one is second. What's the problem with

Re: Btrfs + compression = slow performance and high cpu usage

> I am stuck with a problem of btrfs slow performance when using > compression. [ ... ] That to me looks like an issue with speed, not performance, and in particular with PEBCAK issues. As to high CPU usage, when you find a way to do both compression and checksumming without using much CPU time,

Re: Btrfs + compression = slow performance and high cpu usage

On Fri, 28 Jul 2017 17:40:50 +0100 (BST) "Konstantin V. Gavrilenko" wrote: > Hello list, > > I am stuck with a problem of btrfs slow performance when using compression. > > when the compress-force=lzo mount flag is enabled, the performance drops to > 30-40 mb/s and