On Wed, Sep 20, 2017 at 2:06 PM, shally verma <shallyvermacav...@gmail.com> wrote: > On Mon, Sep 18, 2017 at 7:11 PM, Timofey Titovets <nefelim...@gmail.com> > wrote: >> 2017-09-18 16:28 GMT+03:00 shally verma <shallyvermacav...@gmail.com>: >>> On Mon, Sep 18, 2017 at 1:56 PM, Timofey Titovets <nefelim...@gmail.com> >>> wrote: >>>> 2017-09-18 10:36 GMT+03:00 shally verma <shallyvermacav...@gmail.com>: >>>>> Hi >>>>> >>>>> I wanted to test btrfs compression using fio command but somehow >>>>> during fio writes, I don't see code taking route of compression blocks >>>>> where as If I do a copy to btrfs compression enabled mount point then >>>>> I can easily see code falling through compression.c. >>>>> >>>>> Here's how I do my setup >>>>> >>>>> 1. mkfs.btrfs /dev/sdb1 >>>>> 2. mount -t btrfs -o compress=zlib,compress-force /dev/sdb1 /mnt >>>>> 3. cp <some large test file> /mnt >>>>> 4. dmesg shows print staments from compression.c and zlib.c confirming >>>>> compression routine was invoked during write >>>>> 5. now, copy back from btrfs mount point to home directory also shows >>>>> decompress call invokation >>>>> >>>>> Now, try same with fio commands: >>>>> >>>>> fio command >>>>> >>>>> fio --directory=/mnt/ --numjobs=1 --direct=0 --buffered=1 >>>>> --ioengine=libaio --group_reporting --bs=64k --rw=write --iodepth=128 >>>>> --name=test --size=10G --runtime=180 --time_based >>>>> >>>>> But it seems to write uncompressed data. >>>>> >>>>> Any help here? what's missing? >>>>> >>>>> Thanks >>>>> Shally >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>>>> the body of a message to majord...@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>>> 1. mount -t btrfs -o compress=zlib,compress-force -> compress-force=zlib >>>> 2. Tune fio to generate compressible data >>>> >>> How do I "tune" fio to generate data. I had assumed once compression >>> is enabled on btrfs any system fwrite call will simply compress data >>> into it .Isn't it so? >>> Can you share fio command that I can test? >>> Thanks >>> Shally >>>> >>>> -- >>>> Have a nice day, >>>> Timofey. >> >> That useless to compress uncompressible data. >> Also, as you enable compress, not compress-force >> So after first uncompressible write btrfs just stop trying compress that >> file. >> >> From man fio: >> buffer_compress_percentage=int >> If this is set, then fio will attempt to provide I/O >> buffer content (on WRITEs) that compresses to the specified level. Fio >> does this by providing a mix of random data and a fixed pattern. The >> fixed pattern is either >> zeros, or the pattern specified by buffer_pattern. If >> the pattern option is used, it might skew the compression ratio >> slightly. Note that this is per block size unit, for file/disk wide >> compression level that matches this >> setting, you'll also want to set refill_buffers. >> >> buffer_compress_chunk=int >> See buffer_compress_percentage. This setting allows fio >> to manage how big the ranges of random data and zeroed data is. >> Without this set, fio will provide buffer_compress_percentage of >> blocksize random data, followed by >> the remaining zeroed. With this set to some chunk size >> smaller than the block size, fio can alternate random and zeroed data >> throughout the I/O buffer. >> >> Good luck :) > > Now. I did following: > > 1. mount -t btrfs -o compress-force=zlib /dev/sdb1 mnt > > 2. fio --directory=mnt/ --numjobs=1 --direct=0 --buffered=1 --bs=64k > --rw=write --iodepth=128 --name=test --size=1G > --buffer_compress_percentage=100 --buffer_pattern=0xFF --refill_buffer > --ioengine=libaio > > 1GN file written uncompressed. Here no compression invoked (though > compress-force=zlib) > > 3. cp mnt/test ./ --> copy back fio generated test file from btrfs > mount point to local drive > > 4. hex dump test file (all FFs) -- confirmed that data is compressible > no random data. > > 5. cp test mnt/ --> now, copy same test again back to mount point > (reverse of step 3) . Now, here I see during copying compression is > invoked. > > I am using kernel 4.9 and compress-foce is said to be working for > kernel > 2.13 from wiki ... so I wonder what's so special with cp > command which is not happening during fio writes??? > One more catch... I am initiating fio from non-btrfs filesystem i.e. pwd is ext4 based fs where as mount point is btrfs. Could that make difference?
> Thanks > Shally > > >> -- >> Have a nice day, >> Timofey. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
