Re: [PATCH 0/5] btrfs: lz4/lz4hc compression
On Sat, Jul 07, 2012 at 11:06:44PM +0200, Jan Killius wrote: I get lots of kick the bucket messages with the newest lz4 patches. Is there any way to debug/fix this ? That was a debugging message, harmless, will be removed in another round. david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] btrfs: lz4/lz4hc compression
Hallo, I get lots of kick the bucket messages with the newest lz4 patches. Is there any way to debug/fix this ? Here is a dmesg snippet: [34114.165731] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131587 hdr 8, ino 1948055 [34114.165733] ... kick the bucket [34114.166268] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131587 hdr 8, ino 1948055 [34114.166270] ... kick the bucket [34114.166309] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131518 hdr 8, ino 1948055 [34114.166312] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131492 hdr 8, ino 1948055 [34114.166313] ... kick the bucket [34114.166314] ... kick the bucket [34114.166330] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131549 hdr 8, ino 1948055 [34114.166332] ... kick the bucket [34114.166373] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131521 hdr 8, ino 1948055 [34114.166375] ... kick the bucket [34114.166406] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131565 hdr 8, ino 1948055 [34114.166409] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131587 hdr 8, ino 1948055 [34114.166410] ... kick the bucket [34114.166411] ... kick the bucket [34114.166687] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131587 hdr 8, ino 1948055 [34114.166689] ... kick the bucket [34119.160244] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131587 hdr 8, ino 1948044 [34119.160246] ... kick the bucket [34119.160539] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131587 hdr 8, ino 1948044 [34119.160540] ... kick the bucket [34119.162139] lz4: pg_out 33 32 nr_dest, nr_out 33; len 131072 out_len 131577 hdr 8, ino 1948044 [34119.162144] ... kick the bucket -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] btrfs: lz4/lz4hc compression
On Sat, Jun 23, 2012 at 04:05:43AM -0400, Andrew Mahone wrote: A large portion of the work was done by Dave Sterba, but in flattening the commits attribution has been lost. My contribution has been in debugging some of the cases in which it could crash or produce incorrect data, and cleaning it up for the ML. Sorry, but how could the attribution get _lost_ ? I wrote like 80% of the code and there's no single Signed-off-by for me? Documentation/SubmittingPatches ch. 12 is quite clear how to use the S-O-B line. I'm glad that somebody came and helped to move the patchset forward, I don't mind sharing the credit for the respective parts. I'm intentionally publishing some WIP branches or paritally working research projects in my git repo as I don't have enough time to finalize all of them. In this case it's just your work on patch [3/5] in file lz4_wrapper.c. Andrew Mahone (5): btrfs: lz4: import lz4/lz4hc C and header files btrfs: lz4: add incompat flags and compression types btrfs: lz4: add lz4_wrapper implementing btrfs compression interface btrfs: lz4: add lz4 files in Makefile btrfs: enable lz4/lz4hc compression david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/5] btrfs: lz4/lz4hc compression
WARNING: This is not compatible with the previous lz4 patchset. If you're using experimental compression that isn't in mainline kernels, be prepared to backup and restore or decompress before upgrading, and have backups in case it eats data (which appears not to be a problem any more, but has been during development). These patches add lz4 and lz4hc compression (https://code.google.com/p/lz4/) to btrfs. Compression with lz4 is about the same in throughput and compression ratio as lzo, but decompression is a good deal faster. In practice, this implementation also compresses faster and better than lzo because the existing lzo implementation breaks extents into pages that are compressed individually. The lz4hc compressor is a slower, higher-compression variation that is doesn't quite match zlib on compression throughput or ratio, but is much faster at decompression. Some quick benchmarks, writing and reading a tarball of the Silesia corpus (http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia) in a VM on a tmpfs-backed device. Times are in milliseconds. method write read size none 489 356 202.27MB lzo 1618 632 115.62MB zlib6583 1370 76.07MB lz4 1216 368 102.15MB lz4hc 8372 337 81.38MB And with linux kernel sources (data size is btrfs fi df data size, may be underestimated due to inline extents). method write read size none8255 9880 446.11MB lzo11867 9011 225.36MB zlib 20340 10419 154.22MB lz410642 8417 203.70MB lz4hc 20270 8280 171.68MB A large portion of the work was done by Dave Sterba, but in flattening the commits attribution has been lost. My contribution has been in debugging some of the cases in which it could crash or produce incorrect data, and cleaning it up for the ML. Andrew Mahone (5): btrfs: lz4: import lz4/lz4hc C and header files btrfs: lz4: add incompat flags and compression types btrfs: lz4: add lz4_wrapper implementing btrfs compression interface btrfs: lz4: add lz4 files in Makefile btrfs: enable lz4/lz4hc compression fs/btrfs/Makefile | 2 +- fs/btrfs/compression.c | 2 + fs/btrfs/compression.h | 2 + fs/btrfs/ctree.h | 16 +- fs/btrfs/disk-io.c | 4 + fs/btrfs/ioctl.c | 5 + fs/btrfs/lz4.c | 843 + fs/btrfs/lz4.h | 128 fs/btrfs/lz4_wrapper.c | 419 fs/btrfs/lz4hc.c | 685 fs/btrfs/lz4hc.h | 58 fs/btrfs/super.c | 16 +- 12 files changed, 2174 insertions(+), 6 deletions(-) create mode 100644 fs/btrfs/lz4.c create mode 100644 fs/btrfs/lz4.h create mode 100644 fs/btrfs/lz4_wrapper.c create mode 100644 fs/btrfs/lz4hc.c create mode 100644 fs/btrfs/lz4hc.h -- 1.7.11 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
