On Thu, 01 Dec 2022 20:49:46 +0800 "Yue Hu" <[email protected]> wrote:
> Add a missing change: > - change to generate a ctx for duplicate fragment in compression. > > On Thu, 1 Dec 2022 19:16:15 +0800 > Yue Hu <[email protected]> wrote: > > > From: Yue Hu <[email protected]> > > > > Previously, there's no fragment deduplication when this feature is > > introduced. Let's support it now. > > > > We intend to dedupe the fragments before compression, so that duplicate > > parts will not be written into packed inode. > > > > With this patch, for Linux 5.10.1 + 5.10.87 source code: > > > > [before] > > 32k pcluster + T0 + lz4hc,12 + fragment 450M > > 64k pcluster + T0 + lz4hc,12 + fragment 434M > > 128k pcluster + T0 + lz4hc,12 + fragment 426M > > 32k pcluster + T0 + lz4hc,12 + fragment + dedupe 368M > > 64k pcluster + T0 + lz4hc,12 + fragment + dedupe 380M > > 128k pcluster + T0 + lz4hc,12 + fragment + dedupe 395M > > > > [after] > > 32k pcluster + T0 + lz4hc,12 + fragment 311M > > 64k pcluster + T0 + lz4hc,12 + fragment 295M > > 128k pcluster + T0 + lz4hc,12 + fragment 287M > > 32k pcluster + T0 + lz4hc,12 + fragment + dedupe 286M > > 64k pcluster + T0 + lz4hc,12 + fragment + dedupe 281M > > 128k pcluster + T0 + lz4hc,12 + fragment + dedupe 278M > > > > Tested on SquashFS (which uses level 12 by default for lz4hc): > > > > 32k block + lz4hc 332M > > 64k block + lz4hc 304M > > 128k block + lz4hc 283M > > 256k block + lz4hc 273M > > 256k block + lz4hc + noI 278M > > > > Suggested-by: Gao Xiang <[email protected]> > > Signed-off-by: Yue Hu <[email protected]> > > --- > > v4: > > - renaming include tofcrc/new_fragmentsize > > - move fixup into ctx > > - use may_fixing to check packing fragment or not > > - move sb/inode flag + 64bits case from erofs_pack_fragments() to new > > helper erofs_fragments_commit() > > - move recompress ahead of may_inline case when compressing succeeds > > - update commit message/code comments > > - note that decompress will fail when enable ztailpacking at the same > > time, need some time to debug No need to care may_inline case if we find duplicate fragment. - bool may_inline = (cfg.c_ztailpacking && final); + bool may_inline = (cfg.c_ztailpacking && final && + !inode->fragment_size); Should be included in v5. > > > > v3:
