Re: [PATCH 2/3] erofs: add necessary kmem_cache_create flags for erofs inode cache

2023-08-15 Thread Gao Xiang
On 2023/8/15 17:48, Ferry Meng wrote: To improve memory access efficiency and enable statistics functionality, add SLAB_MEM_SPREAD and SLAB_ACCOUNT flag during erofs_icachep's allocation time. Signed-off-by: Ferry Meng Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH] AOSP: erofs-utils: add missing sbi argument to erofs_blknr in block list

2023-08-15 Thread Gao Xiang
igned-off-by: Yue Hu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 1/2 v2] AOSP: erofs-utils: pass a parameter to write tail end in block list

2023-08-15 Thread Gao Xiang
On 2023/8/15 14:45, Yue Hu wrote: On Tue, 15 Aug 2023 14:18:08 +0800 Gao Xiang wrote: On 2023/8/15 14:21, Yue Hu wrote: On Tue, 15 Aug 2023 13:36:46 +0800 Gao Xiang wrote: On 2023/8/15 13:40, Yue Hu wrote: On Tue, 15 Aug 2023 12:59:56 +0800 Gao Xiang wrote: On 2023/8/15 12

Re: [PATCH 1/2 v2] AOSP: erofs-utils: pass a parameter to write tail end in block list

2023-08-15 Thread Gao Xiang
On 2023/8/15 14:21, Yue Hu wrote: On Tue, 15 Aug 2023 13:36:46 +0800 Gao Xiang wrote: On 2023/8/15 13:40, Yue Hu wrote: On Tue, 15 Aug 2023 12:59:56 +0800 Gao Xiang wrote: On 2023/8/15 12:55, Yue Hu wrote: From: Yue Hu We can determine whether the tail block is the first one

Re: [PATCH 1/2 v2] AOSP: erofs-utils: pass a parameter to write tail end in block list

2023-08-14 Thread Gao Xiang
On 2023/8/15 13:40, Yue Hu wrote: On Tue, 15 Aug 2023 12:59:56 +0800 Gao Xiang wrote: On 2023/8/15 12:55, Yue Hu wrote: From: Yue Hu We can determine whether the tail block is the first one or not during the writing process. Therefore, instead of internally checking the block number

Re: [PATCH 1/2 v2] AOSP: erofs-utils: pass a parameter to write tail end in block list

2023-08-14 Thread Gao Xiang
have no idea why we need this, could you describe the Android block map details for discussion? Thanks, Gao Xiang

Re: [PATCH 08/13] erofs-utils: lib: add erofs_inode_tag_opaque() helper

2023-08-14 Thread Gao Xiang
;& + !strncmp(item->kvbuf, OVL_XATTR_OPAQUE + plen, +item->len[0])) + return true; These looks too complicated to me, how about just connect prefix and suffix strings and do a string comparsion together? Thanks, Gao Xiang

Re: [PATCH 05/13] erofs-utils: lib: keep self maintained devname

2023-08-14 Thread Gao Xiang
On 2023/8/14 14:43, Gao Xiang wrote: On 2023/8/14 11:42, Jingbo Xu wrote: Keep self allocated and maintained devname in erofs_sb_info. Signed-off-by: Jingbo Xu --- ... @@ -95,7 +97,11 @@ int dev_open(struct erofs_sb_info *sbi, const char *dev)   return -EINVAL

Re: [PATCH 07/13] erofs-utils: lib: add erofs_read_xattrs_from_disk() helper

2023-08-14 Thread Gao Xiang
len[1], (long)ret); + len[1] = ret; + } Can this actually happen? Maybe just: DBG_BUGON(ret != len[1]); Thanks, Gao Xiang

Re: [PATCH 05/13] erofs-utils: lib: keep self maintained devname

2023-08-14 Thread Gao Xiang
uot;, dev); @@ -136,8 +142,12 @@ int dev_open_ro(struct erofs_sb_info *sbi, const char *dev) return -errno; } + sbi->devname = strdup(dev); + if (!sbi->devname) { + close(fd); + return -ENOMEM; + } Same here. Thanks, Gao Xiang

Re: [PATCH 04/13] erofs-utils: lib: fix erofs_init_devices() in multidev mode

2023-08-14 Thread Gao Xiang
ot;) Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 03/13] erofs-utils: lib: read i_ino in erofs_read_inode_from_disk()

2023-08-14 Thread Gao Xiang
->st_ino. Read on-disk ino and store it in i_ino[0] given the above background. Signed-off-by: Jingbo Xu Fixes: 5e35b75ad499 ("erofs-utils: introduce fuse implementation") Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 02/13] erofs-utils: lib: add list_splice_tail() helper

2023-08-13 Thread Gao Xiang
On 2023/8/14 11:42, Jingbo Xu wrote: Add list_splice_tail() helper. Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 01/13] erofs-utils: fix overriding of i_rdev for special device

2023-08-13 Thread Gao Xiang
Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 12/24] fs/erofs: Quieten test for filesystem presence

2023-08-13 Thread Gao Xiang
Use debugging rather than errors when unable to find a signature, as is done with other filesystems. Signed-off-by: Simon Glass Reviewed-by: Gao Xiang Thanks, Gao Xiang

[PATCH v6] erofs: DEFLATE compression support

2023-08-10 Thread Gao Xiang
ernatively, DEFLATE could still be used for some specific files since EROFS supports multiple compression algorithms in one image. [1] https://developer.apple.com/documentation/compression/compression_algorithm Reviewed-by: Chao Yu Signed-off-by: Gao Xiang --- changes since v5: - fix data corruption

Re: [PATCH 2/2] erofs-utils: lib: remove prototypes of removed functions

2023-08-10 Thread Gao Xiang
On 2023/8/10 14:46, Jingbo Xu wrote: Remove prototypes of those that have been deleted. Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 1/2] erofs-utils: mkfs: fix double write of long xattr name prefixes

2023-08-10 Thread Gao Xiang
tils: introduce tarerofs") Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH] erofs: refine warning messages for data I/Os

2023-08-09 Thread Gao Xiang
On 2023/8/9 14:06, Ferry Meng wrote: Don't warn users since -EINTR is returned due to user interruption. Also suppress warning messages of readmore. Signed-off-by: Ferry Meng Reviewed-by: Gao Xiang Thanks, Gao Xiang

[PATCH v2 1/2] erofs-utils: lib: add a way to request supported algorithms

2023-08-08 Thread Gao Xiang
dump.erofs needs to print supported algorithms instead of available compressors. In addition, clean up erofs_get_compress_algorithm_id() too. Cc: Guo Xuenan Signed-off-by: Gao Xiang --- changes since v1: - fix `available compressors` print. fsck/main.c | 8 ++-- include

[PATCH v2 2/2] erofs-utils: dump: print more superblock fields

2023-08-08 Thread Gao Xiang
created: Wed Jun 7 17:15:44 2023 Filesystem features: sb_csum mtime 0padding compr_cfgs big_pcluster Filesystem UUID: not available Signed-off-by: Guo Xuenan Signed-off-by: Gao Xiang --- no change. dump/main.c

[PATCH 2/2] erofs-utils: dump: print more superblock fields

2023-08-07 Thread Gao Xiang
created: Wed Jun 7 17:15:44 2023 Filesystem features: sb_csum mtime 0padding compr_cfgs big_pcluster Filesystem UUID: not available Signed-off-by: Guo Xuenan Signed-off-by: Gao Xiang --- dump/main.c

[PATCH 1/2] erofs-utils: lib: add a way to request supported algorithms

2023-08-07 Thread Gao Xiang
dump.erofs needs to print supported algorithms instead of available compressors. In addition, clean up erofs_get_compress_algorithm_id() too. Cc: Guo Xuenan Signed-off-by: Gao Xiang --- include/erofs/compress.h| 1 + lib/compress.c | 33 --- lib/compressor.c

[GIT PULL] erofs fixes for 6.5-rc5

2023-08-03 Thread Gao Xiang
and no potential merge conflict is observed. Thanks, Gao Xiang The following changes since commit 5d0c230f1de8c7515b6567d9afba1f196fb4e2f4: Linux 6.5-rc4 (2023-07-30 13:23:47 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git tags/erofs

[PATCH] erofs-utils: dump: use a new subdir context for erofs_get_pathname()

2023-08-02 Thread Gao Xiang
It's absolutely unsafe to reuse struct erofs_dir_context. Also, we'd like to refactor erofs_get_pathname() in the future. Signed-off-by: Gao Xiang --- lib/dir.c | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/lib/dir.c b/lib/dir.c index e8df9f7..fff0bc0

[PATCH v2 RESEND] erofs-utils: generate preallocated extents for tarerofs

2023-08-01 Thread Gao Xiang
It is also useful to fill data blocks from tarballs at runtime in order to implement image lazy pulling. Let's generate an additional mapfile for such use cases: mkfs.erofs --tar=0,MAPFILE IMAGE TARBALL Signed-off-by: Gao Xiang --- sent out a wrong version. include/erofs/blobchunk.h | 7

[PATCH v2] erofs-utils: generate preallocated extents for tarerofs

2023-08-01 Thread Gao Xiang
It is also useful to fill data blocks from tarballs at runtime in order to implement image lazy pulling. Let's generate an additional mapfile for such use cases: mkfs.erofs --tar=0,MAPFILE IMAGE TARBALL Signed-off-by: Gao Xiang --- changes since v1: - free inode->chunkindexes on the er

Re: [PATCH] erofs-utils: generate preallocated extents for tarerofs

2023-07-31 Thread Gao Xiang
On 2023/7/31 12:51, Jingbo Xu wrote: On 7/29/23 9:32 PM, Gao Xiang wrote: @@ -795,7 +798,8 @@ int main(int argc, char **argv) return 1; } - if (cfg.block_list_file && erofs_droid_blocklist_fopen() < 0) { + if (cfg.bloc

Re: [PATCH] erofs-utils: generate preallocated extents for tarerofs

2023-07-31 Thread Gao Xiang
On 2023/8/1 10:52, Jingbo Xu wrote: On 7/29/23 9:32 PM, Gao Xiang wrote: -int erofs_blob_remap(struct erofs_sb_info *sbi) +int tarerofs_write_chunk_data(struct erofs_inode *inode, erofs_off_t data_offset) +{ + struct erofs_sb_info *sbi = inode->sbi; + unsigned int chunkb

[PATCH] erofs: drop unnecessary WARN_ON() in erofs_kill_sb()

2023-07-31 Thread Gao Xiang
-by: Gao Xiang --- will upstream this commit later this week after it lands -next. fs/erofs/super.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/erofs/super.c b/fs/erofs/super.c index 9d6a3c6158bd..566f68ddfa36 100644 --- a/fs/erofs/super.c +++ b/fs/erofs/super.c @@ -889,8 +889,6

Re: [syzbot] [erofs?] [fat?] WARNING in erofs_kill_sb

2023-07-31 Thread Gao Xiang
On 2023/7/31 20:43, Christian Brauner wrote: On Mon, Jul 31, 2023 at 01:16:22PM +0200, Christoph Hellwig wrote: On Mon, Jul 31, 2023 at 06:58:14PM +0800, Gao Xiang wrote: Previously, deactivate_locked_super() or .kill_sb() will only be called after fill_super is called, and .s_magic

Re: [syzbot] [erofs?] [fat?] WARNING in erofs_kill_sb

2023-07-31 Thread Gao Xiang
r() will also be called if setup_bdev_super() is falled. I'd suggest that removing this WARN_ON() in the related commit, or as a following commit of the related branch of the pull request if possible. Thanks, Gao Xiang

Re: [PATCH v2] erofs: deprecate superblock checksum feature

2023-07-30 Thread Gao Xiang
On 2023/7/30 22:49, Thomas Weißschuh wrote: On 2023-07-30 22:37:19+0800, Gao Xiang wrote: On 2023/7/30 22:28, Thomas Weißschuh wrote: On 2023-07-30 22:01:11+0800, Gao Xiang wrote: On 2023/7/30 21:31, Thomas Weißschuh wrote: On 2023-07-17 19:27:03+0800, Jingbo Xu wrote: Later we're going

Re: [PATCH v2] erofs: deprecate superblock checksum feature

2023-07-30 Thread Gao Xiang
On 2023/7/30 22:28, Thomas Weißschuh wrote: Hi Gao! On 2023-07-30 22:01:11+0800, Gao Xiang wrote: On 2023/7/30 21:31, Thomas Weißschuh wrote: On 2023-07-17 19:27:03+0800, Jingbo Xu wrote: Later we're going to try the self-contained image verification. The current superblock checksum

[PATCH v5] erofs: DEFLATE compression support

2023-07-30 Thread Gao Xiang
ernatively, DEFLATE could still be used for some specific files since EROFS supports multiple compression algorithms in one image. [1] https://developer.apple.com/documentation/compression/compression_algorithm Signed-off-by: Gao Xiang --- changes since v4: - add a missing kfree(strm) in z_erofs_de

Re: [PATCH v4] erofs: DEFLATE compression support

2023-07-30 Thread Gao Xiang
On 2023/7/30 22:05, Gao Xiang wrote: On 2023/7/30 22:01, Chao Yu wrote: On 2023/7/30 21:49, Gao Xiang wrote: On 2023/7/30 20:53, Chao Yu wrote: On 2023/7/16 17:19, Gao Xiang wrote: Add DEFLATE compression as the 3rd supported algorithm. DEFLATE is a popular generic-purpose

Re: [PATCH v4] erofs: DEFLATE compression support

2023-07-30 Thread Gao Xiang
On 2023/7/30 22:01, Chao Yu wrote: On 2023/7/30 21:49, Gao Xiang wrote: On 2023/7/30 20:53, Chao Yu wrote: On 2023/7/16 17:19, Gao Xiang wrote: Add DEFLATE compression as the 3rd supported algorithm. DEFLATE is a popular generic-purpose compression algorithm for quite long time (many

Re: [PATCH v2] erofs: deprecate superblock checksum feature

2023-07-30 Thread Gao Xiang
), there could be two hard-depended hashing algorithms, this increases more dependency especially for embededed devices. Thanks, Gao Xiang

Re: [PATCH v4] erofs: DEFLATE compression support

2023-07-30 Thread Gao Xiang
On 2023/7/30 20:53, Chao Yu wrote: On 2023/7/16 17:19, Gao Xiang wrote: Add DEFLATE compression as the 3rd supported algorithm. DEFLATE is a popular generic-purpose compression algorithm for quite long time (many advanced formats like gzip, zlib, zip, png are all based on that) as Apple

[PATCH] erofs-utils: generate preallocated extents for tarerofs

2023-07-29 Thread Gao Xiang
It is also useful to fill data blocks from tarballs at runtime in order to implement image lazy pulling. Let's generate an additional mapfile for such use cases: mkfs.erofs --tar=0,MAPFILE IMAGE TARBALL Signed-off-by: Gao Xiang --- include/erofs/blobchunk.h | 7 +- include/erofs

Re: [PATCH 9/9] erofs-utils: introduce union mode

2023-07-27 Thread Gao Xiang
return PTR_ERR(bh); + + ret = erofs_mapbh(bh->block); + DBG_BUGON(ret < 0); + + sbi->data_blkaddr = erofs_blknr(sbi, erofs_btell(bh, false)); + erofs_dbg("data blkaddr %d, blks %d", sbi->data_blkaddr, sbi->data_blks); + bh->op = _drop_directly_bhops; + erofs_bdrop(bh, false); + return 0; +} diff --git a/mkfs/Makefile.am b/mkfs/Makefile.am index 603c2f3..dd75485 100644 --- a/mkfs/Makefile.am +++ b/mkfs/Makefile.am @@ -6,4 +6,5 @@ AM_CPPFLAGS = ${libselinux_CFLAGS} mkfs_erofs_SOURCES = main.c mkfs_erofs_CFLAGS = -Wall -I$(top_srcdir)/include mkfs_erofs_LDADD = $(top_builddir)/lib/liberofs.la ${libselinux_LIBS} \ - ${libuuid_LIBS} ${liblz4_LIBS} ${liblzma_LIBS} ${libdeflate_LIBS} + ${libuuid_LIBS} ${liblz4_LIBS} ${liblzma_LIBS} ${zlib_LIBS} \ + ${libdeflate_LIBS} diff --git a/mkfs/main.c b/mkfs/main.c index bc5ed87..4a47b24 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -26,6 +26,7 @@ #include "erofs/compress_hints.h" #include "erofs/blobchunk.h" #include "erofs/fragments.h" +#include "erofs/union.h" #include "../lib/liberofs_private.h" #include "../lib/liberofs_uuid.h" static bool rebuild_mode. ? Thanks, Gao Xiang

Re: [PATCH 6/9] erofs-utils: add inode hash helper

2023-07-27 Thread Gao Xiang
On 2023/7/27 14:55, Jingbo Xu wrote: On 7/27/23 2:12 PM, Gao Xiang wrote: On 2023/7/27 12:57, Jingbo Xu wrote: Add erofs_insert_ihash() helper inserting inode into inode hash table, and erofs_cleanup_ihash() helper cleaning up inode hash table. Also add prototypes of erofs_iget

Re: [PATCH 2/9] erofs-utils: initialize i_nlink to 2 in erofs_init_empty_dir()

2023-07-27 Thread Gao Xiang
On 2023/7/27 12:57, Jingbo Xu wrote: Set dir->i_nlink to 2 as "." and ".." are allocated. tarerofs_init_empty_dir() is removed then. Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 6/9] erofs-utils: add inode hash helper

2023-07-27 Thread Gao Xiang
Why are there still inodes here? I think we should just check: DBG_BUGON(!list_empty(_hashtable[i])); Thanks, Gao Xiang

Re: [PATCH] erofs-utils: fix compilation error due to const static init

2023-07-25 Thread Gao Xiang
. Signed-off-by: Li Yiyan Thanks, applied. Thanks, Gao Xiang

Re: [PATCH 3/3] erofs-utils: tests: enable test cases for tarfs mode

2023-07-25 Thread Gao Xiang
On 2023/7/25 14:00, Jingbo Xu wrote: Now tarfs mode could be tested with "make check FSTYP=erofstar". I think we need another option in addition to FSTYP.

[PATCH 3/3] erofs-utils: merge consecutive chunks if possible

2023-07-24 Thread Gao Xiang
Since EROFS chunk size can be configured on a per-file basis, let's generate indexes with the best chunk size. Signed-off-by: Gao Xiang --- include/erofs/defs.h | 5 + lib/blobchunk.c | 53 2 files changed, 54 insertions(+), 4 deletions

[PATCH 1/3] erofs-utils: lib: fix improper alignment for chunked sparse files

2023-07-24 Thread Gao Xiang
File position should be fixed to aligned with the chunk boundary. Otherwise, incorrect data could be read. Fixes: 7c49e8b195ad ("erofs-utils: support chunk-based sparse files") Signed-off-by: Gao Xiang --- lib/blobchunk.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-)

[PATCH 2/3] erofs-utils: lib: tidy up erofs_blob_getchunk()

2023-07-24 Thread Gao Xiang
Mainly get rid of memory allocation on each chunk. Signed-off-by: Gao Xiang --- lib/blobchunk.c | 92 +++-- 1 file changed, 51 insertions(+), 41 deletions(-) diff --git a/lib/blobchunk.c b/lib/blobchunk.c index c0df2f7..4619057 100644 --- a/lib

Re: [PATCH v6 2/2] erofs: boost negative xattr lookup with bloom filter

2023-07-23 Thread Gao Xiang
flag is non-zero. Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH v2] erofs-utils: lib: fix erofs_iterate_dir() recursion

2023-07-22 Thread Gao Xiang
On Sat, Jul 22, 2023 at 03:04:23PM +0800, Jingbo Xu wrote: > > > On 7/22/23 2:20 PM, Gao Xiang wrote: > > Hi Jingbo, > > > > On Sat, Jul 22, 2023 at 01:40:09PM +0800, Jingbo Xu wrote: > >> ctx->dir may have changed when ctx is reused along erofs_iterate

Re: [PATCH v5 2/2] erofs: boost negative xattr lookup with bloom filter

2023-07-22 Thread Gao Xiang
se (u)intxx_t in the native kernel code. Thanks, Gao Xiang

Re: [PATCH v5 2/2] erofs: boost negative xattr lookup with bloom filter

2023-07-22 Thread Gao Xiang
; vi->xattr_shared_xattrs = kmalloc_array(vi->xattr_shared_count, > sizeof(uint), GFP_KERNEL); > @@ -392,7 +394,10 @@ int erofs_getxattr(struct inode *inode, int index, const > char *name, > void *buffer, size_t buffer_size) > { > int ret; > + uint32_t hashbit; Why using `uint32_t` here rather than `unsigned int`? We don't use `uint32_t` in the kernel codebase. Thanks, Gao Xiang

[PATCH] AOSP: erofs-utils: mkfs: fix block list support for chunked files

2023-07-20 Thread Gao Xiang
files") Signed-off-by: Gao Xiang --- lib/blobchunk.c | 22 +- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/lib/blobchunk.c b/lib/blobchunk.c index 56077dc..44541d5 100644 --- a/lib/blobchunk.c +++ b/lib/blobchunk.c @@ -127,7 +127,7 @@ int erofs_blob_write_chu

[PATCH stable 5.4.y] erofs: fix compact 4B support for 16k block size

2023-07-20 Thread Gao Xiang
server with 16k page size. Fixes: 152a333a5895 ("staging: erofs: add compacted compression indexes support") Link: https://lore.kernel.org/r/20230601112341.56960-1-hsiang...@linux.alibaba.com Signed-off-by: Gao Xiang --- resolve a trivial conflict. fs/erofs/zmap.c | 6 +- 1 file

[PATCH stable 5.10.y] erofs: fix compact 4B support for 16k block size

2023-07-20 Thread Gao Xiang
server with 16k page size. Fixes: 152a333a5895 ("staging: erofs: add compacted compression indexes support") Link: https://lore.kernel.org/r/20230601112341.56960-1-hsiang...@linux.alibaba.com Signed-off-by: Gao Xiang --- resolve a trivial conflict. fs/erofs/zmap.c | 6 +- 1 file

[PATCH stable 5.15.y] erofs: fix compact 4B support for 16k block size

2023-07-20 Thread Gao Xiang
server with 16k page size. Fixes: 152a333a5895 ("staging: erofs: add compacted compression indexes support") Link: https://lore.kernel.org/r/20230601112341.56960-1-hsiang...@linux.alibaba.com Signed-off-by: Gao Xiang --- resolve a trivial conflict. fs/erofs/zmap.c | 6 +- 1 file

[PATCH v3 2/2] erofs-utils: lib: avoid global sbi dependencies (take 1)

2023-07-20 Thread Gao Xiang
, as there's still only one erofs output image. The device context is now folded into sbi. Global sbi still exists but it is only used by binaries directly, e.g. mkfs/dump/fsck. Signed-off-by: Jingbo Xu Signed-off-by: Gao Xiang --- I plan to apply this patch. dump/main.c| 28

Re: [PATCH v4 2/2] erofs: boost negative xattr lookup with bloom filter

2023-07-19 Thread Gao Xiang
@@ -392,7 +394,10 @@ int erofs_getxattr(struct inode *inode, int index, const char *name, void *buffer, size_t buffer_size) { int ret; + uint32_t bit; uint32_t hashbit; ? Thanks, Gao Xiang

[PATCH] erofs: fix wrong primary bvec selection on deduplicated extents

2023-07-19 Thread Gao Xiang
: 5c2a64252c5d ("erofs: introduce partial-referenced pclusters") Signed-off-by: Gao Xiang --- fs/erofs/zdata.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c index b69d89a11dd0..de4f12152b62 100644 --- a/fs/erofs/zdata.c +++ b/fs/ero

Re: [PATCH 3/4] erofs-utils: inline vle_compressmeta_capacity()

2023-07-18 Thread Gao Xiang
On 2023/7/18 13:21, Jingbo Xu wrote: Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang --- lib/compress.c | 12 +++- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/lib/compress.c b/lib/compress.c index 6fb63cb..a871322 100644 --- a/lib

Re: [PATCH 2/4] erofs-utils: lib: fix recursive erofs_iterate_dir()

2023-07-18 Thread Gao Xiang
On 2023/7/18 13:20, Jingbo Xu wrote: ctx->dir may have changed when ctx is reused along erofs_iterate_dir() recursion. Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 1/4] erofs-utils: simplify iloc()

2023-07-18 Thread Gao Xiang
On 2023/7/18 13:20, Jingbo Xu wrote: Signed-off-by: Jingbo Xu Could you please keep in sync with the kernel commit b780d3fc6107464dcc43631a6208c43b6421f1e6. I mean, use the name "erofs_iloc()" instead. Thanks, Gao Xiang --- fsck/main.c | 2 +- include/erofs/

Re: [PATCH v2] erofs: deprecate superblock checksum feature

2023-07-17 Thread Gao Xiang
Xiang Thanks, Gao Xiang

[PATCH] erofs-utils: lib: support GNUTYPE_LONGNAME for tarerofs

2023-07-17 Thread Gao Xiang
), followed by the appropriate number of data blocks with the entry data. In the header for this series, the name will be truncated to the 1st 100 characters of the actual name. Cc: Jingbo Xu Signed-off-by: Gao Xiang --- lib/tar.c | 8 1 file changed, 8 insertions(+) diff --git a/lib

[PATCH v4] erofs: DEFLATE compression support

2023-07-16 Thread Gao Xiang
Add DEFLATE compression as the 3rd supported algorithm. DEFLATE is a popular generic-purpose compression algorithm for quite long time (many advanced formats like gzip, zlib, zip, png are all based on that) as Apple documentation written "If you require interoperability with non-Apple devices,

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-14 Thread Gao Xiang
On 2023/7/14 21:42, Joel Fernandes wrote: On Thu, Jul 13, 2023 at 11:17 PM Gao Xiang wrote: On 2023/7/14 10:16, Paul E. McKenney wrote: On Thu, Jul 13, 2023 at 09:33:35AM -0700, Paul E. McKenney wrote: On Thu, Jul 13, 2023 at 11:33:24AM -0400, Joel Fernandes wrote: ... >From w

Re: [PATCH v3 0/4] erofs-utils: add DEFLATE compression support

2023-07-14 Thread Gao Xiang
On 2023/7/10 19:02, Gao Xiang wrote: Hi folks, This is the EROFS DEFLATE support which I've been working on these months, mainly implementing DEFLATE fixed-sized output approach. Note that it's still some room to improve but I introduce it just to avoid external outdated zlib dependencies

Re: [PATCH v2 REBASED] erofs-utils: introduce tarerofs

2023-07-14 Thread Gao Xiang
On 2023/7/14 14:58, Jingbo Xu wrote: From: Gao Xiang Let's try to add a new mode "tarerofs" for mkfs.erofs. It mainly aims at two use cases: - Convert a tarball (or later tarballs with a merged view) into a full EROFS image [--tar=f]; - Generate an EROFS manifest imag

Re: [PATCH v2 2/2] erofs-utils: introduce tarerofs

2023-07-14 Thread Gao Xiang
On 2023/7/13 20:00, Jingbo Xu wrote: From: Gao Xiang Let's try to add a new mode "tarerofs" for mkfs.erofs. It mainly aims at two use cases: - Convert a tarball (or later tarballs with a merged view) into a full EROFS image [--tar=f]; - Generate an EROFS manifest imag

Re: [PATCH] erofs: deprecate superblock checksum feature

2023-07-13 Thread Gao Xiang
On 2023/7/14 12:50, Jingbo Xu wrote: On 7/14/23 12:29 PM, Gao Xiang wrote: On 2023/7/14 11:38, Jingbo Xu wrote: Later we're going to introduce fs-verity based verification for the whole image.  Make the superblock checksum feature deprecated. I'd suggest that "Later we're

Re: [PATCH] erofs: deprecate superblock checksum feature

2023-07-13 Thread Gao Xiang
On 2023/7/14 11:38, Jingbo Xu wrote: Later we're going to introduce fs-verity based verification for the whole image. Make the superblock checksum feature deprecated. I'd suggest that "Later we're going to try the self-contained image verification. The current superblock checksum

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-13 Thread Gao Xiang
ore we proceed, I also think it'd be better to get some performance numbers first for this (e.g. with dm-verity) and record the numbers in the commit message to justify this. Otherwise, I guess the same question will be raised again and again. Thanks, Gao Xiang

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-13 Thread Gao Xiang
all downstream users which use dm-verity will be impacted and individual end users will not be happy as well. Thanks, Gao Xiang

[GIT PULL] erofs fixes for 6.5-rc2

2023-07-13 Thread Gao Xiang
they are simple enough to be addressed in this cycle. All commits have been in -next for a while and no potential merge conflict is observed. Thanks, Gao Xiang The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5: Linux 6.5-rc1 (2023-07-09 13:53:13 -0700) are available

[PATCH] erofs-utils: add github issue/pull-request templates

2023-07-13 Thread Gao Xiang
EROFS Development is currently on the mailing list _only_. Signed-off-by: Gao Xiang --- .github/ISSUE_TEMPLATE.txt| 9 + .github/PULL_REQUEST_TEMPLATE.txt | 9 + 2 files changed, 18 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE.txt create mode 100644 .github

Re: [PATCH 2/2] erofs-utils: introduce tarerofs

2023-07-13 Thread Gao Xiang
On 2023/7/11 14:12, Jingbo Xu wrote: From: Gao Xiang Let's try to add a new mode "tarerofs" for mkfs.erofs. It mainly aims at two use cases: - Convert a tarball (or later tarballs with a merged view) into a full EROFS image [--tar=f]; - Generate an EROFS manifest imag

Re: [PATCH v3 3/3] erofs-utils: mkfs: enable xattr name filter

2023-07-12 Thread Gao Xiang
On 2023/7/12 20:13, Jingbo Xu wrote: Introduce "--xattr-filter" option to enable the xattr name bloom filter feature. we might need to switch it to -Exattr-name-filter rather than introduce another new option. Thanks, Gao Xiang

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-12 Thread Gao Xiang
On 2023/7/13 12:52, Paul E. McKenney wrote: On Thu, Jul 13, 2023 at 12:41:09PM +0800, Gao Xiang wrote: ... There are lots of performance issues here and even a plumber topic last year to show that, see: [1] https://lore.kernel.org/r/20230519001709.2563-1...@kernel.org [2] https

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-12 Thread Gao Xiang
On 2023/7/13 12:27, Paul E. McKenney wrote: On Thu, Jul 13, 2023 at 10:02:17AM +0800, Gao Xiang wrote: ... sorry forget some. One additional question... What is your plan for kernels built with CONFIG_PREEMPT_COUNT=n? After all, in such kernels, there is no way that I know

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-12 Thread Gao Xiang
On 2023/7/13 12:27, Paul E. McKenney wrote: On Thu, Jul 13, 2023 at 10:02:17AM +0800, Gao Xiang wrote: On 2023/7/13 08:32, Joel Fernandes wrote: On Wed, Jul 12, 2023 at 02:20:56PM -0700, Sandeep Dhavale wrote: [..] As such this patch looks correct to me, one thing I noticed is that you

Re: [PATCH v3 1/2] erofs: update on-disk format for xattr name filter

2023-07-12 Thread Gao Xiang
may change in the future. With this flag we don't need bothering these compatible bits again at that time. Suggested-by: Alexander Larsson Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH v3 2/2] erofs: boost negative xattr lookup with bloom filter

2023-07-12 Thread Gao Xiang
bit = xxh32(name, strlen(name), EROFS_XATTR_FILTER_SEED + index); should we enable xxh32 by using CONFIG_XXHASH? Thanks, Gao Xiang

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-12 Thread Gao Xiang
On 2023/7/13 10:02, Gao Xiang wrote: On 2023/7/13 08:32, Joel Fernandes wrote: On Wed, Jul 12, 2023 at 02:20:56PM -0700, Sandeep Dhavale wrote: [..] As such this patch looks correct to me, one thing I noticed is that you can check rcu_is_watching() like the lockdep-enabled code does

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-12 Thread Gao Xiang
_dispatch_ops(). But as the commit shown, we only need to trigger a workqueue for case 1) and 3) due to performance reasons. Hopefully I show it more clear. Thanks, Gao Xiang

[PATCH v3] erofs: DEFLATE compression support

2023-07-12 Thread Gao Xiang
ernatively, DEFLATE could still be used for some specific files since EROFS supports multiple compression algorithms in one image. [1] https://developer.apple.com/documentation/compression/compression_algorithm Signed-off-by: Gao Xiang --- change since v2: - fix typo "whileas" --> "whi

Re: [PATCH v2] erofs: DEFLATE compression support

2023-07-12 Thread Gao Xiang
Hi Randy, On 2023/7/13 08:01, Randy Dunlap wrote: On 7/12/23 16:46, Gao Xiang wrote: diff --git a/fs/erofs/Kconfig b/fs/erofs/Kconfig index f259d92c9720..d7b5327215f0 100644 --- a/fs/erofs/Kconfig +++ b/fs/erofs/Kconfig @@ -99,6 +99,21 @@ config EROFS_FS_ZIP_LZMA If unsure, say N

Re: [PATCH RESEND] erofs: DEFLATE compression support

2023-07-12 Thread Gao Xiang
Hi Randy, On 2023/7/13 07:46, Randy Dunlap wrote: Hi-- On 7/12/23 16:33, Gao Xiang wrote: diff --git a/fs/erofs/Kconfig b/fs/erofs/Kconfig index f259d92c9720..d7b5327215f0 100644 --- a/fs/erofs/Kconfig +++ b/fs/erofs/Kconfig @@ -99,6 +99,21 @@ config EROFS_FS_ZIP_LZMA If unsure, say N

[PATCH v4 2/4] erofs-utils: fuse,fsck: add DEFLATE algorithm support

2023-07-12 Thread Gao Xiang
This patch adds DEFLATE compression algorithm support to erofsfuse by using zlib (by default) and libdeflate. libdeflate will be used instead of zlib if libdeflate is enabled. Signed-off-by: Gao Xiang --- change since v3: - fix improper Z_EROFS_COMPRESSION_DEFLATE number. configure.ac

[PATCH v2] erofs: DEFLATE compression support

2023-07-12 Thread Gao Xiang
ernatively, DEFLATE could still be used for some specific files since EROFS supports multiple compression algorithms in one image. [1] https://developer.apple.com/documentation/compression/compression_algorithm Signed-off-by: Gao Xiang --- v2: - a small fix, should always zlib_inflateEnd(). fs/ero

[PATCH RESEND] erofs: DEFLATE compression support

2023-07-12 Thread Gao Xiang
ernatively, DEFLATE could still be used for some specific files since EROFS supports multiple compression algorithms in one image. [1] https://developer.apple.com/documentation/compression/compression_algorithm Signed-off-by: Gao Xiang --- fs/erofs/Kconfig| 15 ++ fs/erof

[PATCH] erofs: DEFLATE compression support

2023-07-12 Thread Gao Xiang
ernatively, DEFLATE could still be used for some specific files since EROFS supports multiple compression algorithms in one image. Signed-off-by: Gao Xiang --- fs/erofs/Kconfig| 15 ++ fs/erofs/Makefile | 1 + fs/erofs/compress.h | 2 + fs/erofs/deco

Re: [PATCH] erofs: fix fsdax unavailability for chunk-based regular files

2023-07-11 Thread Gao Xiang
erofs: support reading chunk-based uncompressed files") Signed-off-by: Xin Yin Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH] erofs-utils: add ERR_CAST macro

2023-07-10 Thread Gao Xiang
On 2023/7/11 11:55, Jingbo Xu wrote: Add ERR_CAST macro. Signed-off-by: Jingbo Xu I'd like to apply it with the following commit message: "Add ERR_CAST macro to prepare for the upcoming tarerofs feature." Thanks, Gao Xiang

Re: [PATCH] erofs-utils: zmap: fix small compressed files inlining

2023-07-10 Thread Gao Xiang
h for uncompressed pclusters") Signed-off-by: Yue Hu Thanks, applied. Thanks, Gao Xiang

Re: [PATCH] erofs: enable dax for chunk based regular file

2023-07-10 Thread Gao Xiang
really think it's a regression honestly, how about updating the subject line to: "erofs: fix fsdax unavailability for chunk-based regular files" Fixes: c5aa903a59db ("erofs: support reading chunk-based uncompressed files") Thanks, Gao Xiang

[PATCH v3 4/4] erofs-utils: mkfs: add libdeflate compressor support

2023-07-10 Thread Gao Xiang
enwik8.libdeflate.9.z Anyway, let's use the current APIs for users who needs smaller image sizes for now. Besides, EROFS also supports multiple per-file algorithms in one image, so it can be used for specific files as well. Suggested-by: Eric Biggers Signed-off-by: Gao Xiang --- co

[PATCH v3 3/4] erofs-utils: mkfs: add DEFLATE algorithm support

2023-07-10 Thread Gao Xiang
generated with large windowbits (e.g. > 12 for the IAA accelerator) doesn't seem to be worked properly on those. Signed-off-by: Gao Xiang --- lib/Makefile.am | 2 +- lib/compress.c | 24 + lib/compressor.c | 1 + lib/compressor.h | 1 +

[PATCH v3 2/4] erofs-utils: fuse,fsck: add DEFLATE algorithm support

2023-07-10 Thread Gao Xiang
This patch adds DEFLATE compression algorithm support to erofsfuse by using zlib (by default) and libdeflate. libdeflate will be used instead of zlib if libdeflate is enabled. Signed-off-by: Gao Xiang --- configure.ac | 45 ++ dump/Makefile.am | 2 +- fsck/Makefile.am

[PATCH v3 1/4] erofs-utils: add a built-in DEFLATE compressor

2023-07-10 Thread Gao Xiang
ml/rfc1951 Signed-off-by: Gao Xiang --- lib/Makefile.am|2 + lib/kite_deflate.c | 1270 2 files changed, 1272 insertions(+) create mode 100644 lib/kite_deflate.c diff --git a/lib/Makefile.am b/lib/Makefile.am index e243c1c..b729098 100644

<    1   2   3   4   5   6   7   8   9   10   >