[PATCH] erofs-utils: fix incorrect i_nlink in the unified rebuild logic

2024-06-12 Thread Gao Xiang
Fixes: 203c847cc7d1 ("erofs-utils: unify the tree traversal for the rebuild mode") Closes: https://github.com/erofs/erofsnightly/actions/runs/9492427961/job/26159566596 Signed-off-by: Gao Xiang --- lib/inode.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib

[PATCH 2/5] erofs-utils: lib: use filesystem UUID if the device name is not specified

2024-06-12 Thread Gao Xiang
The device name is not always valid. Signed-off-by: Gao Xiang --- lib/rebuild.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/lib/rebuild.c b/lib/rebuild.c index 618067a..c25d222 100644 --- a/lib/rebuild.c +++ b/lib/rebuild.c @@ -15,6 +15,7 @@ #include

[PATCH 5/5] erofs-utils: lib: get rid of global sbi in lib/inode.c

2024-06-12 Thread Gao Xiang
In order to prepare for incremental builds. Signed-off-by: Gao Xiang --- include/erofs/inode.h | 8 +--- lib/fragments.c | 2 +- lib/inode.c | 47 +++ lib/rebuild.c | 4 ++-- lib/tar.c | 2 +- mkfs/main.c

[PATCH 4/5] erofs-utils: mkfs: assign root NID in the main thread

2024-06-12 Thread Gao Xiang
Thus it could be customized (skiped), especially for incremental builds. Signed-off-by: Gao Xiang --- lib/inode.c | 36 ++-- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/lib/inode.c b/lib/inode.c index 09bf76b..069484d 100644 --- a/lib/inode.c

[PATCH 1/5] erofs-utils: lib: get rid of erofs_prepare_dir_layout()

2024-06-12 Thread Gao Xiang
Just open-code the previous erofs_prepare_dir_file() and rename `erofs_prepare_dir_layout()` to `erofs_prepare_dir_file()`. No logic changes. Signed-off-by: Gao Xiang --- lib/inode.c | 70 - 1 file changed, 31 insertions(+), 39 deletions

[PATCH 3/5] erofs-utils: wrap up superblock reservation for incremental builds

2024-06-12 Thread Gao Xiang
Refactor `erofs_buffer_init()` to wrap up necessary operations for full builds. Introduce another `erofs_buffer_init()` to specify start block address for the upcoming incremental builds. Signed-off-by: Gao Xiang --- include/erofs/cache.h| 2 +- include/erofs/internal.h | 1 + lib

Re: [PATCH 6.9.y] erofs: avoid allocating DEFLATE streams before mounting

2024-06-12 Thread Gao Xiang
On 2024/6/12 20:54, Greg Kroah-Hartman wrote: On Tue, Jun 04, 2024 at 08:33:05PM +0800, Gao Xiang wrote: Hi Greg, ping? Do these backport fixes miss the 6.6, 6.8, 6.9 queues.. Sorry for the delay, all now queued up. well, except for 6.8.y, that branch is now end-of-life, sorry. Thanks

Re: [PATCH v4] erofs-utils: add I/O control for tarerofs stream via `erofs_vfile`

2024-06-11 Thread Gao Xiang
ad(struct erofs_sb_info *sbi, int device_id, void *buf, u64 offset, size_t len) Same here. I will fix manually, but please take care of alignment next time. Thanks, Gao Xiang { if (device_id) @@ -420,3 +420,38 @@ out: #endif return __erofs_copy_file_ran

Re: [PATCH] erofs-utils: add I/O control for tarerofs stream via `erofs_vfile`

2024-06-11 Thread Gao Xiang
t;failed to read : %s\n", erofs_err("failed to read: %s", Otherwise it looks good to me, Thanks, Gao Xiang

[PATCH] erofs-utils: fix the current rebuild mode

2024-06-11 Thread Gao Xiang
`inode->with_diskbuf` can be false in the rebuild mode since inode data has been mapped before. Fixes: 203c847cc7d1 ("erofs-utils: unify the tree traversal for the rebuild mode") Signed-off-by: Gao Xiang --- lib/inode.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-

Re: [PATCH v2] erofs-utils: add I/O control for tarerofs stream via `erofs_vfile`

2024-06-11 Thread Gao Xiang
into erofs_io_read(). Otherwise it looks good to me. Thanks, Gao Xiang +{ + s64 i = 0; + + while (bytes) { + int len = bytes > INT_MAX ? INT_MAX : bytes; + int ret; + + ret = read(fd, buf + i, len); + if (ret

Re: [PATCH v2] build: support building static library liberofsfuse

2024-06-11 Thread Gao Xiang
} ${libzstd_LIBS} ${libqpl_LIBS} +liberofsfuse_a_LIBADD = $(top_builddir)/lib/liberofs.la Since we may not have all dependent libraries, so it's up to users to use liberofsfuse.a and link static (or dynamic) dependencies themselves. I will update this patch manually. Thanks, Gao Xiang +endif

Re: [PATCH] erofs-utils: add I/O control for tarerofs stream via `erofs_vfile`

2024-06-11 Thread Gao Xiang
s->sz && likely(ios->dumpfd < 0) && !ios->vf.ops) { + s64 cur = lseek(ios->vf.fd, sz, SEEK_CUR); Same here, erofs_io_lseek is needed. Thanks, Gao Xiang

Re: Can mkfs.erofs keep specified files uncompressed?

2024-06-10 Thread Gao Xiang
press" in the hint file. I think it's already supported by specifying pclusterblks as 0 to leave these files uncompressed like: 0 0 .* You could try if it works for your use cases.. Thanks, Gao Xiang [1]: https://github.com/erofs/erofs-utils/commit/dac31f7eb228601c457c6338e6df8dabfcbb

[PATCH 1/2] erofs-utils: move erofs_writesb() into lib/

2024-06-07 Thread Gao Xiang
So that external programs can directly use it. Signed-off-by: Gao Xiang --- include/erofs/internal.h | 4 +++ lib/super.c | 52 mkfs/main.c | 64 +++- 3 files changed, 60 insertions(+), 60 deletions

[PATCH 2/2] erofs-utils: lib: split erofs_iflush()

2024-06-07 Thread Gao Xiang
So that external programs can directly use it. Signed-off-by: Gao Xiang --- include/erofs/inode.h | 1 + lib/inode.c | 25 - 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/include/erofs/inode.h b/include/erofs/inode.h index 5d6bc98..46d989c

Re: [PATCH] erofs-utils: support virtual files

2024-06-07 Thread Gao Xiang
On 2024/6/6 19:18, Hongzhen Luo wrote: The current erofs-utils I/O implementation is through file descriptors. The new `erofs_vfile` provides a more flexible way to perform I/Os. Signed-off-by: Hongzhen Luo Applied with minor fixes. Thanks, Gao Xiang

Re: [PATCH] build: export liberofs

2024-06-07 Thread Gao Xiang
and static library in the future (v1.9), so liberofs.la is preferred. Thanks, Gao Xiang

Re: [PATCH] erofs-utils: introduce erofs_vfile

2024-06-06 Thread Gao Xiang
version instead. Thanks, Gao Xiang

Re: [PATCH] erofs: support external crypto for decompression

2024-06-06 Thread Gao Xiang
+/* 16 bytes */ +struct z_erofs_crypto_cfgs { + char crypto_name[8]; + __le16 max_distance; + __le16 max_pclusterblks; + u8 reserved[4]; +} __packed; +#endif Same here, no needed. Thanks, Gao Xiang

Re: [PATCH v2] build: support building static library liberofsfuse

2024-06-06 Thread Gao Xiang
On 2024/6/6 15:39, ComixHe wrote: add new option '--enable-static-fuse' so that we could import erofsfuse as a static library directly into other projects Signed-off-by: ComixHe Reviewed-by: Gao Xiang Thanks, Gao Xiang --- configure.ac | 7 +++ fuse/Makefile.am | 9

Re: [PATCH] build: support building static library liberofsfuse

2024-06-06 Thread Gao Xiang
LIBS} Maybe the following way also works? lib_LIBRARIES = liberofsfuse.a liberofsfuse_a_CFLAGS = ... liberofsfuse_a_CFLAGS += -Dmain=erofsfuse_main ${libfuse2_CFLAGS} ${libfuse3_CFLAGS} ${libselinux_CFLAGS} liberofsfuse_la_LIBADD = I guess we don't need -static + la IMHO... Thanks, Gao Xiang +endif

Re: [PATCH] build: support building static library

2024-06-05 Thread Gao Xiang
On 2024/6/6 11:17, Chen Linxuan wrote: On 2024/6/6 11:13, Gao Xiang wrote: On 2024/6/6 11:06, Chen Linxuan wrote: On 2024/6/6 10:45, Gao Xiang wrote: On 2024/6/6 10:25, Chen Linxuan wrote: Hi, On 2024/6/6 10:22, Gao Xiang wrote: Hi, On 2024/6/6 10:13, Chen Linxuan wrote: Hi Xiang

Re: [PATCH] build: support building static library

2024-06-05 Thread Gao Xiang
On 2024/6/6 11:06, Chen Linxuan wrote: On 2024/6/6 10:45, Gao Xiang wrote: On 2024/6/6 10:25, Chen Linxuan wrote: Hi, On 2024/6/6 10:22, Gao Xiang wrote: Hi, On 2024/6/6 10:13, Chen Linxuan wrote: Hi Xiang! On 2024/5/23 16:05, Gao Xiang wrote: Hi Comix! On 2024/5/23 15:31, ComixHe

Re: [PATCH] build: support building static library

2024-06-05 Thread Gao Xiang
On 2024/6/6 10:25, Chen Linxuan wrote: Hi, On 2024/6/6 10:22, Gao Xiang wrote: Hi, On 2024/6/6 10:13, Chen Linxuan wrote: Hi Xiang! On 2024/5/23 16:05, Gao Xiang wrote: Hi Comix! On 2024/5/23 15:31, ComixHe wrote: In some cases, developer may need to integrate erofs-utils

Re: [PATCH] build: support building static library

2024-06-05 Thread Gao Xiang
Hi, On 2024/6/6 10:13, Chen Linxuan wrote: Hi Xiang! On 2024/5/23 16:05, Gao Xiang wrote: Hi Comix! On 2024/5/23 15:31, ComixHe wrote: In some cases, developer may need to integrate erofs-utils into their proejct as a static library to reduce package dependencies and have more finer control

Re: [PATCH] fs/erofs: fix an overflow issue of unmapped extents

2024-06-05 Thread Gao Xiang
On 2024/6/5 22:05, Jianan Huang wrote: Here the size should be `length - skip`, otherwise it could cause the destination buffer overflow. Reported-by: jianqiang wang Fixes: 65cb73057b65 ("fs/erofs: add lz4 decompression support") Signed-off-by: Jianan Huang Reviewed-by:

[PATCH v2 2/2] erofs-utils: support Intel Query Processing Library

2024-06-05 Thread Gao Xiang
ent/www/us/en/products/docs/accelerator-engines/in-memory-analytics-accelerator.html Cc: "Feghali, Wajdi K" Signed-off-by: Gao Xiang --- changes since v1: - Fix a wrong image size number in commit message; - Minor configure.ac fix. configure.ac | 37 + dum

[PATCH 1/2] erofs-utils: introduce z_erofs_parse_cfgs()

2024-06-05 Thread Gao Xiang
This userspace implementation will be mainly used for the upcoming Intel In-Memory Analytics Accelerator integration. Signed-off-by: Gao Xiang --- include/erofs/internal.h | 1 + lib/decompress.c | 41 lib/super.c | 7 +++ 3

[PATCH 2/2] erofs-utils: support Intel Query Processing Library

2024-06-05 Thread Gao Xiang
ent/www/us/en/products/docs/accelerator-engines/in-memory-analytics-accelerator.html Cc: "Feghali, Wajdi K" Signed-off-by: Gao Xiang --- configure.ac | 37 + dump/Makefile.am | 2 +- fsck/Makefile.am | 4 +- include/erofs/internal.h | 1 + lib

Re: [PATCH v3] erofs-utils: introduce the I/O manager

2024-06-05 Thread Gao Xiang
On 2024/6/5 14:33, Gao Xiang wrote: From: Hongzhen Luo Introduce the I/O manager to provide a more flexible way to specify the virtual storage. Signed-off-by: Hongzhen Luo Signed-off-by: Gao Xiang After rethinking, anyway I plan to drop this patch since it's not clean. Thanks, Gao

Re: [PATCH v3] erofs-utils: introduce the I/O manager

2024-06-05 Thread Gao Xiang
...@linux.alibaba.com/ v1: https://lore.kernel.org/all/20240604093556.3883585-1-hongz...@linux.alibaba.com/ I adjust a bit and apply the following version: https://lore.kernel.org/linux-erofs/20240605063312.3480769-1-hsiang...@linux.alibaba.com/ Thanks, Gao Xiang

[PATCH v3] erofs-utils: introduce the I/O manager

2024-06-05 Thread Gao Xiang
From: Hongzhen Luo Introduce the I/O manager to provide a more flexible way to specify the virtual storage. Signed-off-by: Hongzhen Luo Signed-off-by: Gao Xiang --- changes since v2: - rename "struct erofs_io_manager" to "struct erofs_iomgr_ops"; dump/main.c

Re: [PATCH v2] erofs-utils: introduce the I/O manager

2024-06-04 Thread Gao Xiang
(+ Cc linux-erofs) On 2024/6/5 11:32, Hongzhen Luo wrote: 在 2024/6/5 10:55, Gao Xiang 写道: On 2024/6/4 20:10, Hongzhen Luo wrote: Introduce the I/O manager to provide a more flexible way to specify the virtual storage. Signed-off-by: Hongzhen Luo --- v2: Updated the handling

Re: [PATCH v2] erofs-utils: introduce the I/O manager

2024-06-04 Thread Gao Xiang
On 2024/6/4 20:10, Hongzhen Luo wrote: Introduce the I/O manager to provide a more flexible way to specify the virtual storage. Signed-off-by: Hongzhen Luo --- v2: Updated the handling of `.dev_close` being unimplemented in `erofs_dev_close` and so on. v1:

Re: [PATCH 6.9.y] erofs: avoid allocating DEFLATE streams before mounting

2024-06-04 Thread Gao Xiang
Hi Greg, ping? Do these backport fixes miss the 6.6, 6.8, 6.9 queues.. Thanks, Gao XIang On 2024/5/30 17:21, Gao Xiang wrote: commit 80eb4f62056d6ae709bdd0636ab96ce660f494b2 upstream. Currently, each DEFLATE stream takes one 32 KiB permanent internal window buffer even if there is no running

Re: [PATCH] erofs-utils: introduce io manager

2024-06-04 Thread Gao Xiang
return -EOPNOTSUPP; if (sbi->io_manager && !sbi->io_manager->dev_fsync) return 0; return sbi->io_manager ? sbi->io_manager->dev_fsync(sbi) : builtin_dev_fsync(sbi); + + return sbi->io_manager ? sbi->io_manager->dev_fsync(sbi) : + builtin_dev_fsync(sbi); +} + +int erofs_dev_resize(struct erofs_sb_info *sbi, erofs_blk_t nblocks) +{ + if (sbi->io_manager && !sbi->io_manager->dev_resize) + return -EOPNOTSUPP; if (sbi->io_manager && !sbi->io_manager->dev_resize) return 0; ... + + return sbi->io_manager ? sbi->io_manager->dev_resize(sbi, nblocks) : + builtin_dev_resize(sbi, nblocks); +} Thanks, Gao Xiang

[PATCH 1/2] erofs-utils: lib: wrap up zeropadding calculation

2024-06-04 Thread Gao Xiang
Use a simple helper instead of open-coding. Signed-off-by: Gao Xiang --- lib/decompress.c | 46 ++ 1 file changed, 18 insertions(+), 28 deletions(-) diff --git a/lib/decompress.c b/lib/decompress.c index 58ce7e5..e65b924 100644 --- a/lib/decompress.c

[PATCH 2/2] erofs-utils: record sb_size instead of sb_extslots

2024-06-04 Thread Gao Xiang
Just follow the kernel implementation. Signed-off-by: Gao Xiang --- dump/main.c | 4 ++-- include/erofs/internal.h | 7 --- lib/super.c | 7 ++- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/dump/main.c b/dump/main.c index dd2c620..50f4662

[PATCH] erofs-utils: lib: fix incorrect xattr sharing

2024-05-31 Thread Gao Xiang
There are off-by-one issues after refactoring, and the size of kvbuf should be calculated by EROFS_XATTR_KVSIZE instead. Fixes: 5df285cf405d ("erofs-utils: lib: refactor extended attribute name prefixes") Signed-off-by: Gao Xiang --- lib/xattr.c | 14 -- 1 file

[PATCH 6.8.y] erofs: avoid allocating DEFLATE streams before mounting

2024-05-30 Thread Gao Xiang
with hundreds of CPU cores if DEFLATE is enabled but unused. Fixes: ffa09b3bd024 ("erofs: DEFLATE compression support") Cc: # 6.6+ Reviewed-by: Sandeep Dhavale Signed-off-by: Gao Xiang Link: https://lore.kernel.org/r/20240520090106.2898681-1-hsiang...@linux.alibaba.com ---

[PATCH 6.6.y] erofs: avoid allocating DEFLATE streams before mounting

2024-05-30 Thread Gao Xiang
with hundreds of CPU cores if DEFLATE is enabled but unused. Fixes: ffa09b3bd024 ("erofs: DEFLATE compression support") Cc: # 6.6+ Reviewed-by: Sandeep Dhavale Link: https://lore.kernel.org/r/20240520090106.2898681-1-hsiang...@linux.alibaba.com [ Gao Xiang: resolve trivial conflicts. ]

[PATCH 6.9.y] erofs: avoid allocating DEFLATE streams before mounting

2024-05-30 Thread Gao Xiang
with hundreds of CPU cores if DEFLATE is enabled but unused. Fixes: ffa09b3bd024 ("erofs: DEFLATE compression support") Cc: # 6.6+ Reviewed-by: Sandeep Dhavale Signed-off-by: Gao Xiang Link: https://lore.kernel.org/r/20240520090106.2898681-1-hsiang...@linux.alibaba.com ---

Re: [PATCH v3 00/12] cachefiles: some bugfixes and cleanups for ondemand requests

2024-05-29 Thread Gao Xiang
for a while so it'd be much better to address these upstream. :-) Thanks, Gao Xiang --- Applied to the vfs.fixes branch of the vfs/vfs.git tree. Patches in the vfs.fixes branch should appear in linux-next soon. Please report any outstanding bugs that were missed during review in a new review

Re: [PATCH v4 2/5] lib/lz4: update LZ4 decompressor module

2024-05-28 Thread Gao Xiang
optimization more than old version, if memcpy() assembly doesn't generate well on your platform, it might have some issue. Thanks, Gao Xiang My patch: --- a/lib/lz4.c +++ b/lib/lz4.c @@ -41,6 +41,16 @@ static FORCE_INLINE u16 LZ4_readLE16(const void *src) return get_unaligned_le16(src);

[PATCH] erofs-utils: fix false-positive errors on gcc 4.8.5

2024-05-28 Thread Gao Xiang
Just old compiler bugs. Signed-off-by: Gao Xiang --- lib/data.c | 2 +- lib/dedupe.c| 2 +- lib/fragments.c | 2 ++ 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/data.c b/lib/data.c index a87053f..c139e0c 100644 --- a/lib/data.c +++ b/lib/data.c @@ -420,7 +420,7

Re: [PATCH v4 2/5] lib/lz4: update LZ4 decompressor module

2024-05-24 Thread Gao Xiang
the old codebase was based on) and the new version mainly relies on memcpy/memmove instead of its own versions. Would you mind to check the assembly how memcpy/memset is generated on your platform? Thanks, Gao Xiang Thanks. Regards, Jonathan

[GIT PULL] erofs more updates for 6.10-rc1

2024-05-24 Thread Gao Xiang
he dependency with the detailed description. Another patch gets rid of unnecessary memory allocation out of DEFLATE decompressor. The remaining one is a trivial cleanup. All commits have been in -next and no potential merge conflict is observed. Thanks, Gao Xiang The following changes

Re: [PATCH 1/2] erofs-utils: lib: provide helper to disable hashmap shrinking

2024-05-23 Thread Gao Xiang
On 2024/5/24 05:01, Sandeep Dhavale wrote: This helper sets hasmap.shrink_at to 0. This is helpful to iterate over hashmap using hashmap_iter_next() and use hashmap_remove() in single pass efficeintly. Signed-off-by: Sandeep Dhavale Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 2/2] erofs-utils: lib: improve freeing hashmap in erofs_blob_exit()

2024-05-23 Thread Gao Xiang
yle manually, otherwise it looks good to me... Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH] build: support building static library

2024-05-23 Thread Gao Xiang
])], +[enable_static_fsck="$enableval"], +[enable_static_fsck="no"]) But how could we support static libraries from binaries? I guess you need static liberofs instead? Thanks, Gao Xiang

[PATCH] erofs-utils: lib: fix uncompressed packed inode

2024-05-22 Thread Gao Xiang
Currently, packed inode can be used in the unencoded way too such as xattr prefixes. Signed-off-by: Gao Xiang --- lib/inode.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/lib/inode.c b/lib/inode.c index cbe0810..cd48e55 100644 --- a/lib/inode.c

[PATCH 6.6.y 2/2] erofs: reliably distinguish block based and fscache mode

2024-05-21 Thread Gao Xiang
have been initialised, so use sbi->fsid to distinguish between the two modes. Signed-off-by: Christian Brauner Signed-off-by: Baokun Li Reviewed-by: Jingbo Xu Reviewed-by: Gao Xiang Reviewed-by: Chao Yu Link: https://lore.kernel.org/r/20240419123611.947084-3-libaok...@huawei.com Signed-

[PATCH 6.6.y 1/2] erofs: get rid of erofs_fs_context

2024-05-21 Thread Gao Xiang
needed, replace ctx with sbi, no functional changes. Suggested-by: Jingbo Xu Signed-off-by: Baokun Li Reviewed-by: Jingbo Xu Reviewed-by: Gao Xiang Reviewed-by: Chao Yu Link: https://lore.kernel.org/r/20240419123611.947084-2-libaok...@huawei.com [ Gao Xiang: trivial conflict due to a warning

[PATCH 6.8.y 1/2] erofs: get rid of erofs_fs_context

2024-05-21 Thread Gao Xiang
needed, replace ctx with sbi, no functional changes. Suggested-by: Jingbo Xu Signed-off-by: Baokun Li Reviewed-by: Jingbo Xu Reviewed-by: Gao Xiang Reviewed-by: Chao Yu Link: https://lore.kernel.org/r/20240419123611.947084-2-libaok...@huawei.com [ Gao Xiang: trivial conflict due to a warning

[PATCH 6.8.y 2/2] erofs: reliably distinguish block based and fscache mode

2024-05-21 Thread Gao Xiang
have been initialised, so use sbi->fsid to distinguish between the two modes. Signed-off-by: Christian Brauner Signed-off-by: Baokun Li Reviewed-by: Jingbo Xu Reviewed-by: Gao Xiang Reviewed-by: Chao Yu Link: https://lore.kernel.org/r/20240419123611.947084-3-libaok...@huawei.com Signed-

Re: [PATCH v2 4/5] cachefiles: cyclic allocation of msg_id to avoid reuse

2024-05-20 Thread Gao Xiang
On 2024/5/21 10:36, Baokun Li wrote: On 2024/5/20 22:56, Gao Xiang wrote: Hi Baokun, On 2024/5/20 21:24, Baokun Li wrote: On 2024/5/20 20:54, Gao Xiang wrote: On 2024/5/20 20:42, Baokun Li wrote: On 2024/5/20 18:04, Jeff Layton wrote: On Mon, 2024-05-20 at 12:06 +0800, Baokun Li wrote

Re: [PATCH v2 4/5] cachefiles: cyclic allocation of msg_id to avoid reuse

2024-05-20 Thread Gao Xiang
Hi Baokun, On 2024/5/20 21:24, Baokun Li wrote: On 2024/5/20 20:54, Gao Xiang wrote: On 2024/5/20 20:42, Baokun Li wrote: On 2024/5/20 18:04, Jeff Layton wrote: On Mon, 2024-05-20 at 12:06 +0800, Baokun Li wrote: Hi Jeff, Thank you very much for your review! On 2024/5/19 19:11, Jeff

Re: [PATCH v2 4/5] cachefiles: cyclic allocation of msg_id to avoid reuse

2024-05-20 Thread Gao Xiang
ill the mount instance if there is a malicious user daemon. So in that case, hung task will not be triggered anymore, and you don't need to care about cyclic allocation too. Thanks, Gao Xiang Regards, Baokun +    xas_clear_mark(, XA_FREE_MARK); +    xas_set_mark(, CACHEFILES_REQ_NEW)

Re: [PATCH v2 08/12] cachefiles: never get a new anonymous fd if ondemand_id is valid

2024-05-20 Thread Gao Xiang
u'd better to switch wait_for_completion() to wait_for_completion_killable() at least IMHO anyway. Thanks, Gao Xiang

[PATCH] erofs: avoid allocating DEFLATE streams before mounting

2024-05-20 Thread Gao Xiang
. Fixes: ffa09b3bd024 ("erofs: DEFLATE compression support") Cc: # 6.6+ Signed-off-by: Gao Xiang --- fs/erofs/decompressor_deflate.c | 55 + 1 file changed, 29 insertions(+), 26 deletions(-) diff --git a/fs/erofs/decompressor_deflate.c b/fs/erofs/decompressor

Re: [PATCH v2 03/12] cachefiles: fix slab-use-after-free in cachefiles_ondemand_get_fd()

2024-05-20 Thread Gao Xiang
later patches modify the logic here as well. Or add Fixes tag for patch 2? I think we might better to avoid unnecessary dependencies since it relies on some "AI" magic and often mis-backportes real dependencies. I tend to leave real bugfixes first, and do cleanup next. But please don't leave cleanup patches with "Fixes:" tags anyway since it just misleads people. Thanks, Gao Xiang

[PATCH v2] erofs-utils: unify the tree traversal for the rebuild mode

2024-05-20 Thread Gao Xiang
Let's drop the legacy approach and `tarerofs` will be applied too. Signed-off-by: Gao Xiang --- change since v1: - fix incorrect unencoded data reported by: https://github.com/erofs/erofsnightly/actions/runs/9153196829 include/erofs/internal.h | 7 +- lib/diskbuf.c| 14

Re: [PATCH v2 01/12] cachefiles: remove request from xarry during flush requests

2024-05-19 Thread Gao Xiang
avoid accessing freed requests. Fixes: c8383054506c ("cachefiles: notify the user daemon when looking up cookie") Signed-off-by: Baokun Li Reviewed-by: Jia Zhu Reviewed-by: Gao Xiang Thanks, Gao Xiang --- fs/cachefiles/daemon.c | 1 + 1 file changed, 1 insertion(+) diff --git

Re: [PATCH v2 02/12] cachefiles: remove err_put_fd tag in cachefiles_ondemand_daemon_read()

2024-05-19 Thread Gao Xiang
ed-by: Jia Zhu Reviewed-by: Gao Xiang Thanks, Gao Xiang --- fs/cachefiles/ondemand.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/cachefiles/ondemand.c b/fs/cachefiles/ondemand.c index 4ba42f1fa3b4..fd49728d8bae 100644 --- a/fs/cachefiles/ondemand.c +++ b/fs/

Re: [PATCH] erofs: clean up erofs_show_options()

2024-05-17 Thread Gao Xiang
On 2024/5/17 17:56, Hongzhen Luo wrote: Avoid unnecessary #ifdefs and simplify the code a bit. Signed-off-by: Hongzhen Luo Reviewed-by: Gao Xiang Thanks, Gao Xiang

[PATCH] erofs-utils: mkfs: add `--zfeature-bits` option

2024-05-17 Thread Gao Xiang
Thus, we could traverse all compression features with a number easily in the testcases. Signed-off-by: Gao Xiang --- mkfs/main.c | 160 1 file changed, 124 insertions(+), 36 deletions(-) diff --git a/mkfs/main.c b/mkfs/main.c index c26cb56

[PATCH 2/2] erofs-utils: pretty root directory progressinfo

2024-05-15 Thread Gao Xiang
Avoid `Processing ...` or `file dumped (mode 40755)`.. Signed-off-by: Gao Xiang --- lib/inode.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/inode.c b/lib/inode.c index 8ec87e6..67a572d 100644 --- a/lib/inode.c +++ b/lib/inode.c @@ -1405,10 +1405,11 @@ static

[PATCH 1/2] erofs-utils: correct the default number of workers in the usage

2024-05-15 Thread Gao Xiang
Fixes: 59c36e7a4008 ("erofs-utils: mkfs: use all available processors by default") Signed-off-by: Gao Xiang --- mkfs/main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mkfs/main.c b/mkfs/main.c index 4c3620d..455c152 100644 --- a/mkfs/main.c +++ b/m

[PATCH] erofs-utils: unify the tree traversal for the rebuild mode

2024-05-15 Thread Gao Xiang
Let's drop the legacy approach and `tarerofs` will be applied too. Signed-off-by: Gao Xiang --- include/erofs/internal.h | 7 +- lib/diskbuf.c| 14 +- lib/inode.c | 384 ++- 3 files changed, 232 insertions(+), 173 deletions

[PATCH v3] erofs-utils: add preliminary zstd support [x]

2024-05-14 Thread Gao Xiang
it will take more efforts. [ TODO: Later I tend to add another way to generate fixed-sized input pclusters temporarily for relatively large pcluster sizes as an option since it will have minor impacts to the results. ] Signed-off-by: Gao Xiang --- changes since v2: - use

[GIT PULL] erofs updates for 6.10-rc1

2024-05-13 Thread Gao Xiang
merge conflict is observed. Thanks, Gao Xiang The following changes since commit dd5a440a31fae6e459c0d627162825505361: Linux 6.9-rc7 (2024-05-05 14:06:01 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git tags/erofs-for-6.10-rc1

[PATCH v3] erofs-utils: add preliminary zstd support [x]

2024-05-09 Thread Gao Xiang
it will take more efforts. [ TODO: Later I tend to add another way to generate fixed-sized input pclusters temporarily for relatively large pcluster sizes as an option since it will have minor impacts to the results. ] Signed-off-by: Gao Xiang --- v3: - Fix incorrect assumption

Re: [PATCH] erofs-utils: add preliminary zstd support

2024-05-09 Thread Gao Xiang
I will look into that, thanks! Thanks, Gao Xiang

[PATCH v2] erofs-utils: add preliminary zstd support [x]

2024-05-08 Thread Gao Xiang
it will take more efforts. [ TODO: Later I tend to add another way to generate fixed-sized input pclusters temporarily for relatively large pcluster sizes as an option since it will have minor impacts to the results. ] Signed-off-by: Gao Xiang --- v2: Use a newer helper

[PATCH v2] erofs: Zstandard compression support

2024-05-08 Thread Gao Xiang
From: Gao Xiang Add Zstandard compression as the 4th supported algorithm since it becomes more popular now and some end users have asked this for quite a while [1][2]. Each EROFS physical cluster contains only one valid standard Zstandard frame as described in [3] so that decompression can

[PATCH] erofs: clean up z_erofs_load_full_lcluster()

2024-05-08 Thread Gao Xiang
Only four lcluster types here, remove redundant code. No real logic changes. Signed-off-by: Gao Xiang --- Some random cleanup out of the upcoming big lclusters.. fs/erofs/erofs_fs.h | 5 + fs/erofs/zmap.c | 21 + 2 files changed, 6 insertions(+), 20 deletions

[PATCH] erofs-utils: add preliminary zstd support

2024-05-08 Thread Gao Xiang
it will take more efforts. [ TODO: Later I tend to add another way to generate fixed-sized input pclusters temporarily for relatively large pcluster sizes as an option since it will have minor impacts to the results. ] Signed-off-by: Gao Xiang --- configure.ac | 34

[PATCH] erofs: Zstandard compression support

2024-05-08 Thread Gao Xiang
://lore.kernel.org/r/y08h+z6czdns1...@b-p7tqmd6m-0146.lan [3] https://www.rfc-editor.org/rfc/rfc8478.txt Signed-off-by: Gao Xiang --- fs/erofs/Kconfig | 15 ++ fs/erofs/Makefile| 1 + fs/erofs/compress.h | 4 + fs/erofs/decompressor.c | 7 + fs/erofs

Re: [PATCH AUTOSEL 6.8 26/52] erofs: reliably distinguish block based and fscache mode

2024-05-07 Thread Gao Xiang
[...] Now when erofs_kill_sb() is called, erofs_sb_info must have been initialised, so use sbi->fsid to distinguish between the two modes. Signed-off-by: Christian Brauner Signed-off-by: Baokun Li Reviewed-by: Jingbo Xu Reviewed-by: Gao Xiang Reviewed-by: Chao Yu Link: https://lore.kernel.or

Re: [PATCH AUTOSEL 6.6 21/43] erofs: reliably distinguish block based and fscache mode

2024-05-07 Thread Gao Xiang
[...] Now when erofs_kill_sb() is called, erofs_sb_info must have been initialised, so use sbi->fsid to distinguish between the two modes. Signed-off-by: Christian Brauner Signed-off-by: Baokun Li Reviewed-by: Jingbo Xu Reviewed-by: Gao Xiang Reviewed-by: Chao Yu Link: https://lore.kernel.or

[PATCH v2 1/2] erofs-utils: record pclustersize in bytes instead of blocks

2024-04-30 Thread Gao Xiang
So that we don't need to handle blocksizes everywhere. Signed-off-by: Gao Xiang --- v1: fix CI failures: https://github.com/erofs/erofsnightly/actions/runs/8896341493/job/24428809079 include/erofs/config.h | 4 +++- lib/compress.c | 38

Re: [PATCH] erofs-utils: optimize pthread_cond_signal calling

2024-04-30 Thread Gao Xiang
On 2024/5/1 10:24, Noboru Asai wrote: Call pthread_cond_signal once per file. Signed-off-by: Noboru Asai Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH] erofs-utils: simplify file handling

2024-04-30 Thread Gao Xiang
ncoded_file(inode, ctx->fd, 0); -out: - close(ctx->fd); + fd = open(inode->i_srcpath, O_RDONLY | O_BINARY); At a quick glance, here we need to open i_srcpath again, I tend to avoid it so I use dup instead. Thanks, Gao Xiang

[PATCH 1/2] erofs-utils: record pclustersize in bytes instead of blocks

2024-04-30 Thread Gao Xiang
So that we don't need to handle blocksizes everywhere. Signed-off-by: Gao Xiang --- include/erofs/config.h | 4 +++- lib/compress.c | 30 -- lib/compress_hints.c | 11 ++- lib/config.c | 2 -- mkfs/main.c| 16

[PATCH 2/2] erofs-utils: lib: adjust MicroLZMA default dictionary size

2024-04-30 Thread Gao Xiang
If dict_size is not given, it will be set as max(32k, pclustersize * 8) but no more than Z_EROFS_LZMA_MAX_DICT_SIZE. Also kill an obsolete warning since multi-threaded support is landed. Signed-off-by: Gao Xiang --- lib/compressor_liblzma.c | 19 +++ mkfs/main.c

[GIT PULL] erofs fixes for 6.9-rc7

2024-04-29 Thread Gao Xiang
have been in -next and no potential merge conflict is observed. [1] https://lore.kernel.org/r/15ab9875-5123-7bc2-bb25-fc683129a...@huawei.com Thanks, Gao Xiang The following changes since commit ed30a4a51bb196781c8058073ea720133a65596f: Linux 6.9-rc5 (2024-04-21 12:35:54 -0700) are available

Re: [PATCH -next v3 2/2] erofs: reliably distinguish block based and fscache mode

2024-04-28 Thread Gao Xiang
initialised, so use sbi->fsid to distinguish between the two modes. Signed-off-by: Christian Brauner Signed-off-by: Baokun Li Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH -next v3 1/2] erofs: get rid of erofs_fs_context

2024-04-28 Thread Gao Xiang
, no functional changes. Suggested-by: Jingbo Xu Signed-off-by: Baokun Li Reviewed-by: Gao Xiang Thanks, Gao Xiang

[PATCH] erofs-utils: mkfs: use all available processors by default

2024-04-27 Thread Gao Xiang
Fulfill the needs of most users. Signed-off-by: Gao Xiang --- include/erofs/config.h | 3 +-- lib/compress.c | 16 lib/config.c | 5 - mkfs/main.c| 10 -- 4 files changed, 17 insertions(+), 17 deletions(-) diff --git a/include/erofs

Re: [PATCH v2] erofs-utils: add missing block counting

2024-04-25 Thread Gao Xiang
On 2024/4/25 10:48, Noboru Asai wrote: Hi Gao, Oh, sorry. I knew to access i_blkaddr on uncompressed file, but it didn't occur on the file system for testing, so I overlooked it. I needed to be careful. np, I've fixed it :) Thanks, Gao Xiang

Re: Trying to work with the tests

2024-04-24 Thread Gao Xiang
On 2024/4/24 18:37, Ian Kent wrote: On 24/4/24 17:47, Ian Kent wrote: On 23/4/24 18:34, Gao Xiang wrote: Hi Ian, On 2024/4/22 21:10, Ian Kent wrote: On 22/4/24 17:12, Gao Xiang wrote: Hi Ian, (+Cc Jingbo here). On 2024/4/22 16:31, Ian Kent wrote: I'm new to the list so Hi to all

Re: [PATCH v2] erofs-utils: add missing block counting

2024-04-24 Thread Gao Xiang
On Wed, Apr 24, 2024 at 02:15:58PM +0800, Gao Xiang wrote: > > > On 2024/4/24 13:59, Noboru Asai wrote: > > Add missing block counting when the data to be inlined is not inlined. > > > > --- > > v2: > > - move from erofs_write_tail_end() to erofs_p

Re: Trying to work with the tests

2024-04-24 Thread Gao Xiang
On 2024/4/24 17:47, Ian Kent wrote: On 23/4/24 18:34, Gao Xiang wrote: Hi Ian, On 2024/4/22 21:10, Ian Kent wrote: On 22/4/24 17:12, Gao Xiang wrote: Hi Ian, (+Cc Jingbo here). On 2024/4/22 16:31, Ian Kent wrote: I'm new to the list so Hi to all, I'm working with a heavily patched

Re: [PATCH v2] erofs-utils: add missing block counting

2024-04-24 Thread Gao Xiang
On 2024/4/24 13:59, Noboru Asai wrote: Add missing block counting when the data to be inlined is not inlined. --- v2: - move from erofs_write_tail_end() to erofs_prepare_tail_block() Signed-off-by: Noboru Asai Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH v2] erofs-utils: fsck: extract chunk-based file with hole correctly

2024-04-24 Thread Gao Xiang
t EROFS_MAP_MAPPED is unset, which is not the case for chunk-based files with holes. This patch handles the corner case correctly. Signed-off-by: Yifan Zhao Signed-off-by: Gao Xiang --- fsck/main.c | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/fsck/main.c b

Re: [PATCH] erofs-utils: add missing block counting

2024-04-23 Thread Gao Xiang
to revisit it. erofs_prepare_tail_block() is the original place to decide if a fallback tail block is needed, that is also true for old versions. Thanks, Gao Xiang

Re: [PATCH] erofs-utils: add missing block counting

2024-04-23 Thread Gao Xiang
for this. Thanks, Gao Xiang --- lib/inode.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/inode.c b/lib/inode.c index cf22bbe..727dcee 100644 --- a/lib/inode.c +++ b/lib/inode.c @@ -840,6 +840,7 @@ static int erofs_write_tail_end(struct erofs_inode *inode) inode->idata_size

Re: [PATCH 5/5] cachefiles: add missing lock protection when polling

2024-04-23 Thread Gao Xiang
OLLIN events in ondemand mode") Signed-off-by: Jingbo Xu Reviewed-by: Joseph Qi Reviewed-by: Gao Xiang I'm not sure why this patch didn't send upstream, https://gitee.com/anolis/cloud-kernel/commit/324ecaaa10fefb0e3d94b547e3170e40b90cda1f But since we're now working on upstreaming, so

Re: [PATCH -next] erofs: modify the error message when prepare_ondemand_read failed

2024-04-23 Thread Gao Xiang
(+cc linux-erofs & LKML) On 2024/4/24 10:39, Hongbo Li wrote: When prepare_ondemand_read failed, wrong error message is printed. The prepare_read is also implemented in cachefiles, so we amend it. Signed-off-by: Hongbo Li Reviewed-by: Gao Xiang Could you resend the patch with pr

  1   2   3   4   5   6   7   8   9   10   >