[PATCH v2 8/8] erofs-utils: mkfs: enable inter-file multi-threaded compression

2024-04-21 Thread Gao Xiang
From: Gao Xiang Dispatch deferred ops in another per-sb worker thread. Note that deferred ops are strictly FIFOed. Signed-off-by: Gao Xiang --- include/erofs/internal.h | 6 ++ lib/inode.c | 119 ++- 2 files changed, 123 insertions(+), 2

[PATCH v2 7/8] erofs-utils: lib: introduce non-directory jobitem context

2024-04-21 Thread Gao Xiang
From: Gao Xiang It will describe EROFS_MKFS_JOB_NDIR defer work. Also, start compression before queueing EROFS_MKFS_JOB_NDIR. Signed-off-by: Gao Xiang --- lib/inode.c | 62 +++-- 1 file changed, 51 insertions(+), 11 deletions(-) diff --git

[PATCH v2 6/8] erofs-utils: mkfs: prepare inter-file multi-threaded compression

2024-04-21 Thread Gao Xiang
From: Yifan Zhao This patch separates the compression process into two parts. Specifically, erofs_begin_compressed_file() will trigger compression. erofs_write_compressed_file() will wait for the compression finish and write compressed (meta)data. Note that it's possible that

[PATCH v2 5/8] erofs-utils: lib: split up z_erofs_mt_compress()

2024-04-21 Thread Gao Xiang
From: Gao Xiang The on-disk compressed data write will be moved into a new function erofs_mt_write_compressed_file(). Signed-off-by: Gao Xiang --- lib/compress.c | 162 - 1 file changed, 93 insertions(+), 69 deletions(-) diff --git

[PATCH v2 4/8] erofs-utils: rearrange several fields for multi-threaded mkfs

2024-04-21 Thread Gao Xiang
From: Gao Xiang They should be located in `struct z_erofs_compress_ictx`. Signed-off-by: Gao Xiang --- lib/compress.c | 57 +++--- 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/lib/compress.c b/lib/compress.c index 8ca4033..0bc5426

[PATCH v2 3/8] erofs-utils: lib: split out erofs_commit_compressed_file()

2024-04-21 Thread Gao Xiang
From: Gao Xiang Just split out on-disk compressed metadata commit logic. Signed-off-by: Gao Xiang --- lib/compress.c | 191 +++-- 1 file changed, 105 insertions(+), 86 deletions(-) diff --git a/lib/compress.c b/lib/compress.c index b084446..8ca4033

[PATCH v2 2/8] erofs-utils: lib: prepare for later deferred work

2024-04-21 Thread Gao Xiang
From: Gao Xiang Split out ordered metadata operations and add the following helpers: - erofs_mkfs_jobfn() - erofs_mkfs_go() to handle these mkfs job items for multi-threadding support. Signed-off-by: Gao Xiang --- lib/inode.c | 69 - 1

[PATCH v2 1/8] erofs-utils: use erofs_atomic_t for inode->i_count

2024-04-21 Thread Gao Xiang
From: Gao Xiang Since `inode->i_count` can be touched for more than one thread if multi-threading is enabled. Signed-off-by: Gao Xiang --- patchset v1->v2: - Fix `--all-fragments` functionality; - Fix issues pointed out by Yifan. include/erofs/atomic.h | 10 ++