Re: [PATCH] erofs: rename per-CPU buffers to global buffer pool and make it configurable

On 2024/3/22 11:47, Chunhai Guo wrote: It will cost more time if compressed buffers are allocated on demand for low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep compressed data if in-place decompression is unfulfilled. While it is kind of wasteful of memory for a

[PATCH] erofs: rename per-CPU buffers to global buffer pool and make it configurable

It will cost more time if compressed buffers are allocated on demand for low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep compressed data if in-place decompression is unfulfilled. While it is kind of wasteful of memory for a device with hundreds of CPUs, and only a small

Re: Slow concurrent reads on mounted loopback images

Hi GRANGER, On 2024/3/22 01:49, GRANGER Nicolas wrote: Hi erofs team, I'm working on a small daemon that caches and mounts disk images containing datasets based on usage (https://github.com/CEA-LIST/scratch_manager ). My understanding is that

Slow concurrent reads on mounted loopback images

Hi erofs team, I'm working on a small daemon that caches and mounts disk images containing datasets based on usage (https://github.com/CEA-LIST/scratch_manager). My understanding is that this is a good use-case for EROFS. However, in my experience concurrent reads on loopback mounted disk

Re: [PATCH 2/2] erofs-utils: mkfs: introduce inter-file multi-threaded compression

On 2024/3/21 20:34, Yifan Zhao wrote: On 2024/3/21 10:07, Gao Xiang wrote: On 2024/3/21 02:15, Huang Jianan wrote: On 2024/3/17 22:41, Yifan Zhao wrote: This patch allows parallelizing the compression process of different files in mkfs. Specifically, a traverser thread traverses the

Re: [PATCH 2/2] erofs-utils: mkfs: introduce inter-file multi-threaded compression

On 2024/3/21 10:07, Gao Xiang wrote: On 2024/3/21 02:15, Huang Jianan wrote: On 2024/3/17 22:41, Yifan Zhao wrote: This patch allows parallelizing the compression process of different files in mkfs. Specifically, a traverser thread traverses the files and issues the compression task,

Re: [PATCH 2/2] erofs-utils: mkfs: introduce inter-file multi-threaded compression

Thanks for the review! On 2024/3/21 2:15, Huang Jianan wrote: On 2024/3/17 22:41, Yifan Zhao wrote: This patch allows parallelizing the compression process of different files in mkfs. Specifically, a traverser thread traverses the files and issues the compression task, which is handled by the

[PATCH v2 2/2] erofs-utils: mkfs: introduce inter-file multi-threaded compression

This patch allows parallelizing the compression process of different files in mkfs. Specifically, a traverser thread traverses the files and issues the compression task, which is handled by the workers. Then, the main thread consumes them and writes the compressed data to the device. To this end,

[PATCH v2 1/2] erofs-utils: lib: split function logic in inode.c

This patch splits part of the logic in function erofs_mkfs_build_tree() and erofs_mkfs_build_tree_from_path() into several new functions. This is in preparation for the upcoming inter-file multi-threaded compression feature. Signed-off-by: Yifan Zhao --- lib/inode.c | 161

[PATCH v2 0/2] erofs-utils: mkfs: introduce inter-file multi-threaded compression

changelog since v1: - apply fixes from Jianan's review - remove misincluded changes in list.h Yifan Zhao (2): erofs-utils: lib: split function logic in inode.c erofs-utils: mkfs: introduce inter-file multi-threaded compression include/erofs/compress.h | 16 ++ include/erofs/internal.h |

Re: [PATCH] erofs-utils: move pclustersize to struct z_erofs_compress_sctx

On 2024/3/21 15:02, Noboru Asai wrote: With -E(all-)fragments, pclustersize has a different value per segment, so move it to struct z_erofs_compress_sctx. Signed-off-by: Noboru Asai Thanks, I think it's a good change for the later fragment multi-threaded compression. Thanks, Gao XIang

[PATCH] erofs-utils: move pclustersize to struct z_erofs_compress_sctx

With -E(all-)fragments, pclustersize has a different value per segment, so move it to struct z_erofs_compress_sctx. Signed-off-by: Noboru Asai --- lib/compress.c | 17 + 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/lib/compress.c b/lib/compress.c index