Currently, the creation of EROFS compressed image creation is
single-threaded, which suffers from performance issues. This patch
attempts to address it by compressing the large file in parallel.
Specifically, each input file larger than 16MB is splited into segments,
and each worker thread
This patch introduces a --worker=# parameter for the incoming
multi-threaded compression support. It also introduces a segment size
used in multi-threaded compression, which has the default value 16MB
and cannot be modified.
It also introduces a concept called `segment size` to split large files
Add a workqueue implementation for multi-threading support inspired by
xfsprogs.
Signed-off-by: Yifan Zhao
Suggested-by: Gao Xiang
---
configure.ac | 16 +
include/erofs/internal.h | 3 +
include/erofs/workqueue.h | 35 +++
lib/Makefile.am | 4 ++
From: Gao Xiang
In order to prepare for multi-threaded decompression.
Signed-off-by: Gao Xiang
---
configure.ac | 1 +
include/erofs/config.h | 1 +
lib/config.c | 12
3 files changed, 14 insertions(+)
diff --git a/configure.ac b/configure.ac
index
From: Gao Xiang
Add some helpers (relaxed semantics) in order to prepare for the
upcoming multi-threaded support.
For example, compressor may be initialized more than once in different
worker threads, resulting in noisy warnings.
This patch makes sure that each message will be printed only
changelog since v5:
- use memory buffer instead of tmpfile to store intermediate compressed data
- re-strucutre the 5th patch
Gao Xiang (2):
erofs-utils: add a helper to get available processors
erofs-utils: lib: introduce atomic operations
Yifan Zhao (3):
erofs-utils: introduce
On Fri, Feb 23, 2024 at 12:56:34PM -0500, Steven Rostedt wrote:
> From: "Steven Rostedt (Google)"
>
> [
>This is a treewide change. I will likely re-create this patch again in
>the second week of the merge window of v6.9 and submit it then. Hoping
>to keep the conflicts that it will
On Thu, 14 Mar 2024 09:57:57 -0700
Alison Schofield wrote:
> On Fri, Feb 23, 2024 at 12:56:34PM -0500, Steven Rostedt wrote:
> > From: "Steven Rostedt (Google)"
> >
> > [
> >This is a treewide change. I will likely re-create this patch again in
> >the second week of the merge window of
I have been contributing to erofs for sometime and I would like to help
with code reviews as well.
Signed-off-by: Sandeep Dhavale
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 4f298c4187fb..b130340d71bb 100644
--- a/MAINTAINERS
+++
On 2024/3/15 07:14, Sandeep Dhavale wrote:
I have been contributing to erofs for sometime and I would like to help
with code reviews as well.
Signed-off-by: Sandeep Dhavale
Looks good to me, and thanks for taking your time on erofs project:
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
Add some helpers (relaxed semantics) in order to prepare for the
upcoming multi-threaded support.
For example, compressor may be initialized more than once in different
worker threads, resulting in noisy warnings.
This patch makes sure that each message will be printed only once by
adding
From: Yifan Zhao
Currently, the creation of EROFS compressed image creation is
single-threaded, which suffers from performance issues. This patch
attempts to address it by compressing the large file in parallel.
Specifically, each input file larger than 16MB is splited into segments,
and each
From: Yifan Zhao
Add a workqueue implementation for multi-threading support inspired by
xfsprogs.
Signed-off-by: Yifan Zhao
Suggested-by: Gao Xiang
Signed-off-by: Gao Xiang
---
configure.ac | 16 +
include/erofs/internal.h | 3 +
include/erofs/workqueue.h | 34
In order to prepare for multi-threaded decompression.
Signed-off-by: Yifan Zhao
Signed-off-by: Gao Xiang
---
configure.ac | 1 +
include/erofs/config.h | 1 +
lib/config.c | 12
3 files changed, 14 insertions(+)
diff --git a/configure.ac b/configure.ac
index
From: Yifan Zhao
This patch introduces `--workers=#` parameter for the incoming
multi-threaded compression support.
It also introduces a concept called `segment size` to split large
inodes for multi-threaded compression, which has the fixed value
16MiB and cannot be modified for now.
On 2024/3/14 20:37, Yifan Zhao wrote:
Currently, the creation of EROFS compressed image creation is
single-threaded, which suffers from performance issues. This patch
attempts to address it by compressing the large file in parallel.
Specifically, each input file larger than 16MB is splited
On 2024/3/15 7:14, Sandeep Dhavale via Linux-erofs wrote:
I have been contributing to erofs for sometime and I would like to help
with code reviews as well.
Thank you for the effort and looks good to me. :)
Signed-off-by: Sandeep Dhavale
Acked-by: Chao Yu
Thanks,
On 2024/3/15 09:10, Gao Xiang wrote:
From: Yifan Zhao
Currently, the creation of EROFS compressed image creation is
single-threaded, which suffers from performance issues. This patch
attempts to address it by compressing the large file in parallel.
Specifically, each input file larger than
On 2024/3/15 10:39, Gao Xiang wrote:
On 2024/3/14 20:37, Yifan Zhao wrote:
Currently, the creation of EROFS compressed image creation is
single-threaded, which suffers from performance issues. This patch
attempts to address it by compressing the large file in parallel.
Specifically, each
19 matches
Mail list logo
