have been in -next and no potential merge conflict is
observed.
[1] https://lore.kernel.org/r/15ab9875-5123-7bc2-bb25-fc683129a...@huawei.com
Thanks,
Gao Xiang
The following changes since commit ed30a4a51bb196781c8058073ea720133a65596f:
Linux 6.9-rc5 (2024-04-21 12:35:54 -0700)
are available
initialised, so use sbi->fsid to distinguish between the two modes.
Signed-off-by: Christian Brauner
Signed-off-by: Baokun Li
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
, no functional changes.
Suggested-by: Jingbo Xu
Signed-off-by: Baokun Li
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
Fulfill the needs of most users.
Signed-off-by: Gao Xiang
---
include/erofs/config.h | 3 +--
lib/compress.c | 16
lib/config.c | 5 -
mkfs/main.c| 10 --
4 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/include/erofs
On 2024/4/25 10:48, Noboru Asai wrote:
Hi Gao,
Oh, sorry.
I knew to access i_blkaddr on uncompressed file, but it didn't occur
on the file system for testing, so I overlooked it.
I needed to be careful.
np, I've fixed it :)
Thanks,
Gao Xiang
On 2024/4/24 18:37, Ian Kent wrote:
On 24/4/24 17:47, Ian Kent wrote:
On 23/4/24 18:34, Gao Xiang wrote:
Hi Ian,
On 2024/4/22 21:10, Ian Kent wrote:
On 22/4/24 17:12, Gao Xiang wrote:
Hi Ian,
(+Cc Jingbo here).
On 2024/4/22 16:31, Ian Kent wrote:
I'm new to the list so Hi to all
On Wed, Apr 24, 2024 at 02:15:58PM +0800, Gao Xiang wrote:
>
>
> On 2024/4/24 13:59, Noboru Asai wrote:
> > Add missing block counting when the data to be inlined is not inlined.
> >
> > ---
> > v2:
> > - move from erofs_write_tail_end() to erofs_p
On 2024/4/24 17:47, Ian Kent wrote:
On 23/4/24 18:34, Gao Xiang wrote:
Hi Ian,
On 2024/4/22 21:10, Ian Kent wrote:
On 22/4/24 17:12, Gao Xiang wrote:
Hi Ian,
(+Cc Jingbo here).
On 2024/4/22 16:31, Ian Kent wrote:
I'm new to the list so Hi to all,
I'm working with a heavily patched
On 2024/4/24 13:59, Noboru Asai wrote:
Add missing block counting when the data to be inlined is not inlined.
---
v2:
- move from erofs_write_tail_end() to erofs_prepare_tail_block()
Signed-off-by: Noboru Asai
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
t EROFS_MAP_MAPPED
is unset, which is not the case for chunk-based files with holes.
This patch handles the corner case correctly.
Signed-off-by: Yifan Zhao
Signed-off-by: Gao Xiang
---
fsck/main.c | 11 ++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/fsck/main.c b
to revisit it. erofs_prepare_tail_block() is the
original place to decide if a fallback tail block is needed,
that is also true for old versions.
Thanks,
Gao Xiang
for this.
Thanks,
Gao Xiang
---
lib/inode.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/lib/inode.c b/lib/inode.c
index cf22bbe..727dcee 100644
--- a/lib/inode.c
+++ b/lib/inode.c
@@ -840,6 +840,7 @@ static int erofs_write_tail_end(struct erofs_inode *inode)
inode->idata_size
OLLIN events in
ondemand mode")
Signed-off-by: Jingbo Xu
Reviewed-by: Joseph Qi
Reviewed-by: Gao Xiang
I'm not sure why this patch didn't send upstream,
https://gitee.com/anolis/cloud-kernel/commit/324ecaaa10fefb0e3d94b547e3170e40b90cda1f
But since we're now working on upstreaming, so
(+cc linux-erofs & LKML)
On 2024/4/24 10:39, Hongbo Li wrote:
When prepare_ondemand_read failed, wrong error message is printed.
The prepare_read is also implemented in cachefiles, so we amend it.
Signed-off-by: Hongbo Li
Reviewed-by: Gao Xiang
Could you resend the patch with pr
On 2024/4/10 13:19, syzbot wrote:
Hello,
syzbot tried to test the proposed patch but the build/boot failed:
#syz invalid
Since
https://lore.kernel.org/r/20240408215231.3376659-1-dhav...@google.com
has been folded into the original patch and this issue is only in -next.
Thanks,
Gao Xiang
, no functional changes.
Suggested-by: Jingbo Xu
Signed-off-by: Baokun Li
Thanks, it looks good to me, let's see how it behaves after applying
to -next.
Thanks,
Gao Xiang
Hi Ian,
On 2024/4/22 21:10, Ian Kent wrote:
On 22/4/24 17:12, Gao Xiang wrote:
Hi Ian,
(+Cc Jingbo here).
On 2024/4/22 16:31, Ian Kent wrote:
I'm new to the list so Hi to all,
I'm working with a heavily patched 5.14 kernel and I've gathered together
patches to bring erofs
up to 5.19
if (ret)
- goto out;
+ if (!(map.m_flags & EROFS_MAP_MAPPED)) {
+ memset(raw, 0, count);
I think we could use lseek instead of write
zeros explicitly..
Thanks,
Gao Xiang
quite sure why "fscache/005" fails, hopefully Jingbo could
help you on this.
Thanks!
Gao Xiang
erofs/018 does not run due to "lzma compression is disabled, skipped." message
which I think
is too old a version of xz.
Any insight into cases that could cause these tests to f
From: Gao Xiang
Dispatch deferred ops in another per-sb worker thread. Note that
deferred ops are strictly FIFOed.
Signed-off-by: Gao Xiang
---
include/erofs/internal.h | 6 ++
lib/inode.c | 119 ++-
2 files changed, 123 insertions(+), 2
From: Gao Xiang
It will describe EROFS_MKFS_JOB_NDIR defer work. Also, start
compression before queueing EROFS_MKFS_JOB_NDIR.
Signed-off-by: Gao Xiang
---
lib/inode.c | 62 +++--
1 file changed, 51 insertions(+), 11 deletions(-)
diff --git
that erofs_begin_compressed_file() and
erofs_write_compressed_file() run with different threads even the
global inode context is used, thus add another synchronization point.
Signed-off-by: Yifan Zhao
Co-authored-by: Tong Xin
Signed-off-by: Gao Xiang
---
include/erofs/compress.h | 5 +-
lib/compress.c
From: Gao Xiang
The on-disk compressed data write will be moved into a new function
erofs_mt_write_compressed_file().
Signed-off-by: Gao Xiang
---
lib/compress.c | 162 -
1 file changed, 93 insertions(+), 69 deletions(-)
diff --git a/lib
From: Gao Xiang
They should be located in `struct z_erofs_compress_ictx`.
Signed-off-by: Gao Xiang
---
lib/compress.c | 57 +++---
1 file changed, 31 insertions(+), 26 deletions(-)
diff --git a/lib/compress.c b/lib/compress.c
index 8ca4033..0bc5426
From: Gao Xiang
Just split out on-disk compressed metadata commit logic.
Signed-off-by: Gao Xiang
---
lib/compress.c | 191 +++--
1 file changed, 105 insertions(+), 86 deletions(-)
diff --git a/lib/compress.c b/lib/compress.c
index b084446..8ca4033
From: Gao Xiang
Split out ordered metadata operations and add the following helpers:
- erofs_mkfs_jobfn()
- erofs_mkfs_go()
to handle these mkfs job items for multi-threadding support.
Signed-off-by: Gao Xiang
---
lib/inode.c | 69 -
1
From: Gao Xiang
Since `inode->i_count` can be touched for more than one thread if
multi-threading is enabled.
Signed-off-by: Gao Xiang
---
patchset v1->v2:
- Fix `--all-fragments` functionality;
- Fix issues pointed out by Yifan.
include/erofs/atomic.h | 10 ++
include
mkfs: introduce inner-file multi-threaded
compression")
Signed-off-by: Yifan Zhao
[ Gao Xiang: simplify a bit. ]
Signed-off-by: Gao Xiang
---
v2:
Yifan's patch is almost correct, it just has minor changes.
include/erofs/dedupe.h | 2 +-
lib/compress.c | 14 +++---
li
> }
> > > > > -static int erofs_fc_get_tree(struct fs_context *fc)
> > > > > +static void erofs_ctx_to_info(struct fs_context *fc)
> > > > > {
> > > > > struct erofs_fs_context *ctx = fc->fs_private;
> > > > > + struct erofs_sb_info *sbi = fc->s_fs_info;
> > > > > +
> > > > > + sbi->opt = ctx->opt;
> > > > > + sbi->devs = ctx->devs;
> > > > > + ctx->devs = NULL;
> > > > > + sbi->fsid = ctx->fsid;
> > > > > + ctx->fsid = NULL;
> > > > > + sbi->domain_id = ctx->domain_id;
> > > > > + ctx->domain_id = NULL;
> > > > > +}
> > > > I'm not sure if abstracting this logic into a seperate helper really
> > > > helps understanding the code as the logic itself is quite simple and
> > > > easy to be understood. Usually it's a hint of over-abstraction when a
> > > > simple helper has only one caller.
> > > >
> > > Static functions that have only one caller are compiled inline, so we
> > > don't have to worry about how that affects the code.
> > >
> > > The reason these codes are encapsulated in a separate function is so
> > > that the code reader understands that these codes are integrated
> > > as a whole, and that we shouldn't have to move one or two of these
> > > lines individually.
> > >
> > > But after we get rid of erofs_fs_context, those won't be needed
> > > anymore.
> > Yeah, I understand. It's only coding style concerns.
> >
> >
> >
> Okay, thanks！
I'm fine to get rid of those (erofs_fs_context) as long as the codebase
is more clearer and simple. BTW, for the current codebase, I also think
it's unneeded to have a separate helper called once without extra actual
meaning...
Thanks,
Gao Xiang
>
> --
> With Best Regards,
> Baokun Li
our effort on this issue again!
Thanks,
Gao Xiang
o detect blocks filled with zeros during chunking, we insert block
> filled with zeros (zerochunk) in the hashmap. If we detect a possible
> dedupe, we map it to the hole so there is no physical block assigned.
>
> Signed-off-by: Sandeep Dhavale
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
lled for inline fallback instead.
Let me think more about it and update the patch.
Thanks,
Gao Xiang
Wed Apr 17 16:53:10 2024
> Filesystem features: sb_csum mtime 0padding
> Filesystem UUID:
> e66f6dd1-6882-48c3-9770-fee7c4841a93
>
> Signed-off-by: Sandeep Dhavale
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
On Wed, Apr 17, 2024 at 10:59:53AM +0800, Baokun Li wrote:
> On 2024/4/16 22:49, Gao Xiang wrote:
> > On Tue, Apr 16, 2024 at 02:35:08PM +0200, Christian Brauner wrote:
> > > > > I'm not sure how to resolve it in EROFS itself, anyway...
> > > Instead of allocatin
les since EROFS
> supports multiple compression algorithms in one image.
>
> Signed-off-by: Jianan Huang
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
On Tue, Apr 16, 2024 at 07:55:05PM +0800, Yifan Zhao wrote:
>
> On 4/16/24 4:04 PM, Gao Xiang wrote:
> > From: Gao Xiang
> >
> > They should be located in `struct z_erofs_compress_ictx`.
> >
> > Signed-off-by: Gao Xian
Hi Yifan,
On Tue, Apr 16, 2024 at 07:58:30PM +0800, Yifan Zhao wrote:
>
> On 4/16/24 4:04 PM, Gao Xiang wrote:
> > From: Gao Xiang
> >
> > Split out ordered metadata operations and add the following helpers:
> >
> > - erofs_mkfs_jobfn()
> >
n,
Could you also check this on your side?
Thanks,
Gao Xiang
> From e4f586a41748b6edc05aca36d49b7b39e55def81 Mon Sep 17 00:00:00 2001
> From: Christian Brauner
> Date: Mon, 15 Apr 2024 20:17:46 +0800
> Subject: [PATCH] erofs: reliably distinguish block based and fscache mode
>
> W
From: Gao Xiang
It will describe EROFS_MKFS_JOB_NDIR defer work. Also start
compression before queueing EROFS_MKFS_JOB_NDIR.
Signed-off-by: Gao Xiang
---
lib/inode.c | 62 +++--
1 file changed, 51 insertions(+), 11 deletions(-)
diff --git
Signed-off-by: Gao Xiang
---
include/erofs/compress.h | 5 +-
lib/compress.c | 115 ++-
lib/inode.c | 17 +-
3 files changed, 95 insertions(+), 42 deletions(-)
diff --git a/include/erofs/compress.h b/include/erofs/compress.h
index
From: Gao Xiang
They should be located in `struct z_erofs_compress_ictx`.
Signed-off-by: Gao Xiang
---
lib/compress.c | 55 --
1 file changed, 31 insertions(+), 24 deletions(-)
diff --git a/lib/compress.c b/lib/compress.c
index a2e0d0f..72f33d2
From: Gao Xiang
Dispatch deferred ops in another per-sb worker thread. Note that
deferred ops are strictly FIFOed.
Signed-off-by: Gao Xiang
---
include/erofs/internal.h | 6 ++
lib/inode.c | 121 ++-
2 files changed, 124 insertions(+), 3
From: Gao Xiang
Just split out on-disk compressed metadata commit logic.
Signed-off-by: Gao Xiang
---
lib/compress.c | 191 +++--
1 file changed, 105 insertions(+), 86 deletions(-)
diff --git a/lib/compress.c b/lib/compress.c
index 74c5707..a2e0d0f
From: Gao Xiang
The on-disk compressed data write will be moved into a new function
erofs_mt_write_compressed_file().
Signed-off-by: Gao Xiang
---
lib/compress.c | 172 -
1 file changed, 99 insertions(+), 73 deletions(-)
diff --git a/lib
From: Gao Xiang
Split out ordered metadata operations and add the following helpers:
- erofs_mkfs_jobfn()
- erofs_mkfs_go()
to handle these mkfs job items for multi-threadding support.
Signed-off-by: Gao Xiang
---
lib/inode.c | 68 +
1
From: Gao Xiang
Since it can be touched for more than one thread if multi-threading
is enabled.
Signed-off-by: Gao Xiang
---
include/erofs/atomic.h | 10 ++
include/erofs/inode.h| 2 +-
include/erofs/internal.h | 3 ++-
lib/inode.c | 5 +++--
4 files changed, 16
%u` for `erofs_blksiz()`? since currently EROFS
block size isn't possible to be larger than PAGE_SIZE.
Even if block size > page size is supported, I think we should
not consider too large blocksizes.
Otherwise it looks good to me.
Thanks,
Gao Xiang
se super_block to determine whether it is currently in nodev
> fscahe mode or block device based mode. So if it is recorded in
> erofs_fs_context (aka fc->fs_private), we can't access the recorded data
> unless we pass fc into erofs_kill_sb() as well.
>
If I understand correctly, from the discussion above, I think
there exists a gap between alloc_super() and sb->s_bdev is set.
But .kill_sb() can be called between them and fc is not passed
into .kill_sb().
I'm not sure how to resolve it in EROFS itself, anyway...
Thanks,
Gao Xiang
s needed
anyway.)
Thanks,
Gao Xiang
On 2024/4/13 09:00, Sandeep Dhavale wrote:
On Fri, Apr 12, 2024 at 5:09 PM Gao Xiang wrote:
Hi Sandeep,
On 2024/4/13 06:51, Sandeep Dhavale wrote:
mkfs.erofs supports creating filesystem images with different
blocksizes. Add filesystem blocksize in super block dump so
its easier
-b811a815fa07
Filesystem blocksize: 65536
Signed-off-by: Sandeep Dhavale
Just a minor nit:
Could we move "Filesystem blocksize:" between the line of
"Filesystem magic number:" and "Filesystem blocks:" ?
Otherwise it looks good to me, thanks for this!
Thanks,
Gao Xiang
if (!zeros)
+ return -ENOMEM;
+
+ erofs_sha256(zeros, chunksize, sha256);
+ hash = memhash(sha256, sizeof(sha256));
`zeros` needs to be freed here I guess:
free(zeros);
Thanks,
Gao Xiang
+ chunk = malloc(sizeof(struct erofs_blobchunk));
+
Otherwise, large files won't be imported properly.
Fixes: e3dfe4b8db26 ("erofs-utils: mkfs: support tgz streams for tarerofs")
Fixes: 95d315fd7958 ("erofs-utils: introduce tarerofs")
Signed-off-by: Gao Xiang
---
lib/tar.c | 5 ++---
1 file changed, 2 insertions(+), 3 dele
On 2024/4/9 16:11, Hongzhen Luo wrote:
Use the superblock's UUID to generate the fsid when it's non-null.
Signed-off-by: Hongzhen Luo
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
---
fs/erofs/super.c | 12 +---
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/fs
#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git dev
.
Thanks,
Gao Xiang
patch?
I will add your credit into the original patch.
Thanks,
Gao Xiang
---
fs/erofs/zutil.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fs/erofs/zutil.c b/fs/erofs/zutil.c
index b9b99158bb4e..036024bce9f7 100644
--- a/fs/erofs/zutil.c
+++ b/fs/erofs/zutil.c
@@ -
enable this. So I will apply this first :)
Thanks,
Gao Xiang
Thanks,
Yifan Zhao
On 4/8/24 5:16 PM, Noboru Asai wrote:
In multi-threaded mode, each thread must use a different buffer in
tryrecompress_trailing
function, so change this buffer to non static.
Signed-off-by: Noboru Asai
On 2024/4/8 17:16, Noboru Asai wrote:
In multi-threaded mode, each thread must use a different buffer in
tryrecompress_trailing
function, so change this buffer to non static.
Signed-off-by: Noboru Asai
Thanks for catching this!
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
] https://summer-ospp.ac.cn/org/orgdetail/192db666-dfa0-46d9-9f58-9d8d165e2914
[2] https://erofs.docs.kernel.org
Thanks,
Gao Xiang
% (33.040s->2.861s) as well as the new image size is
decreased by 0.0094% (120 KiB), which is minor.
Signed-off-by: Gao Xiang
---
lib/inode.c | 192 +---
1 file changed, 91 insertions(+), 101 deletions(-)
diff --git a/lib/inode.c b/lib/inode.c
index f419
From: Yifan Zhao
The following new helpers are added to prepare for the upcoming
multi-threaded inter-file compression:
- erofs_mkfs_handle_{non,}directory;
- erofs_write_unencoded_file.
Signed-off-by: Yifan Zhao
Signed-off-by: Gao Xiang
---
lib/inode.c | 106
unk
returns erofs_holechunk.
Does it sound a good idea to you?
Thanks,
Gao Xiang
if (IS_ERR(chunk)) {
ret = PTR_ERR(chunk);
multiple places, put the logic
in helper function for avoiding repetition and easier reading.
Fixes: 7b46f7a0160a (erofs-utils: lib: merge consecutive chunks if possible)
Signed-off-by: Sandeep Dhavale
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
*minextblks = lb;
}
could be better (or just my own perference) since no input + output on a same
variable, otherwise it looks good to me.
If you agree on this minor nit, I could revise it directly or you could send
another version. Both are fine with me :)
Thanks,
Gao Xiang
+
int erofs_bl
on the benchmark logs, 64 pages are sufficient for 95% of
scenarios. This value can be adjusted from the module parameter.
The default value is 0.
This pool is currently only used for the LZ4 decompressor, but it can be
applied to more decompressors if needed.
Signed-off-by: Chunhai Guo
Reviewed-by: Gao Xiang
erofs_rsv_nrpages = 0;
}
Thanks,
Gao Xiang
On 2024/4/2 17:27, Chunhai Guo wrote:
Let's use alloc_pages_bulk_array() for simplicity and get rid of
unnecessary pagepool.
Signed-off-by: Chunhai Guo
---
Would you mind adding a changelog here next time? no
matter how short the text is.
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
fs
_pages[j]);
+ kfree(tmp_pages);
+ }
mutex_unlock(_resize_mutex);
return ret;
return i < z_erofs_gbuf_count ? -ENOMEM: 0;
Otherwise it looks good to me.
Thanks,
Gao Xiang
}
with hundreds of CPUs, and only a small
number of CPUs concurrently decompress most of the time.
This patch renames it as 'global buffer pool' and makes it configurable.
This allows two or more CPUs to share a common buffer to reduce memory
occupation.
Suggested-by: Gao Xiang
Reviewed-by: Gao
Opaque dir flag is needed if the tar tree is used immediately for
the upcoming append mode.
Signed-off-by: Gao Xiang
---
lib/tar.c | 5 +
1 file changed, 5 insertions(+)
diff --git a/lib/tar.c b/lib/tar.c
index 7c271f6..b45657d 100644
--- a/lib/tar.c
+++ b/lib/tar.c
@@ -909,6 +909,11
On 2024/4/1 21:55, Chunhai Guo wrote:
Currently, utils.c is only useful if CONFIG_EROFS_FS_ZIP is on.
So let's rename as zutil.c as well as avoid its inclusion if
CONFIG_EROFS_FS_ZIP is explicitly disabled.
Signed-off-by: Chunhai Guo
Suggested-by: Gao Xiang
Looks good to me,
Reviewed
Hi Linus,
Sorry about missing a title due to my careless. My apologizes...
On Thu, Mar 28, 2024 at 10:54:40AM +0800, Gao Xiang wrote:
> Could you consider this pull request for 6.9-rc2?
>
> Just two minor informative patches. One adds Sandeep Dhavale as a new
> reviewer who could
take a bit more time on this.
The other patch drops experimental warning for FSDAX since it's already
used for memory sharing between hosts and guests.
All commits have been in -next and no potential merge conflict is
observed.
Thanks,
Gao Xiang
The following changes since commit
On 2024/3/27 17:12, Luke Shumaker wrote:
From: "Luke T. Shumaker"
---
doc/magic.man | 4 ++--
Does this send to a wrong mailing list?
Thanks,
Gao Xiang
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/magic.man b/doc/magic.man
index b9845f5c..2ee28
The return value of `lseek(fd, fpos, SEEK_SET)` can overflow the `int`
type. Fix this.
Fixes: 376fb2dbe66d ("erofs-utils: lib: introduce diskbuf")
Signed-off-by: Gao Xiang
---
lib/inode.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/lib/inode.c b/lib/ino
As EXT4/XFS filesystems, FSDAX functionality is considered to be stable.
Let's drop this warning.
Signed-off-by: Gao Xiang
---
fs/erofs/super.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/fs/erofs/super.c b/fs/erofs/super.c
index 6fbb1fba2d31..fc60a5a7794f 100644
--- a/fs/erofs/super.c
Hi,
On Sun, Mar 24, 2024 at 06:56:06PM -0400, Sasha Levin wrote:
> From: Jan Kara
>
> [ Upstream commit 49845720080dff0afd5813eaebf0758b01b6312c ]
>
> Convert erofs to use bdev_open_by_path() and pass the handle around.
>
> CC: Gao Xiang
> CC: Chao Yu
> CC: l
-by: Gao Xiang
---
Side note: it'd be better to have a changelog here against the previous
version next time.
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
Suggested-by: Gao Xiang
---
...
diff --git a/fs/erofs/utils.c b/fs/erofs/utils.c
index e146d09151af..7b552bb8c36e 100644
--- a/fs/erofs/utils.c
+++ b/fs/erofs/utils.c
@@ -284,4 +284,152 @@ void erofs_exit_shrinker(void)
{
shrinker_free(erofs_shrinker_info);
}
+
+struct z_erofs_gbuf
.. Instead of reporting IO errors which implies a corrupted image.
Fixes: 6894ca9623e7 ("erofs-utils: mkfs: Support tar source without data")
Signed-off-by: Gao Xiang
---
include/erofs/blobchunk.h | 1 +
lib/blobchunk.c | 41 +++
buffer to reduce memory
occupation.
Signed-off-by: Chunhai Guo
Suggested-by: Gao Xiang
---
fs/erofs/Makefile | 2 +-
fs/erofs/decompressor.c | 6 +-
fs/erofs/internal.h | 14 ++--
fs/erofs/pcpubuf.c | 148 --
fs/erofs/super.c| 4
will land native solutions for dataset and AI
models and we will evaluate this use cases formally.
Thanks,
Gao Xiang
Best,
Nicolas Granger
On 2024/3/21 20:34, Yifan Zhao wrote:
On 2024/3/21 10:07, Gao Xiang wrote:
On 2024/3/21 02:15, Huang Jianan wrote:
On 2024/3/17 22:41, Yifan Zhao wrote:
This patch allows parallelizing the compression process of different
files in mkfs. Specifically, a traverser thread traverses
On 2024/3/21 15:02, Noboru Asai wrote:
With -E(all-)fragments, pclustersize has a different value per segment,
so move it to struct z_erofs_compress_sctx.
Signed-off-by: Noboru Asai
Thanks, I think it's a good change for the later fragment multi-threaded
compression.
Thanks,
Gao XIang
erofs_mkfs_print_progressinfo(inode);
- err = erofs_mkfs_build_tree(inode, );
+ err = erofs_mkfs_build_tree(inode, , ismt);
if (err) {
root = ERR_PTR(err);
break;
}
+ if (!ismt) {
+ if (S_ISDIR(inode->i_mode)) {
+ inode->next_dirwrite = dumpdir;
+ dumpdir = inode;
+ } else {
+ erofs_iput(inode);
+ }
+#ifdef EROFS_MT_ENABLED
+ } else {
+ erofs_push_queue(z_erofs_mt_queue, );
+#endif
Many branches use EROFS_MT_ENABLED for isolation, how about:
#ifdef EROFS_MT_ENABLED
void erofs_push_queue(struct erofs_queue *q, void *elem);
#else
void erofs_push_queue(struct erofs_queue *q, void *elem) {};
#endif
BTW, apart from that, I don't quite like the erofs_queue naming
since it's quite ambiguous: a data structure or something else?
Thanks,
Gao Xiang
On 2024/3/21 02:15, Huang Jianan wrote:
On 2024/3/15 9:10, Gao Xiang wrote:
In order to prepare for multi-threaded decompression.
multi-threaded compression.
It already merged into -dev. I would not update this
since it's a minor stuff. Let's work on inter-file
stuffs.
Thanks,
Gao
#syz dup KMSAN: uninit-value in z_erofs_lz4_decompress (2)
On 2024/3/19 18:17, syzbot wrote:
Hello,
syzbot has tested the proposed patch and the reproducer did not trigger any
issue:
Reported-and-tested-by: syzbot+88ad8b0517a9d3bb9...@syzkaller.appspotmail.com
Tested on:
commit:
#syz test git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
master
another try with the current upstream.
On 2024/2/20 15:22, syzbot wrote:
Hello,
syzbot tried to test the proposed patch but the build/boot failed:
le Layout Driver Registering...
[ 21.855653][T1] Key
Since pread() can be used during multi-threaded compression, it's
necessary to pass `fpos` in to indicate the absolute offset.
Fixes: aec8487dce4c ("erofs-utils: mkfs: introduce inner-file multi-threaded
compression")
Signed-off-by: Gao Xiang
---
include/erofs/compress.h |
On Sun, Mar 17, 2024 at 02:45:09PM +0800, Yifan Zhao wrote:
> If a segment is smaller than the block size, sizeof(sctx->membuf) should
> be at least as large as the block size, as memory write into the buffer
> is done in block size.
>
> Signed-off-by: Yifan Zhao
> ---
Folded the following diff
seg_ctx;
struct z_erofs_compress_sctx *sctx;
"ictx" means "inode context", "sctx" means "segment context".
If there is no confusion, "ctx" naming can be used in a function.
Also I tend to avoid making a huge diff to make ctx->"sctx or
ic
On 2024/3/15 10:39, Gao Xiang wrote:
On 2024/3/14 20:37, Yifan Zhao wrote:
Currently, the creation of EROFS compressed image creation is
single-threaded, which suffers from performance issues. This patch
attempts to address it by compressing the large file in parallel.
Specifically, each
On 2024/3/15 09:10, Gao Xiang wrote:
From: Yifan Zhao
Currently, the creation of EROFS compressed image creation is
single-threaded, which suffers from performance issues. This patch
attempts to address it by compressing the large file in parallel.
Specifically, each input file larger than
-by: Tong Xin
I did some updates yesterday and I just posted v7.
BTW, I also found an issue that the output cannot be stablized with
"mkfs.erofs -zlz4hc,12 --worker=72" and enwik9 dataset. I'm still
looking into that since it's an unexpected behavior.
Thanks,
Gao Xiang
-by: Gao Xiang
---
v7:
- support -Eztailpacking;
- wq_private -> wq_tls;
- minor updates.
include/erofs/compress.h | 3 +-
lib/compress.c | 548 ---
lib/compressor.c | 2 +
mkfs/main.c | 8 +-
4 files changed, 464 inserti
`__warnonce` atomic booleans to each erofs_compressor_init().
Cc: Yifan Zhao
Signed-off-by: Yifan Zhao
Signed-off-by: Gao Xiang
---
include/erofs/atomic.h | 28
lib/compressor_deflate.c| 11 ---
lib/compressor_libdeflate.c | 6 +-
lib
-off-by: Yifan Zhao
Signed-off-by: Gao Xiang
---
include/erofs/config.h | 4
lib/config.c | 4
mkfs/main.c| 23 +++
3 files changed, 31 insertions(+)
diff --git a/include/erofs/config.h b/include/erofs/config.h
index 73e3ac2..d2f91ff 100644
From: Yifan Zhao
Add a workqueue implementation for multi-threading support inspired by
xfsprogs.
Signed-off-by: Yifan Zhao
Suggested-by: Gao Xiang
Signed-off-by: Gao Xiang
---
configure.ac | 16 +
include/erofs/internal.h | 3 +
include/erofs/workqueue.h | 34
In order to prepare for multi-threaded decompression.
Signed-off-by: Yifan Zhao
Signed-off-by: Gao Xiang
---
configure.ac | 1 +
include/erofs/config.h | 1 +
lib/config.c | 12
3 files changed, 14 insertions(+)
diff --git a/configure.ac b/configure.ac
index
1 - 100 of 3458 matches
Mail list logo
