Re: [PATCH 2/2] erofs-utils: lib: improve freeing hashmap in erofs_blob_exit()

2024-05-24 Thread Sandeep Dhavale via Linux-erofs
On Thu, May 23, 2024 at 10:28 PM Gao Xiang wrote: > > Hi Sandeep, > Hi Gao, > On 2024/5/24 05:01, Sandeep Dhavale wrote: > > Depending on size of the filesystem being built there can be huge number > > of elements in the hashmap. Currently we call hashmap_iter_first() in > > while loop to iterate

[PATCH 0/2] erofs-utils: improve performance of mkfs with dedupe

2024-05-23 Thread Sandeep Dhavale via Linux-erofs
I got a report from AOSP user that performance of mkfs.erofs with dedupe option that mkfs.erofs time increased to very high number. For example creation of 8GB uncompressed erofs image increased from 36seconds to 27minutes when dedupe was enabled. After profiling mkfs.erofs for sample data, I

[PATCH 2/2] erofs-utils: lib: improve freeing hashmap in erofs_blob_exit()

2024-05-23 Thread Sandeep Dhavale via Linux-erofs
Depending on size of the filesystem being built there can be huge number of elements in the hashmap. Currently we call hashmap_iter_first() in while loop to iterate and free the elements. However technically correct, this is inefficient in 2 aspects. - As we are iterating the elements for

[PATCH 1/2] erofs-utils: lib: provide helper to disable hashmap shrinking

2024-05-23 Thread Sandeep Dhavale via Linux-erofs
This helper sets hasmap.shrink_at to 0. This is helpful to iterate over hashmap using hashmap_iter_next() and use hashmap_remove() in single pass efficeintly. Signed-off-by: Sandeep Dhavale --- include/erofs/hashmap.h | 4 1 file changed, 4 insertions(+) diff --git

Re: [PATCH] erofs: avoid allocating DEFLATE streams before mounting

2024-05-20 Thread Sandeep Dhavale via Linux-erofs
On Mon, May 20, 2024 at 2:06 AM Gao Xiang wrote: > > Currently, each DEFLATE stream takes one 32 KiB permanent internal > window buffer even if there is no running instance which uses DEFLATE > algorithm. > > It's unexpected and wasteful on embedded devices with limited resources > and servers

[PATCH v3] erofs-utils: dump: print filesystem blocksize

2024-04-17 Thread Sandeep Dhavale via Linux-erofs
mkfs.erofs supports creating filesystem images with different blocksizes. Add filesystem blocksize in super block dump so its easier to inspect the filesystem. The field is added after FS magic, so the output now looks like: Filesystem magic number: 0xE0F5E1E2 Filesystem

[PATCH v3] erofs-utils: lib: treat data blocks filled with 0s as a hole

2024-04-17 Thread Sandeep Dhavale via Linux-erofs
Add optimization to treat data blocks filled with 0s as a hole. Even though diskspace savings are comparable to chunk based or dedupe, having no block assigned saves us redundant disk IOs during read. To detect blocks filled with zeros during chunking, we insert block filled with zeros

[PATCH v2] erofs-utils: dump: print filesystem blocksize

2024-04-15 Thread Sandeep Dhavale via Linux-erofs
mkfs.erofs supports creating filesystem images with different blocksizes. Add filesystem blocksize in super block dump so its easier to inspect the filesystem. The field is added after FS magic, so the output now looks like: Filesystem magic number: 0xE0F5E1E2 Filesystem

Re: [PATCH v1] erofs-utils: dump: print filesystem blocksize

2024-04-12 Thread Sandeep Dhavale via Linux-erofs
On Fri, Apr 12, 2024 at 5:09 PM Gao Xiang wrote: > > Hi Sandeep, > > On 2024/4/13 06:51, Sandeep Dhavale wrote: > > mkfs.erofs supports creating filesystem images with different > > blocksizes. Add filesystem blocksize in super block dump so > > its easier to inspect the filesystem. > > > > The

Re: [PATCH v2] erofs-utils: lib: treat data blocks filled with 0s as a hole

2024-04-12 Thread Sandeep Dhavale via Linux-erofs
On Fri, Apr 12, 2024 at 5:07 PM Gao Xiang wrote: > > Hi Sandeep, > > On 2024/4/10 06:14, Sandeep Dhavale wrote: > > Add optimization to treat data blocks filled with 0s as a hole. > > Even though diskspace savings are comparable to chunk based or dedupe, > > having no block assigned saves us

[PATCH v1] erofs-utils: dump: print filesystem blocksize

2024-04-12 Thread Sandeep Dhavale via Linux-erofs
mkfs.erofs supports creating filesystem images with different blocksizes. Add filesystem blocksize in super block dump so its easier to inspect the filesystem. The filed is added at last, so the output now looks like: Filesystem magic number: 0xE0F5E1E2 Filesystem blocks:

Re: [PATCH] erofs-utils: lib: fix tarerofs 32-bit overflows

2024-04-11 Thread Sandeep Dhavale via Linux-erofs
Looks good. Reviewed-by: Sandeep Dhavale Thanks, Sandeep.

[PATCH v2] erofs-utils: lib: treat data blocks filled with 0s as a hole

2024-04-09 Thread Sandeep Dhavale via Linux-erofs
Add optimization to treat data blocks filled with 0s as a hole. Even though diskspace savings are comparable to chunk based or dedupe, having no block assigned saves us redundant disk IOs during read. To detect blocks filled with zeros during chunking, we insert block filled with zeros

Re: [PATCH v1] erofs: use raw_smp_processor_id() to get buffer from global buffer pool

2024-04-08 Thread Sandeep Dhavale via Linux-erofs
> > Thanks for catching this, since the original patch is > for next upstream cycle, may I fold this fix in the > original patch? > Hi Gao, Sounds good. As the fix is simple, it makes sense to fold it into the original one. Thanks, Sandeep.

[PATCH v1] erofs: use raw_smp_processor_id() to get buffer from global buffer pool

2024-04-08 Thread Sandeep Dhavale via Linux-erofs
erofs will decompress in the preemptible context (kworker or per cpu thread). As smp_processor_id() cannot be used in preemptible contexts, use raw_smp_processor_id() instead to index into global buffer pool. Reported-by: syzbot+27cc650ef45b379df...@syzkaller.appspotmail.com Fixes: 7a7513292cc6

Re: [PATCH 1/1] erofs-utils: lib: treat data blocks filled with 0s as a hole

2024-04-04 Thread Sandeep Dhavale via Linux-erofs
On Thu, Apr 4, 2024 at 7:00 AM Gao Xiang wrote: > > Hi Sandeep, > > On 2024/4/4 07:57, Sandeep Dhavale wrote: > > Add optimization to treat data blocks filled with 0s as a hole. > > Even though diskspace savings are comparable to chunk based or dedupe, > > having no block assigned saves us

[PATCH 1/1] erofs-utils: lib: treat data blocks filled with 0s as a hole

2024-04-03 Thread Sandeep Dhavale via Linux-erofs
Add optimization to treat data blocks filled with 0s as a hole. Even though diskspace savings are comparable to chunk based or dedupe, having no block assigned saves us redundant disk IOs during read. This patch detects if the block is filled with zeros and marks chunk as erofs_holechunk so there

[PATCH 0/1] Opportunistically making files sparse

2024-04-03 Thread Sandeep Dhavale via Linux-erofs
Hi, We noticed that in android if you build erofs images with ELFs which have higher alignment say 16K or 64K, there was a considerable increase in the size of the uncompressed erofs image. The size increase could be mitigated with -Ededupe or --chunksize=4096 but that still results in lot of

[PATCH v2] erofs-utils: lib: Fix calculation of minextblks when working with sparse files

2024-04-03 Thread Sandeep Dhavale via Linux-erofs
When we work with sparse files (files with holes), we need to consider when the contiguous data block starts after each hole to correctly calculate minextblks so we can merge consecutive chunks later. Now that we need to recalculate minextblks multiple places, put the logic in helper function for

Re: [PATCH v1 1/1] erofs-utils: lib: Fix calculation of minextblks when working with sparse files

2024-04-03 Thread Sandeep Dhavale via Linux-erofs
Hi Gao, Thank you for the review. I will update and send the v2. Thanks, Sandeep.

[PATCH v1 1/1] erofs-utils: lib: Fix calculation of minextblks when working with sparse files

2024-04-03 Thread Sandeep Dhavale via Linux-erofs
When we work with sparse files (files with holes), we need to consider when the contiguous data block starts after each hole to correctly calculate minextblks so we can merge consecutive chunks later. Now that we need to recalculate minextblks multiple places, put the logic in helper function for

[PATCH v1 0/1] erofs-utils: fix handling of sparse files

2024-04-03 Thread Sandeep Dhavale via Linux-erofs
Hi, While working on a enhancement where blocks filled with zeros can be treated as a hole (sparse file), I stumbled upon a bug. I found during mkfs.erofs, the utility handles the sparse files by looking at the holes and marking them as a `erofs_holechunk`. However the calculation of minextblks

[PATCH] MAINTAINERS: erofs: add myself as reviewer

2024-03-14 Thread Sandeep Dhavale via Linux-erofs
I have been contributing to erofs for sometime and I would like to help with code reviews as well. Signed-off-by: Sandeep Dhavale --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index 4f298c4187fb..b130340d71bb 100644 --- a/MAINTAINERS +++

Re: [PATCH] erofs: fix uninitialized page cache reported by KMSAN

2024-03-04 Thread Sandeep Dhavale via Linux-erofs
On Sun, Mar 3, 2024 at 7:54 PM Gao Xiang wrote: > > syzbot reports a KMSAN reproducer [1] which generates a crafted > filesystem image and causes IMA to read uninitialized page cache. > > Later, (rq->outputsize > rq->inputsize) will be formally supported > after either large uncompressed

[PATCH v2] erofs: fix refcount on the metabuf used for inode lookup

2024-02-21 Thread Sandeep Dhavale via Linux-erofs
In erofs_find_target_block() when erofs_dirnamecmp() returns 0, we do not assign the target metabuf. This causes the caller erofs_namei()'s erofs_put_metabuf() at the end to be not effective leaving the refcount on the page. As the page from metabuf (buf->page) is never put, such page cannot be

Re: [PATCH v1] erofs: fix refcount on the metabuf used for inode lookup

2024-02-20 Thread Sandeep Dhavale via Linux-erofs
> > If it looks good to you, could you resend a formal patch? Thanks! > Hi Gao, This looks better and more readable. I will send a v2. Thanks, Sandeep. > Thanks, > Gao Xiang

[PATCH v1] erofs: fix refcount on the metabuf used for inode lookup

2024-02-20 Thread Sandeep Dhavale via Linux-erofs
In erofs_find_target_block() when erofs_dirnamecmp() returns 0, we do not assign the target metabuf. This causes the caller erofs_namei()'s erofs_put_metabuf() at the end to be not effective leaving the refcount on the page. As the page from metabuf (buf->page) is never put, such page cannot be

Re: [PATCH] erofs: fix infinite loop due to a race of filling compressed_bvecs

2024-01-25 Thread Sandeep Dhavale via Linux-erofs
On Thu, Jan 25, 2024 at 4:01 AM Gao Xiang wrote: > > I encountered a race issue after lengthy (~594647 sec) stress tests on > a 64k-page arm64 VM with several 4k-block EROFS images. The timing > is like below: > > z_erofs_try_inplace_io z_erofs_fill_bio_vec >

Re: [PATCH v1] erofs-utils: Fix cross compile with autoconf

2023-10-07 Thread Sandeep Dhavale via Linux-erofs
On Sat, Oct 7, 2023 at 1:25 AM Gao Xiang wrote: > > > > On 2023/10/7 15:16, Sandeep Dhavale wrote: > > On Fri, Oct 6, 2023 at 8:27 PM Gao Xiang > > wrote: > >> > >> Hi Sandeep! > >> > >> On 2023/10/6 06:40, Sandeep Dhavale wrote: > >>> AC_RUN_IFELSE expects the action if cross compiling. If not

Re: [PATCH v1] erofs-utils: Fix cross compile with autoconf

2023-10-07 Thread Sandeep Dhavale via Linux-erofs
On Fri, Oct 6, 2023 at 8:27 PM Gao Xiang wrote: > > Hi Sandeep! > > On 2023/10/6 06:40, Sandeep Dhavale wrote: > > AC_RUN_IFELSE expects the action if cross compiling. If not provided > > cross compilation fails with error "configure: error: cannot run test > > program while cross compiling". > >

[PATCH v1] erofs-utils: Fix cross compile with autoconf

2023-10-05 Thread Sandeep Dhavale via Linux-erofs
AC_RUN_IFELSE expects the action if cross compiling. If not provided cross compilation fails with error "configure: error: cannot run test program while cross compiling". Use 4096 as the buest guess PAGESIZE if cross compiling as it is still the most common page size. Reported-in:

Re: [PATCH v1] erofs-utils: lib: Restore memory address before free()

2023-09-19 Thread Sandeep Dhavale via Linux-erofs
On Tue, Sep 19, 2023 at 7:07 PM Gao Xiang wrote: > > Hi Sandeep, > > On 2023/9/20 05:02, Sandeep Dhavale wrote: > > We move `idx` pointer as we iterate through for loop based on `count`. If > > we error out from the loop, restore the pointer to allocated memory > > before calling free(). > > > >

[PATCH v1] erofs-utils: lib: Restore memory address before free()

2023-09-19 Thread Sandeep Dhavale via Linux-erofs
We move `idx` pointer as we iterate through for loop based on `count`. If we error out from the loop, restore the pointer to allocated memory before calling free(). Fixes: 39147b48b76d ("erofs-utils: lib: add erofs_rebuild_load_tree() helper") Signed-off-by: Sandeep Dhavale --- lib/rebuild.c |

[PATCH v1 7/7] erofs-utils: lib: tar: Initialize the variable to avoid using garbage value

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
The value in variable 'e' is checked without initializing and can wrongly signify end of tar 2 empty blocks. Signed-off-by: Sandeep Dhavale --- lib/tar.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/tar.c b/lib/tar.c index b58bfd5..c4c89ec 100644 --- a/lib/tar.c +++

[PATCH v1 6/7] erofs-utils: lib: Remove redundant assignment

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
The intended assignment is already part of the next for loop initialization. Signed-off-by: Sandeep Dhavale --- lib/namei.c | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/namei.c b/lib/namei.c index 45dbcd3..294d7a3 100644 --- a/lib/namei.c +++ b/lib/namei.c @@ -262,7 +262,6 @@ static

[PATCH v1 5/7] erofs-utils: lib: Fix the memory leak in error path

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
If call to inflateInit2() fails, release the memory allocated for buff before returning. Signed-off-by: Sandeep Dhavale --- lib/decompress.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/decompress.c b/lib/decompress.c index 01f0141..fe8a40c 100644 ---

[PATCH v1 4/7] erofs-utils: lib: Check for error from z_erofs_pack_file_from_fd()

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
If z_erofs_pack_file_from_fd() fails, take the error path to free up the allocated resources. Signed-off-by: Sandeep Dhavale --- lib/compress.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/compress.c b/lib/compress.c index 8c79418..81f277a 100644 --- a/lib/compress.c +++

[PATCH v1 3/7] erofs-utils: lib: Fix memory leak if __erofs_battach() fails

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
Need to free allocated buffer_head bh if __erofs_battach() fails. Signed-off-by: Sandeep Dhavale --- lib/cache.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/cache.c b/lib/cache.c index 5205d57..a9948f0 100644 --- a/lib/cache.c +++ b/lib/cache.c @@ -282,8 +282,10

[PATCH v1 2/7] erofs-utils: lib: Remove redundant line to get padding

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
padding is only used in the next if() block. Remove the redundant call. Signed-off-by: Sandeep Dhavale --- lib/blobchunk.c | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/blobchunk.c b/lib/blobchunk.c index 86b29c1..fcc71b8 100644 --- a/lib/blobchunk.c +++ b/lib/blobchunk.c @@ -91,7

[PATCH v1 1/7] erofs-utils: fsck: Fix potential memory leak in error path

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
Free the memory allocated for 'entry' if strdup() fails. Signed-off-by: Sandeep Dhavale --- fsck/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fsck/main.c b/fsck/main.c index 7f78513..3f86da4 100644 --- a/fsck/main.c +++ b/fsck/main.c @@ -585,8 +585,10 @@ static

[PATCH v1 0/7] Misc fixes in erofs-utils

2023-09-13 Thread Sandeep Dhavale via Linux-erofs
Hi, Below series contains few small misc fixes found after running through clang static checker. Sandeep Dhavale (7): erofs-utils: fsck: Fix potential memory leak in error path erofs-utils: lib: Remove redundant line to get padding erofs-utils: lib: Fix memory leak if __erofs_battach()

[PATCH v1] erofs-utils: fsck: fix support for 16k block size

2023-09-12 Thread Sandeep Dhavale via Linux-erofs
This basically follows the fix in kernel commit 001b8ccd0650 (erofs: fix compact 4B support for 16k block size). Without this patch fsck on images with 16k block size reports corruption which is not correct. Fixes: 76b822726ff8 ("erofs-utils: introduce compacted compression indexes")

Re: [PATCH v1 1/2] erofs-utils: Relax the hardchecks on the blocksize

2023-08-30 Thread Sandeep Dhavale via Linux-erofs
On Wed, Aug 30, 2023 at 4:34 PM Gao Xiang wrote: > > Hi Sandeep, > > On 2023/8/31 07:16, Sandeep Dhavale wrote: > > As erofs-utils supports different block sizes upto > > EROFS_MAX_BLOCK_SIZE, relax the checks so same tools > > can be used to create images for platforms where > > page size can be

[PATCH v1 2/2] erofs-utils: Set mkfs default blocksize based on current platform

2023-08-30 Thread Sandeep Dhavale via Linux-erofs
Set mkfs default blocksize to current platform pagesize. This means mkfs with default options will work on current platform. If we are building image for a platform for a different blocksize, we can override default with -b option up to EROFS_MAX_BLOCK_SIZE. Signed-off-by: Sandeep Dhavale ---

[PATCH v1 1/2] erofs-utils: Relax the hardchecks on the blocksize

2023-08-30 Thread Sandeep Dhavale via Linux-erofs
As erofs-utils supports different block sizes upto EROFS_MAX_BLOCK_SIZE, relax the checks so same tools can be used to create images for platforms where page size can be greater than 4096. Signed-off-by: Sandeep Dhavale --- lib/namei.c | 2 -- mkfs/main.c | 9 + 2 files changed, 5

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-14 Thread Sandeep Dhavale via Linux-erofs
> > Sandeep, thoughts? > I prefer to modify erofs check and contain the fix there as erofs cares about it and it's least invasive. For contexts where erofs cannot tell for sure, it will always schedule kworker (like CONFIG_PREEMPT_COUNT=n). I will also do measurements to see if erofs should

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-13 Thread Sandeep Dhavale via Linux-erofs
> > Sorry, but the current lockdep-support functions need to stay focused > on lockdep. They are not set up for general use, as we already saw > with rcu_is_watching(). > Ok, understood. > If you get a z_erofs_wq_needed() (or whatever) upstream, and if it turns > out that there is an

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-13 Thread Sandeep Dhavale via Linux-erofs
> Thank you for the background. > > > Paul, Joel, > > Shall we fix the rcu_read_lock_*held() regardless of how erofs > > improves the check? > > Help me out here. Exactly what is broken with rcu_read_lock_*held(), > keeping in mind their intended use for lockdep-based debugging? > Hi Paul, With

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-13 Thread Sandeep Dhavale via Linux-erofs
Hello All, Let me answer some of the questions raised here. * Performance aspect EROFS is one of the popular filesystem of choice in Android for read only partitions as it provides 30%+ storage space savings with compression. In addition to microbenchmarks, boot times and cold app launch

Re: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-12 Thread Sandeep Dhavale via Linux-erofs
Hi Joel, Thank you for the review! > Actually even the original code is incorrect (the lockdep version) > since preemptible() cannot be relied upon if CONFIG_PREEMPT_COUNT is > not set. However, that's debug code. In this case, it is a real > user (the fs code). In non-preemptible kernels, we are

[PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC

2023-07-11 Thread Sandeep Dhavale via Linux-erofs
Currently if CONFIG_DEBUG_LOCK_ALLOC is not set - rcu_read_lock_held() always returns 1 - rcu_read_lock_any_held() may return 0 with CONFIG_PREEMPT_RCU This is inconsistent and it was discovered when trying a fix for problem reported [1] with CONFIG_DEBUG_LOCK_ALLOC is not set and

[PATCH v1] erofs: Fix detection of atomic context

2023-06-21 Thread Sandeep Dhavale via Linux-erofs
Current check for atomic context is not sufficient as z_erofs_decompressqueue_endio can be called under rcu lock from blk_mq_flush_plug_list(). See the stacktrace [1] In such case we should hand off the decompression work for async processing rather than trying to do sync decompression in current

Re: EROFS: Detecting atomic contexts

2023-06-21 Thread Sandeep Dhavale via Linux-erofs
On Tue, Jun 20, 2023 at 11:23 PM Gao Xiang wrote: > > > > On 2023/6/21 14:18, Sandeep Dhavale wrote: > > On Tue, Jun 20, 2023 at 5:38 PM Gao Xiang > > wrote: > > ... > > >> > > I think this looks good. rcu_read_lock_any_held() can detect this. > > Okay, could you submit a formal patch with

Re: EROFS: Detecting atomic contexts

2023-06-21 Thread Sandeep Dhavale via Linux-erofs
On Tue, Jun 20, 2023 at 5:38 PM Gao Xiang wrote: > > Hi Sandeep, > > On 2023/6/21 04:38, Sandeep Dhavale wrote: > > Hi, > > I think we are under RCU read lock in the stack at > > blk_mq_flush_plug_list+0x2b0/0x354 > > > > blk_mq_flush_plug_list calls blk_mq_run_dispatch_ops > > which is a macro

Re: EROFS: Detecting atomic contexts

2023-06-20 Thread Sandeep Dhavale via Linux-erofs
Hi, I think we are under RCU read lock in the stack at blk_mq_flush_plug_list+0x2b0/0x354 blk_mq_flush_plug_list calls blk_mq_run_dispatch_ops which is a macro in block/blk-mq.h /* run the code block in @dispatch_ops with rcu/srcu read lock held */ #define __blk_mq_run_dispatch_ops(q,

EROFS: Detecting atomic contexts

2023-06-14 Thread Sandeep Dhavale via Linux-erofs
Hi, One of the partners reported that they are seeing the warning log which points out that the sleeping function is being called from invalid context. We have not seen this in internal testing and the partner does not have a repro yet. Below is the log. [17185.137395] [T701615] CpuMonitorServi:

Re: [PATCH] erofs: use HIPRI by default if per-cpu kthreads are enabled

2023-05-22 Thread Sandeep Dhavale via Linux-erofs
On Mon, May 22, 2023 at 2:21 AM Gao Xiang wrote: > > As Sandeep shown [1], high priority RT per-cpu kthreads are > typically helpful for Android scenarios to minimize the scheduling > latencies. > > Switch EROFS_FS_PCPU_KTHREAD_HIPRI on by default if > EROFS_FS_PCPU_KTHREAD is on since it's the

Re: [PATCH v5] erofs: add per-cpu threads for decompression as an option

2023-02-27 Thread Sandeep Dhavale via Linux-erofs
On Mon, Feb 27, 2023 at 9:01 PM Gao Xiang wrote: > > Hi Sandeep, > > On 2023/2/28 12:47, Sandeep Dhavale via Linux-erofs wrote: > > Hi all, > > I completed the tests and the results are consistent with > > our previous observation. We can see that removing WQ_UNBOUN

Re: [PATCH v5] erofs: add per-cpu threads for decompression as an option

2023-02-27 Thread Sandeep Dhavale via Linux-erofs
Hi all, I completed the tests and the results are consistent with our previous observation. We can see that removing WQ_UNBOUND helps but the scheduling latency by using high priority per cpu kthreads is even lower. Below is the table. |-+---+---+--+---| |

Re: [PATCH v5] erofs: add per-cpu threads for decompression as an option

2023-02-23 Thread Sandeep Dhavale via Linux-erofs
On Thu, Feb 23, 2023 at 11:08 AM Gao Xiang wrote: > > > > On 2023/2/24 02:52, Gao Xiang wrote: > > Hi Eric, > > > > On 2023/2/24 02:29, Eric Biggers wrote: > >> Hi, > >> > >> On Wed, Feb 08, 2023 at 05:33:22PM +0800, Gao Xiang wrote: > >>> From: Sandeep Dhavale > >>> > >>> Using per-cpu thread

Re: [PATCH v4] erofs: replace erofs_unzipd workqueue with per-cpu threads

2023-02-07 Thread Sandeep Dhavale via Linux-erofs
On Mon, Feb 6, 2023 at 6:55 PM Gao Xiang wrote: > > > > On 2023/2/7 03:41, Sandeep Dhavale wrote: > > On Mon, Feb 6, 2023 at 2:01 AM Gao Xiang wrote: > >> > >> Hi Sandeep, > >> > >> On Fri, Jan 06, 2023 at 07:35:01AM +, Sandeep Dhavale wrote: > >>> Using per-cpu thread pool we can reduce the

Re: [PATCH v4] erofs: replace erofs_unzipd workqueue with per-cpu threads

2023-02-06 Thread Sandeep Dhavale via Linux-erofs
On Mon, Feb 6, 2023 at 2:01 AM Gao Xiang wrote: > > Hi Sandeep, > > On Fri, Jan 06, 2023 at 07:35:01AM +, Sandeep Dhavale wrote: > > Using per-cpu thread pool we can reduce the scheduling latency compared > > to workqueue implementation. With this patch scheduling latency and > > variation is

Re: [PATCH] workqueue: Add WQ_SCHED_FIFO

2023-01-18 Thread Sandeep Dhavale via Linux-erofs
On Sat, Jan 14, 2023 at 1:01 PM Nathan Huckleberry wrote: > > On Fri, Jan 13, 2023 at 6:20 PM Gao Xiang wrote: > > > > Hi Nathan! > > > > On 2023/1/14 05:07, Nathan Huckleberry wrote: > > > Add a WQ flag that allows workqueues to use SCHED_FIFO with the least > > > imporant RT priority. This

[PATCH v4] erofs: replace erofs_unzipd workqueue with per-cpu threads

2023-01-05 Thread Sandeep Dhavale via Linux-erofs
Using per-cpu thread pool we can reduce the scheduling latency compared to workqueue implementation. With this patch scheduling latency and variation is reduced as per-cpu threads are high priority kthread_workers. The results were evaluated on arm64 Android devices running 5.10 kernel. The

Re: [PATCH v3] erofs: replace erofs_unzipd workqueue with per-cpu threads

2023-01-05 Thread Sandeep Dhavale via Linux-erofs
On Thu, Jan 5, 2023 at 10:02 PM Gao Xiang wrote: > > Hi Sandeep, > > On 2023/1/6 13:41, Sandeep Dhavale wrote: > > Using per-cpu thread pool we can reduce the scheduling latency compared > > to workqueue implementation. With this patch scheduling latency and > > variation is reduced as per-cpu

[PATCH v3] erofs: replace erofs_unzipd workqueue with per-cpu threads

2023-01-05 Thread Sandeep Dhavale via Linux-erofs
Using per-cpu thread pool we can reduce the scheduling latency compared to workqueue implementation. With this patch scheduling latency and variation is reduced as per-cpu threads are high priority kthread_workers. The results were evaluated on arm64 Android devices running 5.10 kernel. The

[PATCH v2] erofs: replace erofs_unzipd workqueue with per-cpu threads

2023-01-05 Thread Sandeep Dhavale via Linux-erofs
Using per-cpu thread pool we can reduce the scheduling latency compared to workqueue implementation. With this patch scheduling latency and variation is reduced as per-cpu threads are high priority kthread_workers. The results were evaluated on arm64 Android devices running 5.10 kernel. The

Re: [PATCH] EROFS: Replace erofs_unzipd workqueue with per-cpu threads

2022-12-26 Thread Sandeep Dhavale via Linux-erofs
On Wed, Dec 21, 2022 at 6:48 PM Gao Xiang wrote: > > Hi Sandeen, > > On Thu, Dec 22, 2022 at 01:15:29AM +, Sandeep Dhavale wrote: > > Using per-cpu thread pool we can reduce the scheduling latency compared > > to workqueue implementation. With this patch scheduling latency and > > variation

[PATCH] EROFS: Replace erofs_unzipd workqueue with per-cpu threads

2022-12-21 Thread Sandeep Dhavale via Linux-erofs
Using per-cpu thread pool we can reduce the scheduling latency compared to workqueue implementation. With this patch scheduling latency and variation is reduced as per-cpu threads are SCHED_FIFO kthread_workers. The results were evaluated on arm64 Android devices running 5.10 kernel. The table