Re: [dm-devel] [PATCH blktests] dm: add a regression test

2023-04-25 Thread Mike Snitzer
On Tue, Apr 25 2023 at 8:15P -0400, Shin'ichiro Kawasaki wrote: > On Apr 25, 2023 / 16:22, Yu Kuai wrote: > > Hi, > > > > 在 2023/01/12 9:05, Shinichiro Kawasaki 写道: > > > Hello Yu, thanks for the patch. I think it is good to have this test case > > > to > > > avoid repeating the same

[dm-devel] [git pull] device mapper changes for 6.4

2023-04-24 Thread Mike Snitzer
dm: improve hash_locks sizing and hash function Li Lingfeng (1): dm: don't lock fs when the map is NULL in process of resume Michael Weiß (1): dm verity: emit audit events on verification failure and more Mike Snitzer (16): dm bufio: use WARN_ON in dm_bufio_client_destroy and

Re: [dm-devel] [PATCH v5-fix 1/5] block: Don't invalidate pagecache for invalid falloc modes

2023-04-20 Thread Mike Snitzer
On Wed, Apr 19 2023 at 9:47P -0400, Sarthak Kukreti wrote: > Only call truncate_bdev_range() if the fallocate mode is > supported. This fixes a bug where data in the pagecache > could be invalidated if the fallocate() was called on the > block device with an invalid mode. > > Fixes:

Re: [dm-devel] [PATCH v4 1/4] block: Introduce provisioning primitives

2023-04-19 Thread Mike Snitzer
On Wed, Apr 19 2023 at 11:36P -0400, Darrick J. Wong wrote: > On Tue, Apr 18, 2023 at 03:12:04PM -0700, Sarthak Kukreti wrote: > > Introduce block request REQ_OP_PROVISION. The intent of this request > > is to request underlying storage to preallocate disk space for the given > > block range.

Re: [dm-devel] block: fix a crash when bio_for_each_folio_all iterates over an empty bio

2023-04-18 Thread Mike Snitzer
re: https://listman.redhat.com/archives/dm-devel/2023-April/053916.html Reviewed-by: Mike Snitzer -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH v6 06/18] dm: Add support for block PR read keys/reservation

2023-04-17 Thread Mike Snitzer
On Fri, Apr 07 2023 at 4:05P -0400, Mike Christie wrote: > This adds support in dm for the block PR read keys and read reservation > callouts. > > Signed-off-by: Mike Christie Acked-by: Mike Snitzer -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman

Re: [dm-devel] Possible deadlock detected in Linux 6.2.0 in dm_get_inactive_table (dm-ioctl.c)

2023-04-17 Thread Mike Snitzer
ed is the report, log, and reproducers generated by syzkaller > Please let me know if there is any additional information that I can > provide to help debug this issue. > Thanks! Thanks for the report, I've staged this fix: From: Mike Snitzer Subject: [PATCH] dm ioctl: fix nested lo

Re: [dm-devel] [PATCH v3 2/3] dm: Add support for block provisioning

2023-04-14 Thread Mike Snitzer
On Thu, Apr 13 2023 at 8:02P -0400, Sarthak Kukreti wrote: > Add support to dm devices for REQ_OP_PROVISION. The default mode > is to passthrough the request to the underlying device, if it > supports it. dm-thinpool uses the provision request to provision > blocks for a dm-thin device.

Re: [dm-devel] [PATCH v3 2/3] dm: Add support for block provisioning

2023-04-14 Thread Mike Snitzer
On Fri, Apr 14 2023 at 9:32P -0400, Joe Thornber wrote: > On Fri, Apr 14, 2023 at 7:52 AM Sarthak Kukreti > wrote: > > > Add support to dm devices for REQ_OP_PROVISION. The default mode > > is to passthrough the request to the underlying device, if it > > supports it. dm-thinpool uses the

Re: [dm-devel] dm: Add error information printing for dm_register_target()

2023-04-04 Thread Mike Snitzer
On Sat, Mar 18 2023 at 9:16P -0400, Yangtao Li wrote: > Ensure that all error handling branches print error information. In this > way, when this function fails, the upper-layer functions can directly > return an error code without missing debugging information. Otherwise, > the error message

[dm-devel] [git pull] device mapper fixes for 6.3-rc5

2023-03-30 Thread Mike Snitzer
TduvU8ufKw== =RpDD -END PGP SIGNATURE- ---- Mike Snitzer (2): dm: fix improper splitting for abnormal bios dm: fix __send_duplicate_bios() to always allow for splitting IO drivers/md/dm.c | 9 ++--- 1 file changed, 6

Re: [dm-devel] [PATCH v5 01/18] block: Add PR callouts for read keys and reservation

2023-03-28 Thread Mike Snitzer
On Fri, Mar 24 2023 at 2:17P -0400, Mike Christie wrote: > Add callouts for reading keys and reservations. This allows LIO to support > the READ_KEYS and READ_RESERVATION commands and will allow dm-multipath > to optimize it's error handling so it can check if it's getting an error > because

Re: [dm-devel] dm-clone: Request option to send discard to source device during hydration

2023-03-28 Thread Mike Snitzer
On Mon, Mar 27 2023 at 4:24P -0400, Gwendal Grignou wrote: > On ChromeOS, we are working on migrating file backed loopback devices > to thinpool logical volumes using dm-clone on the Chromebook local > SSD. > Dm-clone hydration workflow is a great fit but the design of dm-clone > assumes a

[dm-devel] [dm-6.4 PATCH v3 15/20] dm bio prison v1: add dm_cell_key_has_valid_range

2023-03-27 Thread Mike Snitzer
in (having first imposed max_discard_granularity based splitting). dm_cell_key_has_valid_range() will merely WARN_ON_ONCE if it returns false because if it does: it is programmer error that should be caught with proper testing. So relax the BUG_ONs to be WARN_ON_ONCE. Signed-off-by: Mike Snitzer

[dm-devel] [dm-6.4 PATCH v3 19/20] dm bio prison v1: prepare to intelligently size dm_bio_prison's prison_regions

2023-03-27 Thread Mike Snitzer
Add num_locks member to dm_bio_prison struct and use it rather than the NR_LOCKS magic value (64). Next commit will size the dm_bio_prison's prison_regions according to dm_num_sharded_locks(). Signed-off-by: Mike Snitzer --- drivers/md/dm-bio-prison-v1.c | 21 +++-- 1 file

[dm-devel] [dm-6.4 PATCH v3 18/20] dm bufio: intelligently size dm_buffer_cache's buffer_trees

2023-03-27 Thread Mike Snitzer
Size the dm_buffer_cache's number of buffer_tree structs using dm_num_sharded_locks(). Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 2250799a70e4

[dm-devel] [dm-6.4 PATCH v3 17/20] dm bufio: prepare to intelligently size dm_buffer_cache's buffer_trees

2023-03-27 Thread Mike Snitzer
Add num_locks member to dm_buffer_cache struct and use it rather than the NR_LOCKS magic value (64). Next commit will size the dm_buffer_cache's buffer_trees according to dm_num_sharded_locks(). Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 48

[dm-devel] [dm-6.4 PATCH v3 20/20] dm bio prison v1: intelligently size dm_bio_prison's prison_regions

2023-03-27 Thread Mike Snitzer
Size the dm_bio_prison's number of prison_region structs using dm_num_sharded_locks(). Signed-off-by: Mike Snitzer --- drivers/md/dm-bio-prison-v1.c | 11 ++- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-bio-prison-v1.c b/drivers/md/dm-bio-prison-v1.c

[dm-devel] [dm-6.4 PATCH v3 16/20] dm: add dm_num_sharded_locks()

2023-03-27 Thread Mike Snitzer
, but as-is it will serve as a more informed basis for sizing the sharded lock structs in dm-bufio's dm_buffer_cache (buffer_trees) and dm-bio-prison-v1's dm_bio_prison (prison_regions). Signed-off-by: Mike Snitzer --- drivers/md/dm.h | 10 ++ 1 file changed, 10 insertions(+) diff --git

[dm-devel] [dm-6.4 PATCH v3 13/20] dm: split discards further if target sets max_discard_granularity

2023-03-27 Thread Mike Snitzer
'max_discard_granularity' flag that if set will cause DM core to split discard bios relative to 'max_discard_sectors'. This treats 'discard_granularity' as a "min_discard_granularity" and 'max_discard_sectors' as a "max_discard_granularity". Requested-by: Joe Thornber Signed-off-by: Mike S

[dm-devel] [dm-6.4 PATCH v3 12/20] dm thin: speed up cell_defer_no_holder()

2023-03-27 Thread Mike Snitzer
From: Joe Thornber Reduce the time that a spinlock is held in cell_defer_no_holder(). Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-thin.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-thin.c b/drivers/md/dm

[dm-devel] [dm-6.4 PATCH v3 14/20] dm bio prison v1: improve concurrent IO performance

2023-03-27 Thread Mike Snitzer
by updating dm-thin.c to set limits->max_discard_sectors in terms of BIO_PRISON_MAX_RANGE _and_ setting the thin and thin-pool targets' max_discard_granularity to true. Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-bio-prison-v1.c | 87 + driv

[dm-devel] [dm-6.4 PATCH v3 11/20] dm bufio: use multi-page bio vector

2023-03-27 Thread Mike Snitzer
From: Mikulas Patocka The kernel supports multi page bio vector entries, so we can use them in dm-bufio as an optimization. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 24 1 file changed, 4 insertions(+), 20 deletions

[dm-devel] [dm-6.4 PATCH v3 09/20] dm bufio: move dm_bufio_client members to avoid spanning cachelines

2023-03-27 Thread Mike Snitzer
Movement also consolidates holes in dm_bufio_client struct. But the overall size of the struct isn't changed. Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md

[dm-devel] [dm-6.4 PATCH v3 08/20] dm bufio: add lock_history optimization for cache iterators

2023-03-27 Thread Mike Snitzer
-by: Mike Snitzer --- drivers/md/dm-bufio.c | 119 +++--- 1 file changed, 111 insertions(+), 8 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 1e000ec73bd6..9ac50024006d 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c

[dm-devel] [dm-6.4 PATCH v3 07/20] dm bufio: improve concurrent IO performance

2023-03-27 Thread Mike Snitzer
in the cache then there's a big speed up. Eg, one test has 16 'hotspot' threads simulating btree lookups while another thread dirties the whole device. In this case the hotspot threads acquire the buffers about 25 times faster. Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer

[dm-devel] [dm-6.4 PATCH v3 10/20] dm bufio: use waitqueue_active in __free_buffer_wake

2023-03-27 Thread Mike Snitzer
From: Mikulas Patocka Save one spinlock by using waitqueue_active. We hold the bufio lock at this place, so no one can add entries to the waitqueue at this point. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 7 ++- 1 file changed, 6 insertions

[dm-devel] [dm-6.4 PATCH v3 03/20] dm bufio: never crash if dm_bufio_in_request()

2023-03-27 Thread Mike Snitzer
All these instances are entirely avoidable given that they speak to coding mistakes that result in inappropriate use. Proper testing during development will catch any such coding bug so its best to relax all of these from BUG_ON to WARN_ON_ONCE. Signed-off-by: Mike Snitzer --- drivers/md/dm

[dm-devel] [dm-6.4 PATCH v3 05/20] dm bufio: add LRU abstraction

2023-03-27 Thread Mike Snitzer
commits. As such, this commit will cause "defined but not used" compiler warnings that will be resolved by the next 2 commits. Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 235 ++ 1 file changed, 235

[dm-devel] [dm-6.4 PATCH v3 04/20] dm bufio: don't bug for clear developer oversight

2023-03-27 Thread Mike Snitzer
Reasonable to relax to WARN_ON because these are easily avoided but do offer some assurance future coding mistakes won't occur (if changes tested properly). Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 13 - 1 file changed, 8 insertions(+), 5 deletions(-) diff --git

[dm-devel] [dm-6.4 PATCH v3 06/20] dm bufio: add dm_buffer_cache abstraction

2023-03-27 Thread Mike Snitzer
members to allow compilation of this intermediate commit (they will be elided in the next commit). This commit will cause "defined but not used" compiler warnings that will be resolved by the next commit. Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-buf

[dm-devel] [dm-6.4 PATCH v3 01/20] dm bufio: remove unused dm_bufio_release_move interface

2023-03-27 Thread Mike Snitzer
From: Joe Thornber Was used by multi-snapshot DM target that never went upstream. Signed-off-by: Joe Thornber Acked-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c| 77 include/linux/dm-bufio.h | 6 2 files changed

[dm-devel] [dm-6.4 PATCH v3 02/20] dm bufio: use WARN_ON in dm_bufio_client_destroy and dm_bufio_exit

2023-03-27 Thread Mike Snitzer
Using BUG_ON when tearing down is excessive. Relax these to WARN_ONs. Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 79434b38f368..dac9f1f84c34 100644

[dm-devel] [dm-6.4 PATCH v3 00/20] dm bufio, thin: improve concurrent IO performance

2023-03-27 Thread Mike Snitzer
1: improve concurrent IO performance Mike Snitzer (11): dm bufio: use WARN_ON in dm_bufio_client_destroy and dm_bufio_exit dm bufio: never crash if dm_bufio_in_request() dm bufio: don't bug for clear developer oversight dm bufio: move dm_bufio_client members to avoid spanning cachelines

Re: [dm-devel] [dm-6.4 PATCH v2 3/9] dm bufio: improve concurrent IO performance

2023-03-24 Thread Mike Snitzer
On Fri, Mar 24 2023 at 7:11P -0400, Jens Axboe wrote: > On 3/24/23 4:53?PM, Mike Snitzer wrote: > > On Fri, Mar 24 2023 at 3:34P -0400, > > Jens Axboe wrote: > > > >> Just some random drive-by comments. > >> > >>> diff --git a/driver

Re: [dm-devel] [dm-6.4 PATCH v2 3/9] dm bufio: improve concurrent IO performance

2023-03-24 Thread Mike Snitzer
On Fri, Mar 24 2023 at 3:34P -0400, Jens Axboe wrote: > Just some random drive-by comments. > > > diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c > > index 1de1bdcda1ce..a58f8ac3ba75 100644 > > --- a/drivers/md/dm-bufio.c > > +++ b/drivers/md/dm-bufio.c > > +static void

[dm-devel] [dm-6.4 PATCH v2 8/9] dm: split discards further if target sets max_discard_granularity

2023-03-24 Thread Mike Snitzer
'max_discard_granularity' flag that if set will cause DM core to split discard bios relative to 'max_discard_sectors'. This treats 'discard_granularity' as a "min_discard_granularity" and 'max_discard_sectors' as a "max_discard_granularity". Requested-by: Joe Thornber Signed-off-by: Mike S

[dm-devel] [dm-6.4 PATCH v2 9/9] dm bio prison v1: improve concurrent IO performance

2023-03-24 Thread Mike Snitzer
by updating dm-thin.c to set limits->max_discard_sectors in terms of BIO_PRISON_MAX_RANGE _and_ setting the thin and thin-pool targets' max_discard_granularity to true. Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-bio-prison-v1.c | 87 --- driv

[dm-devel] [dm-6.4 PATCH v2 0/9] dm bufio, thin: improve concurrent IO performance

2023-03-24 Thread Mike Snitzer
mprove concurrent IO performance Mike Snitzer (3): dm bufio: move dm_buffer struct dm bufio: move dm_bufio_client members to avoid spanning cachelines dm: split discards further if target sets max_discard_granularity Mikulas Patocka (2): dm bufio: use waitqueue_active in __free_buffer_wake

[dm-devel] [dm-6.4 PATCH v2 4/9] dm bufio: move dm_bufio_client members to avoid spanning cachelines

2023-03-24 Thread Mike Snitzer
Movement also consolidates holes in dm_bufio_client struct. But the overall size of the struct isn't changed. Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md

[dm-devel] [dm-6.4 PATCH v2 5/9] dm bufio: use waitqueue_active in __free_buffer_wake

2023-03-24 Thread Mike Snitzer
From: Mikulas Patocka Save one spinlock by using waitqueue_active. We hold the bufio lock at this place, so no one can add entries to the waitqueue at this point. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 7 ++- 1 file changed, 6 insertions

[dm-devel] [dm-6.4 PATCH v2 3/9] dm bufio: improve concurrent IO performance

2023-03-24 Thread Mike Snitzer
acquire the buffers about 25 times faster. Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 1769 ++--- 1 file changed, 1292 insertions(+), 477 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index

[dm-devel] [dm-6.4 PATCH v2 7/9] dm thin: speed up cell_defer_no_holder()

2023-03-24 Thread Mike Snitzer
From: Joe Thornber Reduce the time that a spinlock is held in cell_defer_no_holder(). Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-thin.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-thin.c b/drivers/md/dm

[dm-devel] [dm-6.4 PATCH v2 6/9] dm bufio: use multi-page bio vector

2023-03-24 Thread Mike Snitzer
From: Mikulas Patocka The kernel supports multi page bio vector entries, so we can use them in dm-bufio as an optimization. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 24 1 file changed, 4 insertions(+), 20 deletions

[dm-devel] [dm-6.4 PATCH v2 1/9] dm bufio: remove unused dm_bufio_release_move interface

2023-03-24 Thread Mike Snitzer
From: Joe Thornber Was used by multi-snapshot DM target that never went upstream. Signed-off-by: Joe Thornber Acked-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c| 77 include/linux/dm-bufio.h | 6 2 files changed

[dm-devel] [dm-6.4 PATCH v2 2/9] dm bufio: move dm_buffer struct

2023-03-24 Thread Mike Snitzer
Movement prepares for finer grained dm_buffer changes, in the next commit, to be more easily seen. Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 97 ++- 1 file changed, 50 insertions(+), 47 deletions(-) diff --git a/drivers/md/dm-bufio.c b

[dm-devel] [git pull] device mapper fixes for 6.3-rc4

2023-03-24 Thread Mike Snitzer
Jiasheng Jiang (1): dm stats: check for and propagate alloc_percpu failure Mike Snitzer (1): dm crypt: avoid accessing uninitialized tasklet Mikulas Patocka (1): dm crypt: add cond_resched() to dmcrypt_write() drivers/md/dm-crypt.c | 16 ++-- drivers/md/dm-stats.c | 7

Re: [dm-devel] [dm-6.4 PATCH 5/8] dm bufio: improve concurrent IO performance

2023-03-23 Thread Mike Snitzer
On Thu, Mar 23 2023 at 3:51P -0400, Joe Thornber wrote: > Mike, > > I'm Nacking this patch here. You've taken the comment from my latest > patch, and attached it to something else. I don't know if you've taken an > older patch and updated it, or taken the latest patch and downgraded it. I

Re: [dm-devel] [dm-6.4 PATCH 1/8] dm: split discards further if target sets max_discard_granularity

2023-03-23 Thread Mike Snitzer
On Thu, Mar 23 2023 at 4:00P -0400, Joe Thornber wrote: > Also nacking this patch since you refused to test it. I said I'd test > since you haven't, but you couldn't wait. [Pleaase don't top-post.] That's fine. I had already dropped this patch from linux-next due to my testing showing it

[dm-devel] [dm-6.4 PATCH 4/8] dm bufio: move dm_buffer struct

2023-03-22 Thread Mike Snitzer
Movement prepares for finer grained dm_buffer changes, in the next commit, to be more easily seen. Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 97 ++- 1 file changed, 50 insertions(+), 47 deletions(-) diff --git a/drivers/md/dm-bufio.c b

[dm-devel] [dm-6.4 PATCH 7/8] dm bufio: rename buffer_cache struct to dm_buffer_cache

2023-03-22 Thread Mike Snitzer
Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 52 +-- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index b418ee8e3853..e7b901baa878 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers

[dm-devel] [dm-6.4 PATCH 1/8] dm: split discards further if target sets max_discard_granularity

2023-03-22 Thread Mike Snitzer
'max_discard_granularity' flag that if set will cause DM core to split discard bios relative to 'max_discard_sectors'. This treats 'discard_granularity' as a "min_discard_granularity" and 'max_discard_sectors' as a "max_discard_granularity". Requested-by: Joe Thornber Signed-off-by: Mike S

[dm-devel] [dm-6.4 PATCH 3/8] dm bufio: remove unused dm_bufio_release_move interface

2023-03-22 Thread Mike Snitzer
From: Joe Thornber Was used by multi-snapshot DM target that never went upstream. Signed-off-by: Joe Thornber Acked-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c| 77 include/linux/dm-bufio.h | 6 2 files changed

[dm-devel] [dm-6.4 PATCH 8/8] dm thin: speed up cell_defer_no_holder()

2023-03-22 Thread Mike Snitzer
From: Joe Thornber Reduce the time that a spinlock is held in cell_defer_no_holder(). Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-thin.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-thin.c b/drivers/md/dm

[dm-devel] [dm-6.4 PATCH 5/8] dm bufio: improve concurrent IO performance

2023-03-22 Thread Mike Snitzer
acquire the buffers about 25 times faster. Signed-off-by: Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 1769 ++--- 1 file changed, 1292 insertions(+), 477 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index

[dm-devel] [dm-6.4 PATCH 0/8] dm bufio, thin: improve concurrent IO performance

2023-03-22 Thread Mike Snitzer
current IO performance dm bufio: remove unused dm_bufio_release_move interface dm bufio: improve concurrent IO performance dm thin: speed up cell_defer_no_holder() Mike Snitzer (4): dm: split discards further if target sets max_discard_granularity dm bufio: move dm_buffer struct dm b

[dm-devel] [dm-6.4 PATCH 6/8] dm bufio: move dm_bufio_client members to avoid spanning cachelines

2023-03-22 Thread Mike Snitzer
Movement also consolidates holes in dm_bufio_client struct. But the overall size of the struct isn't changed. Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md

[dm-devel] [dm-6.4 PATCH 2/8] dm bio prison v1: improve concurrent IO performance

2023-03-22 Thread Mike Snitzer
Joe Thornber Signed-off-by: Mike Snitzer --- drivers/md/dm-bio-prison-v1.c | 87 +++ drivers/md/dm-bio-prison-v1.h | 10 drivers/md/dm-thin.c | 9 ++-- 3 files changed, 72 insertions(+), 34 deletions(-) diff --git a/drivers/md/dm-bio-prison-v1.c b/drivers

Re: [dm-devel] dm verity: log audit events for dm-verity target

2023-03-17 Thread Mike Snitzer
On Fri, Mar 17 2023 at 4:00P -0400, Michael Weiß wrote: > On 02.03.23 03:25, Paul Moore wrote: > > On Wed, Mar 1, 2023 at 6:34 AM Michael Weiß > > wrote: > >> > >> dm-verity signals integrity violations by returning I/O errors > >> to user space. To identify integrity violations by a

Re: [dm-devel] [bug report] dm crypt: conditionally enable code needed for tasklet usecases

2023-03-09 Thread Mike Snitzer
On Thu, Mar 09 2023 at 9:42P -0500, Dan Carpenter wrote: > On Thu, Mar 09, 2023 at 05:35:20PM +0300, Dan Carpenter wrote: > > --> 2758 if (test_bit(DM_CRYPT_NO_READ_WORKQUEUE, >flags) || > >^ > > 2759

Re: [dm-devel] dm crypt: initialize tasklet in crypt_io_init()

2023-03-08 Thread Mike Snitzer
On Wed, Mar 08 2023 at 2:19P -0500, Mike Snitzer wrote: > On Wed, Mar 08 2023 at 8:55P -0500, > Ignat Korchagin wrote: > > > Perhaps instead we can just pass an additional flag from > > tasklet_schedule to indicate to the function that we're running in a > > taskle

Re: [dm-devel] dm crypt: initialize tasklet in crypt_io_init()

2023-03-08 Thread Mike Snitzer
On Wed, Mar 08 2023 at 8:55P -0500, Ignat Korchagin wrote: > On Wed, Mar 8, 2023 at 2:56 AM Hou Tao wrote: > > > > Hi, > > > > On 3/7/2023 10:47 PM, Mike Snitzer wrote: > > > On Mon, Mar 06 2023 at 9:12P -0500, > > > Hou Tao wrote: > > &

Re: [dm-devel] dm crypt: initialize tasklet in crypt_io_init()

2023-03-07 Thread Mike Snitzer
On Mon, Mar 06 2023 at 9:12P -0500, Hou Tao wrote: > Hi, > > On 3/7/2023 3:31 AM, Mike Snitzer wrote: > > On Mon, Mar 06 2023 at 8:49P -0500, > > Hou Tao wrote: > > > >> From: Hou Tao > >> > >> When neither no_read_workqueue no

Re: [dm-devel] dm crypt: initialize tasklet in crypt_io_init()

2023-03-06 Thread Mike Snitzer
On Mon, Mar 06 2023 at 8:49P -0500, Hou Tao wrote: > From: Hou Tao > > When neither no_read_workqueue nor no_write_workqueue are enabled, > tasklet_trylock() in crypt_dec_pending() may still return false due to > an uninitialized state, and dm-crypt will do io completion in io_queue > instead

Re: [dm-devel] dm-crypt: add cond_resched() to dmcrypt_write()

2023-03-06 Thread Mike Snitzer
On Mon, Mar 06 2023 at 11:17P -0500, Mikulas Patocka wrote: > The loop in dmcrypt_write may be running for unbounded amount of time, > thus we need cond_resched() in it. > > This commit fixes the following warning: > > [ 3391.153255][ C12] watchdog: BUG: soft lockup - CPU#12 stuck for 23s!

Re: [dm-devel] dm-crypt: reexport sysfs of kcryptd workqueue

2023-02-27 Thread Mike Snitzer
On Wed, Feb 22 2023 at 10:18P -0500, yangerkun wrote: > From: yangerkun > > 'a2b8b2d97567 ("dm crypt: export sysfs of kcryptd workqueue")' give us > idea to set specific CPU or limit max_active crypt work. However sysfs > will report a warnning and fail 'cryptsetup refresh test' since the >

Re: [dm-devel] dm-crypt: fix softlockup in dmcrypt_write

2023-02-27 Thread Mike Snitzer
On Mon, Feb 27 2023 at 1:03P -0500, Mike Snitzer wrote: > On Mon, Feb 27 2023 at 12:55P -0500, > Mike Snitzer wrote: > > > On Sun, Feb 26 2023 at 8:31P -0500, > > yangerkun wrote: > > > > > > > > > > > 在 2023/2/26 10:01, Bart Va

Re: [dm-devel] dm-crypt: fix softlockup in dmcrypt_write

2023-02-27 Thread Mike Snitzer
On Mon, Feb 27 2023 at 12:55P -0500, Mike Snitzer wrote: > On Sun, Feb 26 2023 at 8:31P -0500, > yangerkun wrote: > > > > > > > 在 2023/2/26 10:01, Bart Van Assche 写道: > > > On 2/22/23 19:19, yangerkun wrote: > > > > @@ -1924,6

Re: [dm-devel] dm-crypt: fix softlockup in dmcrypt_write

2023-02-27 Thread Mike Snitzer
On Sun, Feb 26 2023 at 8:31P -0500, yangerkun wrote: > > > 在 2023/2/26 10:01, Bart Van Assche 写道: > > On 2/22/23 19:19, yangerkun wrote: > > > @@ -1924,6 +1926,10 @@ static int dmcrypt_write(void *data) > > >   BUG_ON(rb_parent(write_tree.rb_node)); > > > +    if

Re: [dm-devel] [git pull] device mapper changes for 6.3

2023-02-20 Thread Mike Snitzer
On Mon, Feb 20 2023 at 12:31P -0500, Mike Snitzer wrote: > Hi Linus, > > The following changes since commit 4a6a7bc21d4726c5772e47525e6039852555b391: > > block: Default to use cgroup support for BFQ (2023-01-30 09:42:42 -0700) > > are available in the Git repo

[dm-devel] [git pull] device mapper changes for 6.3

2023-02-20 Thread Mike Snitzer
g dm_get_mdptr() Jiapeng Chong (1): dm integrity: Remove bi_sector that's only used by commented debug code Joe Thornber (2): dm cache: free background tracker's queued work in btracker_destroy dm cache: Add some documentation to dm-cache-background-tracker.h Mike Snitz

[dm-devel] [PATCH 2/6] dm: add cond_resched() to dm_wq_requeue_work()

2023-02-17 Thread Mike Snitzer
Otherwise the while() loop in dm_wq_requeue_work() can result in a "dead loop" on systems that have preemption disabled. This is particularly problematic on single cpu systems. Fixes: 8b211aaccb915 ("dm: add two stage requeue mechanism") Cc: sta...@vger.kernel.org Signed-

[dm-devel] [PATCH 6/6] dm ioctl: remove unnecessary check when using dm_get_mdptr()

2023-02-17 Thread Mike Snitzer
f-by: Hou Tao Signed-off-by: Mike Snitzer --- drivers/md/dm-ioctl.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index 067dfc08d4c3..50a1259294d1 100644 --- a/drivers/md/dm-ioctl.c +++ b/drivers/md/dm-ioctl.c @@ -789

[dm-devel] [PATCH 4/6] dm cache: add cond_resched() to various workqueue loops

2023-02-17 Thread Mike Snitzer
Otherwise on resource constrained systems these workqueues may be too greedy. Signed-off-by: Mike Snitzer --- drivers/md/dm-cache-target.c | 4 1 file changed, 4 insertions(+) diff --git a/drivers/md/dm-cache-target.c b/drivers/md/dm-cache-target.c index 23195701dc15..dbbcfa580078 100644

[dm-devel] [PATCH 5/6] dm ioctl: assert _hash_lock is held in __hash_remove

2023-02-17 Thread Mike Snitzer
Also update dm_early_create() to take _hash_lock when calling both __get_name_cell and __hash_remove -- given dm_early_create()'s early boot usecase this locking isn't about correctness but it allows lockdep_assert_held() to be added to __hash_remove. Signed-off-by: Mike Snitzer --- drivers/md

[dm-devel] [PATCH 3/6] dm thin: add cond_resched() to various workqueue loops

2023-02-17 Thread Mike Snitzer
Otherwise on resource constrained systems these workqueues may be too greedy. Signed-off-by: Mike Snitzer --- drivers/md/dm-thin.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c index affd91a53042..6cd105c1cef3 100644 --- a/drivers/md/dm-thin.c

[dm-devel] [PATCH 1/6] dm: add cond_resched() to dm_wq_work()

2023-02-17 Thread Mike Snitzer
From: Pingfan Liu Otherwise the while() loop in dm_wq_work() can result in a "dead loop" on systems that have preemption disabled. This is particularly problematic on single cpu systems. Cc: sta...@vger.kernel.org Signed-off-by: Pingfan Liu Acked-by: Ming Lei Signed-off-by: Mi

[dm-devel] [PATCH 0/6] dm: some changes just staged for 6.3

2023-02-17 Thread Mike Snitzer
Hou Tao (1): dm ioctl: remove unnecessary check when using dm_get_mdptr() Mike Snitzer (4): dm: add cond_resched() to dm_wq_requeue_work() dm thin: add cond_resched() to various workqueue loops dm cache: add cond_resched() to various workqueue loops dm ioctl: assert _hash_lock is held

Re: [dm-devel] blk-mq: enforce op-specific segment limits in blk_insert_cloned_request

2023-02-17 Thread Mike Snitzer
On Fri, Feb 17 2023 at 11:16P -0500, Christoph Hellwig wrote: > On Thu, Feb 16, 2023 at 12:27:02PM -0700, Uday Shankar wrote: > > * Description: > > *@rq may have been made based on weaker limitations of upper-level > > queues > > *in request stacking drivers, and it may violate the

Re: [dm-devel] blk-mq: enforce op-specific segment limits in blk_insert_cloned_request

2023-02-16 Thread Mike Snitzer
On Thu, Feb 16 2023 at 2:27P -0500, Uday Shankar wrote: > On Wed, Feb 15, 2023 at 10:09:36PM -0800, Christoph Hellwig wrote: > > I'd just remove the debug check entirely > > Older kernels have these checks in a separate function called > blk_cloned_rq_check_limits, which carries the following

Re: [dm-devel] A hang bug of dm on s390x

2023-02-16 Thread Mike Snitzer
[Top-posting but please don't...] I've staged this fix for 6.3 inclusion and marked it for stable@: https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-6.3=0ca44fcef241768fd25ee763b3d203b9852f269b Ming, I also staged this similar fix (not reasoned through

Re: [dm-devel] Planning to WARN_ON() when flush_scheduled_work() is used.

2023-02-14 Thread Mike Snitzer
On Sat, Feb 11 2023 at 1:19P -0500, Tetsuo Handa wrote: > Hello. > > Like commit c4f135d64382 ("workqueue: Wrap flush_workqueue() using a macro") > says, flush_scheduled_work() is dangerous and will be forbidden. But I haven't > heard from you after

[dm-devel] [PATCH 36/39] dm ioctl: prefer strscpy() instead of strlcpy()

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-ioctl.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index e1eddb91cc2b..1fed9a9719f6 100644 --- a/drivers/md/dm

[dm-devel] [PATCH 39/39] dm clone: prefer kvmalloc_array()

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-clone-target.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-clone-target.c b/drivers/md/dm-clone-target.c index 29e0b85eeaf0..f38a27604c7a 100644

[dm-devel] [PATCH 38/39] dm: declare variables static when sensible

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-cache-target.c | 2 +- drivers/md/dm-thin.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/md/dm-cache-target.c b/drivers/md/dm-cache-target.c index

[dm-devel] [PATCH 37/39] dm: fix suspect indent whitespace

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-cache-target.c | 2 +- drivers/md/dm-raid.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/md/dm-cache-target.c b/drivers/md/dm-cache-target.c index

[dm-devel] [PATCH 32/39] dm: avoid 'do {} while(0)' loop in single statement macros

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-integrity.c | 4 ++-- drivers/md/dm-mpath.c | 4 +--- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index

[dm-devel] [PATCH 30/39] dm log: avoid trailing semicolon in macro

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-log.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-log.c b/drivers/md/dm-log.c index f1bc14b6fd7c..94c5d8a19231 100644 --- a/drivers/md/dm-log.c +++ b

[dm-devel] [PATCH 31/39] dm log: avoid multiple line dereference

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-log.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/md/dm-log.c b/drivers/md/dm-log.c index 94c5d8a19231..5f8c9c246a7e 100644 --- a/drivers/md/dm-log.c +++ b

[dm-devel] [PATCH 34/39] dm integrity: change macros min/max() -> min_t/max_t where appropriate

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-integrity.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index b70defcdd982..16aa68988363 100644

[dm-devel] [PATCH 35/39] dm: avoid void function return statements

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-io-rewind.c | 2 -- drivers/md/dm-log-userspace-base.c | 8 drivers/md/dm-zoned-target.c | 1 - 3 files changed, 11 deletions(-) diff --git a/drivers/md/dm-io

[dm-devel] [PATCH 22/39] dm: avoid using symbolic permissions

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 16 drivers/md/dm-kcopyd.c| 2 +- drivers/md/dm-mpath.c | 3 +-- drivers/md/dm-rq.c| 8 drivers/md/dm-stats.c

[dm-devel] [PATCH 33/39] dm: fix use of sizeof() macro

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-bufio.c | 6 +++--- drivers/md/dm-integrity.c | 16 drivers/md/dm-kcopyd.c | 2 +- drivers/md/dm-writecache.c | 6 +++--- 4 files changed, 15 insertions(+), 15

[dm-devel] [PATCH 28/39] dm: don't indent labels

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-bio-prison-v1.c | 2 +- drivers/md/dm-io.c | 2 +- drivers/md/dm-linear.c | 2 +- drivers/md/dm-zoned-metadata.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions

[dm-devel] [PATCH 25/39] dm: avoid useless 'else' after 'break' or return'

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-stripe.c| 8 ++--- drivers/md/dm-thin-metadata.c | 4 +-- drivers/md/dm-verity-target.c | 33 ++- drivers/md/dm

[dm-devel] [PATCH 29/39] dm ioctl: have constant on the right side of the test

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-ioctl.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index ec278675a8ca..e1eddb91cc2b 100644 --- a/drivers/md/dm

[dm-devel] [PATCH 24/39] dm: favour __packed versus "__attribute__ ((packed))"

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/persistent-data/dm-btree-internal.h | 4 ++-- drivers/md/persistent-data/dm-space-map-common.h | 8 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/md

[dm-devel] [PATCH 26/39] dm: add missing blank line after declarations/fix those

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-raid.c | 2 +- drivers/md/dm.c | 5 + drivers/md/persistent-data/dm-array.c | 4 ++-- drivers/md/persistent-data/dm-btree.c | 4 ++-- drivers/md

[dm-devel] [PATCH 27/39] dm: avoid inline filenames

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-ima.c | 3 +-- drivers/md/dm-ima.h | 3 +-- drivers/md/dm-init.c | 1 - 3 files changed, 2 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-ima.c b/drivers/md/dm-ima.c index

[dm-devel] [PATCH 18/39] dm: remove unnecessary braces from single statement blocks

2023-02-13 Thread Mike Snitzer
From: Heinz Mauelshagen Signed-off-by: Heinz Mauelshagen Signed-off-by: Mike Snitzer --- drivers/md/dm-integrity.c | 48 --- drivers/md/dm-stats.c | 9 ++-- drivers/md/dm-stripe.c| 10 ++-- drivers/md/dm-thin

<    1   2   3   4   5   6   7   8   9   10   >