Re: [dm-devel] [PATCH v13 3/9] block: add emulation for copy

2023-06-29 Thread Ming Lei
Hi Nitesh, On Wed, Jun 28, 2023 at 12:06:17AM +0530, Nitesh Shetty wrote: > For the devices which does not support copy, copy emulation is added. > It is required for in-kernel users like fabrics, where file descriptor is I can understand copy command does help for FS GC and fabrics storages,

Re: [dm-devel] [PATCH v3] blk-mq: enforce op-specific segment limits in blk_insert_cloned_request

2023-03-01 Thread Ming Lei
_op and enforcing the appropriate segment > limit - max_discard_segments for REQ_OP_DISCARDs and max_segments for > everything else. > > Signed-off-by: Uday Shankar Reviewed-by: Ming Lei Thanks, Ming -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] A hang bug of dm on s390x

2023-02-15 Thread Ming Lei
On Wed, Feb 15, 2023 at 07:23:40PM +0800, Pingfan Liu wrote: > Hi guys, > > I encountered a hang issue on a s390x system. The tested kernel is > not preemptible and booting with "nr_cpus=1" > > The test steps: > umount /home > lvremove /dev/rhel_s390x-kvm-011/home > ##

Re: [dm-devel] [RFC for-6.2/block V2] block: Change the granularity of io ticks from ms to ns

2022-12-07 Thread Ming Lei
On Wed, Dec 07, 2022 at 10:32:04PM +, Gulam Mohamed wrote: > As per the review comment from Jens Axboe, I am re-sending this patch > against "for-6.2/block". > > > Use ktime to change the granularity of IO accounting in block layer from > milli-seconds to nano-seconds to get the proper

Re: [dm-devel] [PATCH v5 02/10] block: Add copy offload support infrastructure

2022-12-07 Thread Ming Lei
On Wed, Dec 07, 2022 at 11:24:00AM +0530, Nitesh Shetty wrote: > On Tue, Nov 29, 2022 at 05:14:28PM +0530, Nitesh Shetty wrote: > > On Thu, Nov 24, 2022 at 08:03:56AM +0800, Ming Lei wrote: > > > On Wed, Nov 23, 2022 at 03:37:12PM +0530, Nitesh Shetty wrote: > > > >

Re: [dm-devel] [RFC] block: Change the granularity of io ticks from ms to ns

2022-12-06 Thread Ming Lei
On Wed, Dec 07, 2022 at 10:19:08AM +0800, Yu Kuai wrote: > Hi, > > 在 2022/12/07 2:15, Gulam Mohamed 写道: > > Use ktime to change the granularity of IO accounting in block layer from > > milli-seconds to nano-seconds to get the proper latency values for the > > devices whose latency is in

Re: [dm-devel] [PATCH v5 02/10] block: Add copy offload support infrastructure

2022-11-23 Thread Ming Lei
On Wed, Nov 23, 2022 at 03:37:12PM +0530, Nitesh Shetty wrote: > On Wed, Nov 23, 2022 at 04:04:18PM +0800, Ming Lei wrote: > > On Wed, Nov 23, 2022 at 11:28:19AM +0530, Nitesh Shetty wrote: > > > Introduce blkdev_issue_copy which supports source and destination bdevs, > > &

Re: [dm-devel] [PATCH v5 02/10] block: Add copy offload support infrastructure

2022-11-23 Thread Ming Lei
On Wed, Nov 23, 2022 at 11:28:19AM +0530, Nitesh Shetty wrote: > Introduce blkdev_issue_copy which supports source and destination bdevs, > and an array of (source, destination and copy length) tuples. > Introduce REQ_COPY copy offload operation flag. Create a read-write > bio pair with a token as

[dm-devel] [PATCH] blk-mq: don't add non-pt request with ->end_io to batch

2022-10-27 Thread Ming Lei
nitzer Reported-by: Changhui Zhong Signed-off-by: Ming Lei --- include/linux/blk-mq.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index ba18e9bdb799..d6119c5d1069 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-30 Thread Ming Lei
On Wed, Jun 29, 2022 at 09:14:54PM -0400, Kent Overstreet wrote: > On Thu, Jun 30, 2022 at 08:47:13AM +0800, Ming Lei wrote: > > Or if I misunderstood your point, please cook a patch and I am happy to > > take a close look, and posting one very raw idea with random data >

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-29 Thread Ming Lei
On Wed, Jun 29, 2022 at 02:11:54PM -0400, Kent Overstreet wrote: > On Wed, Jun 29, 2022 at 02:07:08AM -0400, Mike Snitzer wrote: > > Please try to dial down the hyperbole and judgment. Ming wrote this > > code. And you haven't been able to point out anything _actually_ wrong > > with it (yet). > >

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-28 Thread Ming Lei
On Tue, Jun 28, 2022 at 12:36:17PM -0400, Kent Overstreet wrote: > On Tue, Jun 28, 2022 at 03:49:28PM +0800, Ming Lei wrote: > > On Tue, Jun 28, 2022 at 12:26:10AM -0400, Kent Overstreet wrote: > > > On Mon, Jun 27, 2022 at 03:36:22PM +0800, Ming Lei wrote: > > > > N

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-28 Thread Ming Lei
On Tue, Jun 28, 2022 at 12:13:06PM -0600, Jens Axboe wrote: > On 6/27/22 10:20 PM, Kent Overstreet wrote: > > On Mon, Jun 27, 2022 at 03:36:22PM +0800, Ming Lei wrote: > >> On Sun, Jun 26, 2022 at 04:14:58PM -0400, Kent Overstreet wrote: > >>> On Fri, Jun 24, 2022

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-28 Thread Ming Lei
On Tue, Jun 28, 2022 at 12:26:10AM -0400, Kent Overstreet wrote: > On Mon, Jun 27, 2022 at 03:36:22PM +0800, Ming Lei wrote: > > Not mention bio_iter, bvec_iter has been 32 bytes, which is too big to > > hold in per-io data structure. With this patch, 8bytes is enough > &

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-28 Thread Ming Lei
On Tue, Jun 28, 2022 at 12:20:16AM -0400, Kent Overstreet wrote: > On Mon, Jun 27, 2022 at 03:36:22PM +0800, Ming Lei wrote: > > On Sun, Jun 26, 2022 at 04:14:58PM -0400, Kent Overstreet wrote: > > > On Fri, Jun 24, 2022 at 10:12:52PM +0800, Ming Lei wrote: > > > &g

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-27 Thread Ming Lei
On Sun, Jun 26, 2022 at 02:37:22PM -0700, Eric Biggers wrote: > On Fri, Jun 24, 2022 at 10:12:52PM +0800, Ming Lei wrote: > > diff --git a/block/blk-crypto.c b/block/blk-crypto.c > > index a496aaef85ba..caae2f429fc7 100644 > > --- a/block/blk-crypto.c > > +++ b/block

Re: [dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-27 Thread Ming Lei
On Sun, Jun 26, 2022 at 04:14:58PM -0400, Kent Overstreet wrote: > On Fri, Jun 24, 2022 at 10:12:52PM +0800, Ming Lei wrote: > > Commit 7759eb23fd98 ("block: remove bio_rewind_iter()") removes > > the similar API because the following reasons: > > &g

[dm-devel] [PATCH 5.20 4/4] dm: add two stage requeue

2022-06-24 Thread Ming Lei
d original bio for requeue, and we recover the original bio by bio_rewind(). 2) the 2nd stage requeue is same with original requeue, but io->orig_bio points to new cloned bio which matches with the requeued dm io. Signed-off-by: Ming Lei --- drivers/md/dm-core.h | 11 +++- drivers/md/dm

[dm-devel] [PATCH 5.20 3/4] dm: improve handling for DM_REQUEUE and AGAIN

2022-06-24 Thread Ming Lei
AIN, and clear REQ_POLLED for BLK_STS_DM_REQUEUE too, for the sake of simplicity, given BLK_STS_DM_REQUEUE is very unusual 2) queue md->wq explicitly in __dm_io_complete(), so requeue handling becomes more robust Signed-off-by: Ming Lei --- drivers/md/dm.

[dm-devel] [PATCH 5.20 2/4] dm: add new helper for handling dm_io requeue

2022-06-24 Thread Ming Lei
Add helper of dm_handle_requeue() for handling dm_io requeue. Signed-off-by: Ming Lei --- drivers/md/dm.c | 17 - 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 2b75f1ef7386..a9e5e429c150 100644 --- a/drivers/md/dm.c +++ b

[dm-devel] [PATCH 5.20 0/4] block/dm: add bio_rewind for improving dm requeue

2022-06-24 Thread Ming Lei
inal bio for handling requeue. Ming Lei (4): block: add bio_rewind() API dm: add new helper for handling dm_io requeue dm: improve handling for DM_REQUEUE and AGAIN dm: add two stage requeue block/bio-integrity.c | 19 block/bio.c | 19 block/blk-crypto-internal

[dm-devel] [PATCH 5.20 1/4] block: add bio_rewind() API

2022-06-24 Thread Ming Lei
ust for handling BLK_STS_DM_REQUEUE which is actually one unusual event. 4) Not like original rewind API, this one needn't to add .bi_done, and no any effect on fast path Cc: Eric Biggers Cc: Kent Overstreet Cc: Dmitry Monakhov Cc: Martin K. Petersen Signed-off-by: Ming Lei --- block/bio-integr

[dm-devel] [PATCH] dm: fix dm io BLK_STS_DM_REQUEUE

2022-06-23 Thread Ming Lei
zinski Fixes: 7dd76d1feec7 ("dm: improve bio splitting and associated IO accounting") Signed-off-by: Ming Lei --- drivers/md/dm-core.h | 2 +- drivers/md/dm.c | 10 ++ 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-co

Re: [dm-devel] [PATCH 5/8] dm: always setup ->orig_bio in alloc_io

2022-04-16 Thread Ming Lei
On Fri, Apr 15, 2022 at 05:06:55PM -0400, Mike Snitzer wrote: > On Thu, Apr 14 2022 at 8:14P -0400, > Ming Lei wrote: > > > On Thu, Apr 14, 2022 at 01:45:33PM -0400, Mike Snitzer wrote: > > > On Wed, Apr 13 2022 at 11:57P -0400, > > > Ming Lei wrote: > >

Re: [dm-devel] [PATCH 5/8] dm: always setup ->orig_bio in alloc_io

2022-04-14 Thread Ming Lei
On Thu, Apr 14, 2022 at 01:45:33PM -0400, Mike Snitzer wrote: > On Wed, Apr 13 2022 at 11:57P -0400, > Ming Lei wrote: > > > On Wed, Apr 13, 2022 at 10:25:45PM -0400, Mike Snitzer wrote: > > > On Wed, Apr 13 2022 at 8:36P -0400, > > > Ming Lei wrote: > >

Re: [dm-devel] [PATCH 5/8] dm: always setup ->orig_bio in alloc_io

2022-04-13 Thread Ming Lei
On Wed, Apr 13, 2022 at 10:25:45PM -0400, Mike Snitzer wrote: > On Wed, Apr 13 2022 at 8:36P -0400, > Ming Lei wrote: > > > On Wed, Apr 13, 2022 at 01:58:54PM -0400, Mike Snitzer wrote: > > > > > > The bigger issue with this patch is that you've caused >

Re: [dm-devel] [PATCH 5/8] dm: always setup ->orig_bio in alloc_io

2022-04-13 Thread Ming Lei
On Wed, Apr 13, 2022 at 01:58:54PM -0400, Mike Snitzer wrote: > On Wed, Apr 13 2022 at 8:26P -0400, > Ming Lei wrote: > > > On Wed, Apr 13, 2022 at 02:12:47AM -0400, Mike Snitzer wrote: > > > On Tue, Apr 12 2022 at 9:56P -0400, > > > Ming Lei wrote: > >

Re: [dm-devel] [PATCH 5/8] dm: always setup ->orig_bio in alloc_io

2022-04-13 Thread Ming Lei
On Wed, Apr 13, 2022 at 02:12:47AM -0400, Mike Snitzer wrote: > On Tue, Apr 12 2022 at 9:56P -0400, > Ming Lei wrote: > > > On Tue, Apr 12, 2022 at 04:52:40PM -0400, Mike Snitzer wrote: > > > On Tue, Apr 12 2022 at 4:56P -0400, > > > Ming Lei wrote: > >

Re: [dm-devel] [PATCH 5/8] dm: always setup ->orig_bio in alloc_io

2022-04-12 Thread Ming Lei
On Tue, Apr 12, 2022 at 04:52:40PM -0400, Mike Snitzer wrote: > On Tue, Apr 12 2022 at 4:56P -0400, > Ming Lei wrote: > > > The current DM codes setup ->orig_bio after __map_bio() returns, > > and not only cause kernel panic for dm zone, but also a bit ugly &g

Re: [dm-devel] [PATCH 3/8] dm: pass 'dm_io' instance to dm_io_acct directly

2022-04-12 Thread Ming Lei
On Tue, Apr 12, 2022 at 04:28:59PM -0400, Mike Snitzer wrote: > On Tue, Apr 12 2022 at 4:56P -0400, > Ming Lei wrote: > > > All the other 4 parameters are retrieved from the 'dm_io' instance, so > > not necessary to pass all four to dm_io_acct(). > > > > S

[dm-devel] [PATCH 8/8] dm: put all polled io into one single list

2022-04-12 Thread Ming Lei
If bio_split() isn't involved, it is a bit overkill to link dm_io into hlist, given there is only single dm_io in the list, so convert to single list for holding all dm_io instances associated with this bio. Signed-off-by: Ming Lei --- drivers/md/dm-core.h | 2 +- drivers/md/dm.c | 46

[dm-devel] [PATCH 7/8] dm: improve target io referencing

2022-04-12 Thread Ming Lei
be completed until both two sides are done. Signed-off-by: Ming Lei --- drivers/md/dm.c | 51 - 1 file changed, 38 insertions(+), 13 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 3c3ba6b4e19b..2987f7cf7b47 100644 --- a/drivers/md

[dm-devel] [PATCH 6/8] dm: don't grab target io reference in dm_zone_map_bio

2022-04-12 Thread Ming Lei
dm_zone_map_bio() is only called from __map_bio in which the io's reference is grabbed already, and the reference won't be released until the bio is submitted, so no necessary to do it dm_zone_map_bio any more. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Ming Lei

[dm-devel] [PATCH 5/8] dm: always setup ->orig_bio in alloc_io

2022-04-12 Thread Ming Lei
y fixes dm-zone's kernel panic, but also cleans up dm io accounting & split a bit. Signed-off-by: Ming Lei --- drivers/md/dm-core.h | 8 ++- drivers/md/dm.c | 51 ++-- 2 files changed, 32 insertions(+), 27 deletions(-) diff --git a/drivers/

[dm-devel] [PATCH 4/8] dm: switch to bdev based io accounting interface

2022-04-12 Thread Ming Lei
DM won't account sectors in flush IO, also we can retrieve sectors from 'dm_io' for avoiding to allocate & update new original bio, which will be done in the following patch. So switch to bdev based io accounting interface. Signed-off-by: Ming Lei --- drivers/md/dm.c

[dm-devel] [PATCH 3/8] dm: pass 'dm_io' instance to dm_io_acct directly

2022-04-12 Thread Ming Lei
All the other 4 parameters are retrieved from the 'dm_io' instance, so not necessary to pass all four to dm_io_acct(). Signed-off-by: Ming Lei --- drivers/md/dm.c | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 62f7af815ef8

[dm-devel] [PATCH 2/8] dm: don't pass bio to __dm_start_io_acct and dm_end_io_acct

2022-04-12 Thread Ming Lei
dm->orig_bio is always passed to __dm_start_io_acct and dm_end_io_acct, so it isn't necessary to take one bio parameter for the two helpers. Signed-off-by: Ming Lei --- drivers/md/dm.c | 19 --- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/drivers/md/dm.

[dm-devel] [PATCH 1/8] block: replace disk based account with bdev's

2022-04-12 Thread Ming Lei
. Signed-off-by: Ming Lei --- block/blk-core.c | 15 --- drivers/block/zram/zram_drv.c | 5 +++-- include/linux/blkdev.h| 7 --- 3 files changed, 15 insertions(+), 12 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 937bb6b86331..a3ae13b129ff

[dm-devel] [PATCH 0/8] dm: io accounting & polling improvement

2022-04-12 Thread Ming Lei
Hello Guys, The 1st patch adds bdev based io accounting interface. The 2nd ~ 5th patches improves dm's io accounting & split, meantime fixes kernel panic on dm-zone. The other patches improves io polling & dm io reference handling. Ming Lei (8): block: replace disk based account wit

Re: [dm-devel] [PATCH 1/3] dm: don't grab target io reference in dm_zone_map_bio

2022-04-11 Thread Ming Lei
On Tue, Apr 12, 2022 at 09:28:46AM +0900, Damien Le Moal wrote: > On 4/12/22 09:09, Ming Lei wrote: > > On Tue, Apr 12, 2022 at 08:33:04AM +0900, Damien Le Moal wrote: > >> On 4/11/22 23:18, Ming Lei wrote: > >>>>>> This fixes the issue: > >>>>

Re: [dm-devel] [PATCH 1/3] dm: don't grab target io reference in dm_zone_map_bio

2022-04-11 Thread Ming Lei
On Tue, Apr 12, 2022 at 08:33:04AM +0900, Damien Le Moal wrote: > On 4/11/22 23:18, Ming Lei wrote: > >>>> This fixes the issue: > >>>> > >>>> diff --git a/drivers/md/dm.c b/drivers/md/dm.c > >>>> index 3c5fad7c4ee6..3dd6735450c5 1

Re: [dm-devel] [PATCH 1/3] dm: don't grab target io reference in dm_zone_map_bio

2022-04-11 Thread Ming Lei
On Mon, Apr 11, 2022 at 04:42:59PM +0900, Damien Le Moal wrote: > On 4/11/22 16:34, Ming Lei wrote: > > On Mon, Apr 11, 2022 at 11:55:14AM +0900, Damien Le Moal wrote: > >> On 4/11/22 11:19, Damien Le Moal wrote: > >>> On 4/11/22 10:04, Ming Lei wrote: > >&g

Re: [dm-devel] [PATCH] dm: dm-zone: Fix NULL pointer dereference in dm_zone_map_bio()

2022-04-11 Thread Ming Lei
On Mon, Apr 11, 2022 at 06:38:38PM +0900, Damien Le Moal wrote: > Commit 0fbb4d93b38b ("dm: add dm_submit_bio_remap interface") changed > the alloc_io() function to delay the initialization of struct dm_io > orig_bio field, leaving this field as NULL until the first call to >

Re: [dm-devel] [PATCH 1/3] dm: don't grab target io reference in dm_zone_map_bio

2022-04-11 Thread Ming Lei
On Mon, Apr 11, 2022 at 11:55:14AM +0900, Damien Le Moal wrote: > On 4/11/22 11:19, Damien Le Moal wrote: > > On 4/11/22 10:04, Ming Lei wrote: > >> On Mon, Apr 11, 2022 at 09:50:57AM +0900, Damien Le Moal wrote: > >>> On 4/11/22 09:36, Ming Lei wrote: > >&g

Re: [dm-devel] [PATCH 1/3] dm: don't grab target io reference in dm_zone_map_bio

2022-04-10 Thread Ming Lei
On Mon, Apr 11, 2022 at 09:50:57AM +0900, Damien Le Moal wrote: > On 4/11/22 09:36, Ming Lei wrote: > > On Mon, Apr 11, 2022 at 09:18:56AM +0900, Damien Le Moal wrote: > >> On 4/9/22 02:12, Ming Lei wrote: > >>> dm_zone_map_bio() is only called from __map_bio in

Re: [dm-devel] [PATCH 1/3] dm: don't grab target io reference in dm_zone_map_bio

2022-04-10 Thread Ming Lei
On Mon, Apr 11, 2022 at 09:18:56AM +0900, Damien Le Moal wrote: > On 4/9/22 02:12, Ming Lei wrote: > > dm_zone_map_bio() is only called from __map_bio in which the io's > > reference is grabbed already, and the reference won't be released > > until the bio is submitted,

[dm-devel] [PATCH 3/3] dm: put all polled io into one single list

2022-04-08 Thread Ming Lei
If bio_split() isn't involved, it is a bit overkill to link dm_io into hlist, given there is only single dm_io in the list, so convert to single list for holding all dm_io instances associated with this bio. Signed-off-by: Ming Lei --- drivers/md/dm-core.h | 2 +- drivers/md/dm.c | 46

[dm-devel] [PATCH 2/3] dm: improve target io referencing

2022-04-08 Thread Ming Lei
be completed until both two sides are done. Signed-off-by: Ming Lei --- drivers/md/dm.c | 36 +--- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index b8424a4b4725..528559ca2f91 100644 --- a/drivers/md/dm.c +++ b

[dm-devel] [PATCH 1/3] dm: don't grab target io reference in dm_zone_map_bio

2022-04-08 Thread Ming Lei
dm_zone_map_bio() is only called from __map_bio in which the io's reference is grabbed already, and the reference won't be released until the bio is submitted, so no necessary to do it dm_zone_map_bio any more. Cc: Damien Le Moal Signed-off-by: Ming Lei --- drivers/md/dm-core.h | 7

[dm-devel] [PATCH 0/3] dm: improvement on io referencing and io poll

2022-04-08 Thread Ming Lei
Hello, The 1st patch removes get/put io reference in dm_zone_map_bio. The 2nd one improves io referencing. The 3rd patch switches to hold dm_io instance in single list. Ming Lei (3): dm: don't grab target io reference in dm_zone_map_bio dm: improve target io referencing dm: put all

Re: [dm-devel] [RFC PATCH] io_uring: reissue in case -EAGAIN is returned after io issue returns

2022-04-05 Thread Ming Lei
On Mon, Apr 04, 2022 at 12:51:30PM -0400, Mike Snitzer wrote: > On Sun, Apr 03 2022 at 7:45P -0400, > Ming Lei wrote: > > > -EAGAIN still may return after io issue returns, and REQ_F_REISSUE is > > set in io_complete_rw_iopoll(), but the req never gets chance to be handled

Re: [dm-devel] [PATCH v6 2/2] dm: support bio polling

2022-03-09 Thread Ming Lei
On Wed, Mar 09, 2022 at 09:11:26AM -0700, Jens Axboe wrote: > On 3/8/22 6:13 PM, Ming Lei wrote: > > On Tue, Mar 08, 2022 at 06:02:50PM -0700, Jens Axboe wrote: > >> On 3/7/22 11:53 AM, Mike Snitzer wrote: > >>> From: Ming Lei > >>> > >>> Sup

Re: [dm-devel] [PATCH v6 2/2] dm: support bio polling

2022-03-08 Thread Ming Lei
On Tue, Mar 08, 2022 at 06:02:50PM -0700, Jens Axboe wrote: > On 3/7/22 11:53 AM, Mike Snitzer wrote: > > From: Ming Lei > > > > Support bio(REQ_POLLED) polling in the following approach: > > > > 1) only support io polling on normal READ/WRITE, and other abnor

Re: [dm-devel] [PATCH v5 2/2] dm: support bio polling

2022-03-06 Thread Ming Lei
On Mon, Mar 07, 2022 at 10:41:31AM +0800, Ming Lei wrote: > On Sun, Mar 06, 2022 at 07:25:11PM -0700, Jens Axboe wrote: > > On 3/6/22 7:20 PM, Ming Lei wrote: > > > On Sun, Mar 06, 2022 at 06:48:15PM -0700, Jens Axboe wrote: > > >> On 3/6/22 2:

Re: [dm-devel] [PATCH v5 2/2] dm: support bio polling

2022-03-06 Thread Ming Lei
On Sun, Mar 06, 2022 at 07:25:11PM -0700, Jens Axboe wrote: > On 3/6/22 7:20 PM, Ming Lei wrote: > > On Sun, Mar 06, 2022 at 06:48:15PM -0700, Jens Axboe wrote: > >> On 3/6/22 2:29 AM, Christoph Hellwig wrote: > >>>> +/* > >>>> + * Reuse ->bi_e

Re: [dm-devel] [PATCH v5 2/2] dm: support bio polling

2022-03-06 Thread Ming Lei
On Sun, Mar 06, 2022 at 06:48:15PM -0700, Jens Axboe wrote: > On 3/6/22 2:29 AM, Christoph Hellwig wrote: > >> +/* > >> + * Reuse ->bi_end_io as hlist head for storing all dm_io instances > >> + * associated with this bio, and this bio's bi_end_io has to be > >> + * stored in one of 'dm_io'

Re: [dm-devel] [PATCH v4 0/2] block/dm: support bio polling

2022-03-04 Thread Ming Lei
On Fri, Mar 04, 2022 at 04:26:21PM -0500, Mike Snitzer wrote: > Hi, > > I've rebased Ming's latest [1] ontop of dm-5.18 [2] (which is based on > for-5.18/block). End result available in dm-5.18-biopoll branch [3] > > These changes add bio polling support to DM. Tested with linear and > striped

Re: [dm-devel] [PATCH V3 0/3] block/dm: support bio polling

2022-03-01 Thread Ming Lei
On Tue, Mar 01, 2022 at 04:19:42PM -0500, Mike Snitzer wrote: > On Mon, Feb 28 2022 at 7:58P -0500, > Ming Lei wrote: > > > On Mon, Feb 28, 2022 at 11:27:44AM -0500, Mike Snitzer wrote: > > > > > > Hey Ming, > > > > > > I'd like us to f

Re: [dm-devel] [PATCH V3 0/3] block/dm: support bio polling

2022-02-28 Thread Ming Lei
On Mon, Feb 28, 2022 at 11:27:44AM -0500, Mike Snitzer wrote: > On Wed, Jun 23 2021 at 3:40P -0400, > Ming Lei wrote: > > > Hello Guys, > > > > Based on Christoph's bio based polling model[1], implement DM bio polling > > with one very simple appro

Re: [dm-devel] [PATCH] dm rq: clear cloned bio ->bi_bdev to fix I/O accounting

2022-01-17 Thread Ming Lei
rep_clone currently > still point to the original bi_bdev. This is harmless because dm-mpath It breaks io accounting, so not harmless, but the code in this patch is fine: Reviewed-by: Ming Lei Thanks, Ming -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH] dm rq: clear cloned bio ->bi_bdev to fix I/O accounting

2022-01-15 Thread Ming Lei
uctor(struct bio *bio, struct > bio *bio_orig, > info->orig = bio_orig; > info->tio = tio; > bio->bi_end_io = end_clone_bio; > + bio->bi_bdev = NULL; Reviewed-by: Ming Lei -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH 1/2] block: add resubmit_bio_noacct()

2022-01-10 Thread Ming Lei
On Mon, Jan 10, 2022 at 02:03:16PM -0500, Mike Snitzer wrote: > On Mon, Jan 10 2022 at 12:35P -0500, > Christoph Hellwig wrote: > > > On Mon, Jan 10, 2022 at 03:51:40PM +0800, Ming Lei wrote: > > > Add block layer API of resubmit_bio_noacct() for handling blk-throttle &g

Re: [dm-devel] [PATCH 1/2] block: add resubmit_bio_noacct()

2022-01-10 Thread Ming Lei
On Mon, Jan 10, 2022 at 09:35:22AM -0800, Christoph Hellwig wrote: > On Mon, Jan 10, 2022 at 03:51:40PM +0800, Ming Lei wrote: > > Add block layer API of resubmit_bio_noacct() for handling blk-throttle > > iops limit correctly. Typical use case is that bio split, and it isn't >

[dm-devel] [PATCH 1/2] block: add resubmit_bio_noacct()

2022-01-10 Thread Ming Lei
-by: Ming Lei --- block/blk-core.c | 12 block/blk-merge.c | 4 +--- include/linux/blkdev.h | 1 + 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index fd029c86d6ac..733fec7dc5d6 100644 --- a/block/blk-core.c +++ b/block

[dm-devel] [PATCH 0/2] block/dm: add resubmit_bio_noacct for fixing iops throttling

2022-01-09 Thread Ming Lei
ice mapper via the added API, and the issue is reported by lining. Ming Lei (2): block: add resubmit_bio_noacct() dm: use resubmit_bio_noacct to submit split bio block/blk-core.c | 12 block/blk-merge.c | 4 +--- drivers/md/dm.c| 2 +- include/linux/blkdev.h

[dm-devel] [PATCH 2/2] dm: use resubmit_bio_noacct to submit split bio

2022-01-09 Thread Ming Lei
Reported-by: lining Cc: Tejun Heo Cc: Chunguang Xu Signed-off-by: Ming Lei --- drivers/md/dm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 280918cdcabd..8a58379e737c 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1562

Re: [dm-devel] [PATCH 3/3] dm: mark dm queue as blocking if any underlying is blocking

2022-01-06 Thread Ming Lei
On Thu, Jan 06, 2022 at 10:40:51AM -0500, Mike Snitzer wrote: > On Tue, Dec 21 2021 at 9:14P -0500, > Ming Lei wrote: > > > dm request based driver doesn't set BLK_MQ_F_BLOCKING, so dm_queue_rq() > > is supposed to not sleep. > > > > However, blk_insert_cloned

Re: [dm-devel] [PATCH 0/3] blk-mq/dm-rq: support BLK_MQ_F_BLOCKING for dm-rq

2021-12-22 Thread Ming Lei
On Tue, Dec 21, 2021 at 08:21:39AM -0800, Christoph Hellwig wrote: > On Tue, Dec 21, 2021 at 10:14:56PM +0800, Ming Lei wrote: > > Hello, > > > > dm-rq may be built on blk-mq device which marks BLK_MQ_F_BLOCKING, so > > dm_mq_queue_rq() may become to sleep curr

[dm-devel] [PATCH 3/3] dm: mark dm queue as blocking if any underlying is blocking

2021-12-21 Thread Ming Lei
to block current context, then rcu warning is triggered. Fixes the issue by marking dm request based queue as BLK_MQ_F_BLOCKING if any underlying queue is marked as BLK_MQ_F_BLOCKING, meantime we need to allocate srcu beforehand. Signed-off-by: Ming Lei --- drivers/md/dm-rq.c| 5

[dm-devel] [PATCH 2/3] block: add blk_alloc_disk_srcu

2021-12-21 Thread Ming Lei
Add blk_alloc_disk_srcu() so that we can allocate srcu inside request queue for supporting blocking ->queue_rq(). dm-rq needs this API. Signed-off-by: Ming Lei --- block/genhd.c | 5 +++-- include/linux/genhd.h | 12 2 files changed, 11 insertions(+), 6 deletions(-) d

[dm-devel] [PATCH 1/3] block: split having srcu from queue blocking

2021-12-21 Thread Ming Lei
q. So add one new flag of QUEUE_FLAG_BLOCKING for supporting blocking ->queue_rq only, and use one private field to describe if request queue has allocated srcu instance. Signed-off-by: Ming Lei --- block/blk-core.c | 2 +- block/blk-mq.c | 6 +++--- block/blk-mq.h | 2

[dm-devel] [PATCH 0/3] blk-mq/dm-rq: support BLK_MQ_F_BLOCKING for dm-rq

2021-12-21 Thread Ming Lei
allocating tagset, this way is a bit special, so we need to pre-allocate srcu payload, then use the queue flag of QUEUE_FLAG_BLOCKING for locking dispatch. Ming Lei (3): block: split having srcu from queue blocking block: add blk_alloc_disk_srcu dm: mark dm queue as blocking if any underlying

Re: [dm-devel] Random high CPU utilization in blk-mq with the none scheduler

2021-12-13 Thread Ming Lei
On Tue, Dec 14, 2021 at 12:31:23AM +, Dexuan Cui wrote: > > From: Ming Lei > > Sent: Sunday, December 12, 2021 11:38 PM > > Ming, thanks so much for the detailed analysis! > > > From the log: > > > > 1) dm-mpath: > > - queue depth: 2048 > &

Re: [dm-devel] [PATCH 2/3] scsi: make sure that request queue queiesce and unquiesce balanced

2021-11-02 Thread Ming Lei
Hi James, On Mon, Nov 01, 2021 at 09:43:27PM -0400, James Bottomley wrote: > On Thu, 2021-10-21 at 22:59 +0800, Ming Lei wrote: > > For fixing queue quiesce race between driver and block layer(elevator > > switch, update nr_requests, ...), we need to support concurr

[dm-devel] [PATCH 2/3] scsi: make sure that request queue queiesce and unquiesce balanced

2021-10-21 Thread Ming Lei
contexts, so do it in scsi core with one per-device bit flag & global spinlock, basically zero cost since request queue quiesce is seldom triggered. Reported-by: Yi Zhang Fixes: e70feb8b3e68 ("blk-mq: support concurrent queue quiesce/unquiesce") Signed-off-by: Ming Lei --- drivers/s

[dm-devel] [PATCH 3/3] dm: don't stop request queue after the dm device is suspended

2021-10-21 Thread Ming Lei
not necessary to stop queue again. With this way, request queue quiesce and unquiesce can be balanced. Reported-by: Yi Zhang Fixes: e70feb8b3e68 ("blk-mq: support concurrent queue quiesce/unquiesce") Signed-off-by: Ming Lei --- drivers/md/dm.c | 10 -- 1 file changed, 10

[dm-devel] [PATCH 1/3] scsi: avoid to quiesce sdev->request_queue two times

2021-10-21 Thread Ming Lei
the flag, then scsi_internal_device_block() calls blk_mq_quiesce_queue_nowait() two times actually. Fix the double quiesce and keep quiesce and unquiesce balanced. Reported-by: Yi Zhang Fixes: e70feb8b3e68 ("blk-mq: support concurrent queue quiesce/unquiesce") Signed-off-by: Ming Lei --

[dm-devel] [PATCH 0/3] block: keep quiesce & unquiesce balanced for scsi/dm

2021-10-21 Thread Ming Lei
ually. So fix dm and scsi and make srp/002 pass again. Ming Lei (3): scsi: avoid to quiesce sdev->request_queue two times scsi: make sure that request queue queiesce and unquiesce balanced dm: don't stop request queue after the dm device is suspended drivers/md/dm.c| 10 --

Re: [dm-devel] dm-rq: don't queue request during suspend

2021-10-07 Thread Ming Lei
On Wed, Oct 06, 2021 at 10:15:58AM -0400, Mike Snitzer wrote: > On Thu, Sep 23 2021 at 5:11P -0400, > Ming Lei wrote: > > > DM uses blk-mq's quiesce/unquiesce to stop/start device mapper queue. > > > > But blk-mq's unquiesce may come from outside events, such as el

[dm-devel] [PATCH] dm-rq: don't queue request during suspend

2021-09-23 Thread Ming Lei
running updating nr_requests and dm-mpath suspend/resume stress test. Signed-off-by: Ming Lei --- drivers/md/dm-rq.c | 8 1 file changed, 8 insertions(+) diff --git a/drivers/md/dm-rq.c b/drivers/md/dm-rq.c index 5b95eea517d1..a896dea9750e 100644 --- a/drivers/md/dm-rq.c +++ b/drivers/md/dm

Re: [dm-devel] [PATCH v3 2/8] scsi/sr: add error handling support for add_disk()

2021-09-06 Thread Ming Lei
On Mon, Aug 30, 2021 at 02:25:32PM -0700, Luis Chamberlain wrote: > We never checked for errors on add_disk() as this function > returned void. Now that this is fixed, use the shiny new > error handling. > > Reviewed-by: Christoph Hellwig > Signed-off-by: Luis Chamberlain > --- >

Re: [dm-devel] [PATCH v3 1/8] scsi/sd: add error handling support for add_disk()

2021-09-06 Thread Ming Lei
On Mon, Aug 30, 2021 at 02:25:31PM -0700, Luis Chamberlain wrote: > We never checked for errors on add_disk() as this function > returned void. Now that this is fixed, use the shiny new > error handling. > > Reviewed-by: Christoph Hellwig > Signed-off-by: Luis Chamberlain > --- >

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-30 Thread Ming Lei
On Mon, Jun 21, 2021 at 07:33:34PM +0800, JeffleXu wrote: > > > On 6/18/21 10:39 PM, Ming Lei wrote: > > From 47e523b9ee988317369eaadb96826323cd86819e Mon Sep 17 00:00:00 2001 > > From: Ming Lei > > Date: Wed, 16 Jun 2021 16:13:46 +0800 > > Subject: [RFC PATC

[dm-devel] [PATCH V3 2/3] block: add ->poll_bio to block_device_operations

2021-06-23 Thread Ming Lei
Prepare for supporting IO polling for bio based driver. Add ->poll_bio callback so that bio driver can provide their own logic for polling bio. Signed-off-by: Ming Lei --- block/blk-core.c | 13 + block/genhd.c | 2 ++ include/linux/blkdev.h | 1 + 3 files chan

[dm-devel] [PATCH V3 1/3] block: add helper of blk_queue_poll

2021-06-23 Thread Ming Lei
There has been 3 users, and will be more, so add one such helper. Reviewed-by: Chaitanya Kulkarni Reviewed-by: Jeffle Xu Reviewed-by: Hannes Reinecke Signed-off-by: Ming Lei --- block/blk-core.c | 5 ++--- block/blk-sysfs.c| 4 ++-- drivers/nvme/host/core.c | 2 +- include

[dm-devel] [PATCH V3 3/3] dm: support bio polling

2021-06-23 Thread Ming Lei
ia bio->bi_bio_drv_data; call dec_pending() after the target io is done in .poll_bio() 4) enable QUEUE_FLAG_POLL if all underlying queues enable QUEUE_FLAG_POLL, which is based on Jeffle's previous patch. Signed-off-by: Ming Lei --- drivers/md/dm-table.c | 24 drivers/md/dm.c

[dm-devel] [PATCH V3 0/3] block/dm: support bio polling

2021-06-23 Thread Ming Lei
V2: - drop patch to add new fields into bio - support io polling for dm native bio splitting - add comment Ming Lei (3): block: add helper of blk_queue_poll block: add ->poll_bio to block_device_operations dm: support bio polling block/blk-core.c | 18 +++--- bl

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-21 Thread Ming Lei
On Tue, Jun 22, 2021 at 10:26:15AM +0800, JeffleXu wrote: > > > On 6/21/21 10:04 PM, Ming Lei wrote: > > On Mon, Jun 21, 2021 at 07:33:34PM +0800, JeffleXu wrote: > >> > >> > >> On 6/18/21 10:39 PM, Ming Lei wrote: > >>> From 47e523b9ee9

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-21 Thread Ming Lei
On Mon, Jun 21, 2021 at 07:33:34PM +0800, JeffleXu wrote: > > > On 6/18/21 10:39 PM, Ming Lei wrote: > > From 47e523b9ee988317369eaadb96826323cd86819e Mon Sep 17 00:00:00 2001 > > From: Ming Lei > > Date: Wed, 16 Jun 2021 16:13:46 +0800 > > Subject: [RFC PATC

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-21 Thread Ming Lei
On Mon, Jun 21, 2021 at 09:36:56AM +0200, Christoph Hellwig wrote: > On Thu, Jun 17, 2021 at 06:35:49PM +0800, Ming Lei wrote: > > + /* > > +* Only support bio polling for normal IO, and the target io is > > +* exactly inside the dm io instance >

Re: [dm-devel] [RFC PATCH V2 2/3] block: add ->poll_bio to block_device_operations

2021-06-21 Thread Ming Lei
On Mon, Jun 21, 2021 at 09:25:02AM +0200, Christoph Hellwig wrote: > > + struct gendisk *disk = bio->bi_bdev->bd_disk; > > + struct request_queue *q = disk->queue; > > blk_qc_t cookie = READ_ONCE(bio->bi_cookie); > > int ret; > > > > - if (cookie == BLK_QC_T_NONE ||

Re: [dm-devel] [RFC PATCH V2 1/3] block: add helper of blk_queue_poll

2021-06-21 Thread Ming Lei
On Mon, Jun 21, 2021 at 09:20:36AM +0200, Christoph Hellwig wrote: > On Thu, Jun 17, 2021 at 06:35:47PM +0800, Ming Lei wrote: > > There has been 3 users, and will be more, so add one such helper. > > > > Reviewed-by: Chaitanya Kulkarni > > Reviewed-by: Jeffle

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-18 Thread Ming Lei
On Fri, Jun 18, 2021 at 04:56:45PM -0400, Mike Snitzer wrote: > [you really should've changed the subject of this email to > "[RFC PATCH V3 3/3] dm: support bio polling"] > > On Fri, Jun 18 2021 at 10:39P -0400, > Ming Lei wrote: > > > From 47e523b9ee988317369e

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-18 Thread Ming Lei
>From 47e523b9ee988317369eaadb96826323cd86819e Mon Sep 17 00:00:00 2001 From: Ming Lei Date: Wed, 16 Jun 2021 16:13:46 +0800 Subject: [RFC PATCH V3 3/3] dm: support bio polling Support bio(REQ_POLLED) polling in the following approach: 1) only support io polling on normal READ/WRITE, and ot

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-18 Thread Ming Lei
Hello Jeffle, On Fri, Jun 18, 2021 at 04:19:10PM +0800, JeffleXu wrote: > > > On 6/17/21 6:35 PM, Ming Lei wrote: > > Support bio(REQ_POLLED) polling in the following approach: > > > > 1) only support io polling on normal READ/WRITE, and other abnormal IOs > &g

Re: [dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-17 Thread Ming Lei
On Thu, Jun 17, 2021 at 06:35:49PM +0800, Ming Lei wrote: > Support bio(REQ_POLLED) polling in the following approach: > > 1) only support io polling on normal READ/WRITE, and other abnormal IOs > still fallback on IRQ mode, so the target io is exactly inside the dm > io. > &g

[dm-devel] [RFC PATCH V2 3/3] dm: support bio polling

2021-06-17 Thread Ming Lei
ia bio->bi_bio_drv_data; call dec_pending() after the target io is done in .poll_bio() 4) enable QUEUE_FLAG_POLL if all underlying queues enable QUEUE_FLAG_POLL, which is based on Jeffle's previous patch. Signed-off-by: Ming Lei --- drivers/md/dm-table.c | 24 + drivers/md/dm.c

[dm-devel] [RFC PATCH V2 2/3] block: add ->poll_bio to block_device_operations

2021-06-17 Thread Ming Lei
Prepare for supporting IO polling for bio based driver. Add ->poll_bio callback so that bio driver can provide their own logic for polling bio. Signed-off-by: Ming Lei --- block/blk-core.c | 18 +- block/genhd.c | 3 +++ include/linux/blkdev.h | 2 ++ 3 fi

[dm-devel] [RFC PATCH V2 1/3] block: add helper of blk_queue_poll

2021-06-17 Thread Ming Lei
There has been 3 users, and will be more, so add one such helper. Reviewed-by: Chaitanya Kulkarni Reviewed-by: Jeffle Xu Reviewed-by: Hannes Reinecke Signed-off-by: Ming Lei --- block/blk-core.c | 5 ++--- block/blk-sysfs.c| 4 ++-- drivers/nvme/host/core.c | 2 +- include

[dm-devel] [RFC PATCH V2 0/3] block/dm: support bio polling

2021-06-17 Thread Ming Lei
implements bio polling for device mapper. Any comments are welcome. V2: - drop patch to add new fields into bio - support io polling for dm native bio splitting - add comment Ming Lei (3): block: add helper of blk_queue_poll block: add ->poll_bio to block_device_operati

  1   2   3   4   5   6   7   >