Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-07 Thread Ming Lei
On Thu, Dec 07, 2017 at 09:11:54PM +, Bart Van Assche wrote: > On Thu, 2017-12-07 at 09:31 +0800, Ming Lei wrote: > > But if you always call blk_mq_sched_mark_restart_hctx() before a new > > dispatch, that may affect performance on NVMe which may never trigger > > BLK_S

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-06 Thread Ming Lei
On Thu, Dec 07, 2017 at 12:10:51AM +0100, Holger Hoffstätte wrote: > On 12/05/17 08:52, Ming Lei wrote: > > Before commit 0df21c86bdbf ("scsi: implement .get_budget and .put_budget > > for blk-mq"), we run queue after 3ms if queue is idle and SCSI device > > q

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-06 Thread Ming Lei
On Wed, Dec 06, 2017 at 04:07:17PM +, Bart Van Assche wrote: > On Wed, 2017-12-06 at 09:52 +0800, Ming Lei wrote: > > On Wed, Dec 06, 2017 at 12:28:25AM +0800, Ming Lei wrote: > > > On Tue, Dec 05, 2017 at 04:08:20PM +, Bart Van Assche wrote: > > > > The patc

[PATCH] block: print error via printk_ratelimited in blk_partition_remap()

2017-12-06 Thread Ming Lei
.de> Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-core.c b/block/blk-core.c index b8881750a3ac..32878c2ccece 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -2074,7 +2074,7 @@ stati

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Ming Lei
On Wed, Dec 06, 2017 at 12:28:25AM +0800, Ming Lei wrote: > On Tue, Dec 05, 2017 at 04:08:20PM +, Bart Van Assche wrote: > > On Tue, 2017-12-05 at 15:52 +0800, Ming Lei wrote: > > > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c > > > index db9556

Re: [PATCH V4] scsi_debugfs: fix crash in scsi_show_rq()

2017-12-05 Thread Ming Lei
On Wed, Nov 15, 2017 at 08:04:49PM +0800, Ming Lei wrote: > On Wed, Nov 15, 2017 at 07:28:00PM +0900, James Bottomley wrote: > > On Wed, 2017-11-15 at 18:09 +0800, Ming Lei wrote: > > > On Tue, Nov 14, 2017 at 10:14:52AM -0800, James Bottomley wrote: > > > > >

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Ming Lei
On Tue, Dec 05, 2017 at 04:41:46PM +, Bart Van Assche wrote: > On Wed, 2017-12-06 at 00:28 +0800, Ming Lei wrote: > > This is still a workaround for RESTART, see my comment before: > > > > https://marc.info/?l=linux-block=151217500929341=2 > > A quote from tha

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-05 Thread Ming Lei
On Tue, Dec 05, 2017 at 04:22:33PM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 13:00 +0800, Ming Lei wrote: > > No, do not mix two different things in one patch, especially the fix part > > need to be backported to stable. > > > > The fix part should aim at V4

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Ming Lei
On Tue, Dec 05, 2017 at 04:08:20PM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 15:52 +0800, Ming Lei wrote: > > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c > > index db9556662e27..1816dd8259b3 100644 > > --- a/drivers/scsi/scsi_lib.c > > ++

[PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-04 Thread Ming Lei
eduler dd if=/dev/$DEVICE of=/dev/null bs=1M iflag=direct count=1 & sleep 5 echo "0" >/sys/bus/pseudo/drivers/scsi_debug/opts wait echo "SUCCESS" Fixes: 0df21c86bdbf ("scsi: implement .get_budget and .put_budget for blk-mq") Signed-off-by: Ming Lei <ming@re

Re: [PATCH] SCSI: delay run queue if device is blocked in scsi_dev_queue_ready()

2017-12-04 Thread Ming Lei
On Tue, Dec 05, 2017 at 01:16:24PM +0800, Ming Lei wrote: > On Mon, Dec 04, 2017 at 11:48:07PM +, Holger Hoffstätte wrote: > > On Tue, 05 Dec 2017 06:45:08 +0800, Ming Lei wrote: > > > > > On Mon, Dec 04, 2017 at 03:09:20PM +, Bart Van Assche wrote: > >

Re: [PATCH] SCSI: delay run queue if device is blocked in scsi_dev_queue_ready()

2017-12-04 Thread Ming Lei
On Mon, Dec 04, 2017 at 11:48:07PM +, Holger Hoffstätte wrote: > On Tue, 05 Dec 2017 06:45:08 +0800, Ming Lei wrote: > > > On Mon, Dec 04, 2017 at 03:09:20PM +, Bart Van Assche wrote: > >> On Sun, 2017-12-03 at 00:31 +0800, Ming Lei wrote: > >> > Fixe

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-04 Thread Ming Lei
On Tue, Dec 05, 2017 at 01:59:51AM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 09:15 +0800, Ming Lei wrote: > > On Mon, Dec 04, 2017 at 04:38:08PM -0800, Bart Van Assche wrote: > > > Since the next patch will make it possible that scsi_show_rq() gets > > > ca

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-04 Thread Ming Lei
On Mon, Dec 04, 2017 at 10:42:28PM -0500, Martin K. Petersen wrote: > > Hi Ming, > > > Please cook a patch for fixing the crash issue only, since we need > > to backport the fix to stable kernel. > > I thought you were going to submit a V5 that addressed James' concerns? > > -- > Martin K.

Re: [PATCH] blk-mq: Fix several SCSI request queue lockups

2017-12-04 Thread Ming Lei
On Tue, Dec 05, 2017 at 01:13:43AM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 09:04 +0800, Ming Lei wrote: > > Then no reason to revert commit(0df21c86bdbf scsi: implement .get_budget an > > .put_budget for blk-mq) for one issue which may never happen in rea

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-04 Thread Ming Lei
uest timeout and SCSI command flags. This patch also > fixes a bug that was reported by Ming Lei. See also Ming Lei, > scsi_debugfs: fix crash in scsi_show_rq(), linux-scsi, 7 November > 2017 (https://marc.info/?l=linux-block=151006655317188). Please cook a patch for fixing the crash issu

Re: [PATCH] blk-mq: Fix several SCSI request queue lockups

2017-12-04 Thread Ming Lei
On Tue, Dec 05, 2017 at 12:29:59AM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 08:20 +0800, Ming Lei wrote: > > Also it is a bit odd to see request in hctx->dispatch now, and it can only > > happen now when scsi_target_queue_ready() returns false, so I guess you

Re: [PATCH] blk-mq: Fix several SCSI request queue lockups

2017-12-04 Thread Ming Lei
On Mon, Dec 04, 2017 at 11:32:27PM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 07:01 +0800, Ming Lei wrote: > > On Mon, Dec 04, 2017 at 10:48:18PM +, Bart Van Assche wrote: > > > On Tue, 2017-12-05 at 06:42 +0800, Ming Lei wrote: > > > > On Mon, Dec 04,

Re: [PATCH] blk-mq: Fix several SCSI request queue lockups

2017-12-04 Thread Ming Lei
On Mon, Dec 04, 2017 at 10:48:18PM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 06:42 +0800, Ming Lei wrote: > > On Mon, Dec 04, 2017 at 09:30:32AM -0800, Bart Van Assche wrote: > > > * A systematic lockup for SCSI queues with queue depth 1. The > > > followi

Re: [PATCH] SCSI: delay run queue if device is blocked in scsi_dev_queue_ready()

2017-12-04 Thread Ming Lei
On Mon, Dec 04, 2017 at 03:09:20PM +, Bart Van Assche wrote: > On Sun, 2017-12-03 at 00:31 +0800, Ming Lei wrote: > > Fixes: 0df21c86bdbf ("scsi: implement .get_budget and .put_budget for > > blk-mq") > > It might be safer to revert commit 0df21c86bdbf instea

Re: [PATCH] blk-mq: Fix several SCSI request queue lockups

2017-12-04 Thread Ming Lei
it is too risky to try to fix all bugs > introduced by commit 0df21c86bdbf before kernel v4.15 is released. > Hence revert that commit. What is the risk? > > Fixes: commit 0df21c86bdbf ("scsi: implement .get_budget and .put_budget for > blk-mq") > Signed-off-by: Bart V

Re: blk-mq + bfq: udevd hang on usb2 storages

2017-12-04 Thread Ming Lei
On Fri, Dec 01, 2017 at 06:04:29PM +0100, Alban Browaeys wrote: > I initially reported as https://bugzilla.kernel.org/show_bug.cgi?id=198 > 023 . > > I have now bisected this issue to commit a6a252e6491443c1c1 "blk-mq- > sched: decide how to handle flush rq via RQF_FLUSH_SEQ". > > This is with

Re: [PATCH] SCSI: delay run queue if device is blocked in scsi_dev_queue_ready()

2017-12-04 Thread Ming Lei
On Mon, Dec 04, 2017 at 09:44:55AM +0100, Johannes Thumshirn wrote: > Ming Lei <ming@redhat.com> writes: > > > > I am happy to do that, but recently I am very busy, so it may be done > > a bit late by me. > > > > But anyone should reproduce the

Re: [PATCH] SCSI: delay run queue if device is blocked in scsi_dev_queue_ready()

2017-12-04 Thread Ming Lei
On Mon, Dec 04, 2017 at 09:19:33AM +0100, Johannes Thumshirn wrote: > > Hi Ming, > > Ming Lei <ming@redhat.com> writes: > > This issue can be triggered by the following script: > > > > #!/bin/sh > > rmmod scsi_debug > > modprobe

[PATCH] SCSI: delay run queue if device is blocked in scsi_dev_queue_ready()

2017-12-02 Thread Ming Lei
ct count=1 & sleep 5 echo "0" >/sys/bus/pseudo/drivers/scsi_debug/opts wait echo "SUCCESS" Fixes: 0df21c86bdbf ("scsi: implement .get_budget and .put_budget for blk-mq") Signed-off-by: Ming Lei <ming@redhat.com> ---

Re: [PATCH 4/7] blk-mq: Avoid that request processing stalls when sharing tags

2017-12-01 Thread Ming Lei
On Sat, Dec 02, 2017 at 01:05:05AM +, Bart Van Assche wrote: > On Sat, 2017-12-02 at 09:00 +0800, Ming Lei wrote: > > On Sat, Dec 02, 2017 at 12:48:51AM +, Bart Van Assche wrote: > > > Further tests have shown that the lockup I referred to does not occur &

Re: [PATCH 4/7] blk-mq: Avoid that request processing stalls when sharing tags

2017-12-01 Thread Ming Lei
On Sat, Dec 02, 2017 at 01:05:05AM +, Bart Van Assche wrote: > On Sat, 2017-12-02 at 09:00 +0800, Ming Lei wrote: > > On Sat, Dec 02, 2017 at 12:48:51AM +, Bart Van Assche wrote: > > > Further tests have shown that the lockup I referred to does not occur &

Re: [PATCH 4/7] blk-mq: Avoid that request processing stalls when sharing tags

2017-12-01 Thread Ming Lei
On Sat, Dec 02, 2017 at 12:48:51AM +, Bart Van Assche wrote: > On Sat, 2017-12-02 at 08:36 +0800, Ming Lei wrote: > > On Fri, Dec 01, 2017 at 07:52:14PM +, Bart Van Assche wrote: > > > On Fri, 2017-12-01 at 10:58 +0800, Ming Lei wrote: > > > > On Thu, Nov 30,

Re: [PATCH V2 0/2] block: fix queue freeze and cleanup

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 04:49:31PM -0200, Mauricio Faria de Oliveira wrote: > Hi Bart, > > On 12/01/2017 02:08 PM, Bart Van Assche wrote: > > The test chenxiang ran does not prove that there is anything wrong with v2. > > Maybe chenxiang hit the issue described > >

Re: [PATCH 4/7] blk-mq: Avoid that request processing stalls when sharing tags

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 07:52:14PM +, Bart Van Assche wrote: > On Fri, 2017-12-01 at 10:58 +0800, Ming Lei wrote: > > On Thu, Nov 30, 2017 at 04:08:45PM -0800, Bart Van Assche wrote: > > > blk_mq_sched_mark_restart_hctx() must be called before > > > > Could

Re: [PATCH V2 0/2] block: fix queue freeze and cleanup

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 04:08:49PM +, Bart Van Assche wrote: > On Fri, 2017-12-01 at 13:36 -0200, Mauricio Faria de Oliveira wrote: > > On 11/29/2017 12:57 AM, chenxiang (M) wrote: > > > I applied this v2 patchset to kernel 4.15-rc1, running fio on a SATA > > > disk, then disable the disk

Re: blk-mq + bfq: udevd hang on usb2 storages

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 06:04:29PM +0100, Alban Browaeys wrote: > I initially reported as https://bugzilla.kernel.org/show_bug.cgi?id=198 > 023 . > > I have now bisected this issue to commit a6a252e6491443c1c1 "blk-mq- > sched: decide how to handle flush rq via RQF_FLUSH_SEQ". > > This is with

Re: [PATCH V2 0/2] block: fix queue freeze and cleanup

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 01:36:13PM -0200, Mauricio Faria de Oliveira wrote: > Hi Ming Lei, > > On 11/29/2017 12:57 AM, chenxiang (M) wrote: > > I applied this v2 patchset to kernel 4.15-rc1, running fio on a SATA > > disk, then disable the disk with sysfs interface >

Re: blk-mq + bfq IO hangs after writing partition table

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 02:46:39PM +, Ivan Kozik wrote: > On Fri, Dec 1, 2017 at 1:35 PM, Ming Lei <ming@redhat.com> wrote: > > Basically all IO hang happen inside get_request(), and seems this issue > > isn't related with recent change in V4.15, could

Re: blk-mq + bfq IO hangs after writing partition table

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 12:18:25PM +, Ivan Kozik wrote: > On Fri, Dec 1, 2017 at 8:50 AM, Ming Lei <ming@redhat.com> wrote: > > Could you run the following script[1] and provide us the result after > > the IO hang is triggered? > > > > #./dump-

Re: blk-mq + bfq IO hangs after writing partition table

2017-12-01 Thread Ming Lei
On Fri, Dec 01, 2017 at 06:52:37AM +, i...@ludios.org wrote: > > Hi, > > I think I am triggering a blk-mq + bfq bug that I can reproduce 100% > of the time by using gdisk (1.0.1-1 in Debian stretch) to write a > partition table to a USB flash drive. After it is triggered, IO hangs > forever

Re: [PATCH 5/7] blk-mq: Avoid that blk_mq_mark_tag_wait() triggers a queue stall

2017-11-30 Thread Ming Lei
ue is shared. > > Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> > Cc: Ming Lei <ming@redhat.com> > Cc: Omar Sandoval <osan...@fb.com> > Cc: Christoph Hellwig <h...@lst.de> > Cc: Hannes Reinecke <h...@suse.de> > Cc: Johannes T

Re: [PATCH 6/7] blk-mq: Rerun hardware queues after having called .put_budget()

2017-11-30 Thread Ming Lei
> returns BLK_STS_RESOURCE") > Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> > Cc: Ming Lei <ming@redhat.com> > Cc: Omar Sandoval <osan...@fb.com> > Cc: Christoph Hellwig <h...@lst.de> > Cc: Hannes Reinecke <h...@suse.de> > Cc: Joh

Re: [PATCH 4/7] blk-mq: Avoid that request processing stalls when sharing tags

2017-11-30 Thread Ming Lei
ore b347689ffbca, which doesn't change this RESTART mechanism, so please explain a bit why it is a fix on commit b347689ffbca. > Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> > Cc: Ming Lei <ming@redhat.com> > Cc: Omar Sandoval <osan...@fb.com> > Cc: Christ

Re: [PATCH V2 1/5] dm-mpath: don't call blk_mq_delay_run_hw_queue() in case of BLK_STS_RESOURCE

2017-11-30 Thread Ming Lei
On Mon, Nov 27, 2017 at 05:14:46PM +, Bart Van Assche wrote: > On Mon, 2017-11-27 at 13:07 +0800, Ming Lei wrote: > > If .queue_rq() returns BLK_STS_RESOURCE, blk-mq will rerun the queue in > > the three situations: > > > > 1) if BLK_MQ_S_SCHED_RESTART is set > &

[PATCH V3] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-29 Thread Ming Lei
q_usage_counter also track legacy requests") Cc: Wen Xiong <wenxi...@us.ibm.com> Tested-by: "chenxiang (M)" <chenxian...@hisilicon.com> Tested-by: Mauricio Faria de Oliveira <mauri...@linux.vnet.ibm.com> Signed-off-by: Ming Lei <ming@redhat.com> --- V3:

Re: [PATCH V2 0/2] block: fix queue freeze and cleanup

2017-11-28 Thread Ming Lei
Hi Chenxiang, On Wed, Nov 29, 2017 at 10:57:06AM +0800, chenxiang (M) wrote: > 在 2017/11/27 20:41, Ming Lei 写道: > > On Thu, Nov 23, 2017 at 12:47:58PM +0800, Ming Lei wrote: > > > Hi Jens, > > > > > > The 1st patch runs queue in blk_freeze_queue_sta

Re: [PATCH V2 0/2] block: fix queue freeze and cleanup

2017-11-27 Thread Ming Lei
On Thu, Nov 23, 2017 at 12:47:58PM +0800, Ming Lei wrote: > Hi Jens, > > The 1st patch runs queue in blk_freeze_queue_start() for fixing one > regression by 055f6e18e08f("block: Make q_usage_counter also track legacy > requests"). > > The 2nd patch drians blkcg par

[PATCH V2 5/5] blk-mq: issue request directly for blk_insert_cloned_request

2017-11-26 Thread Ming Lei
off-by: Ming Lei <ming@redhat.com> --- block/blk-core.c | 3 +-- block/blk-mq.c | 32 +--- block/blk-mq.h | 3 +++ drivers/md/dm-rq.c | 19 --- 4 files changed, 49 insertions(+), 8 deletions(-) diff --git a/block/blk-core.c b/block/b

[PATCH V2 3/5] blk-mq: move actual issue into one helper

2017-11-26 Thread Ming Lei
No functional change, just to clean up code a bit, so that the following change of using direct issue for blk_mq_request_bypass_insert() which is needed by DM can be easier to do. Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-mq.c | 39 +++-

[PATCH V2 4/5] blk-mq: return dispatch result to caller in blk_mq_try_issue_directly

2017-11-26 Thread Ming Lei
In the following patch, we will use blk_mq_try_issue_directly() for DM to return the dispatch result, and DM need this informatin to improve IO merge. Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-mq.c | 27 +-- 1 file changed, 17 insertions(+), 10 del

[PATCH V2 1/5] dm-mpath: don't call blk_mq_delay_run_hw_queue() in case of BLK_STS_RESOURCE

2017-11-26 Thread Ming Lei
makes sure that dm-rq returns BLK_STS_RESOURCE to blk-mq only when underlying queue is out of resource, so we switch to return DM_MAPIO_DELAY_REQUEU if either MPATHF_QUEUE_IO or MPATHF_PG_INIT_REQUIRED is set in multipath_clone_and_map(). Signed-off-by: Ming Lei <ming@redhat.com> --- driv

[PATCH V2 2/5] dm-mpath: return DM_MAPIO_REQUEUE in case of rq allocation failure

2017-11-26 Thread Ming Lei
gets improved. Signed-off-by: Ming Lei <ming@redhat.com> --- drivers/md/dm-mpath.c | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c index 8fe3f45407ce..1fa1ff5bfe73 100644 --- a/drivers/md/dm-mpath.c +++ b/d

[PATCH V2 2/2] block: drain blkcg part of request_queue in blk_cleanup_queue()

2017-11-22 Thread Ming Lei
t.ibm.com> Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-core.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 1038706edd87..f3f6f11a5b31 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -689,8 +689,7 @@ void

[PATCH V2 0/2] block: fix queue freeze and cleanup

2017-11-22 Thread Ming Lei
eanup. V2: - follow Bart's suggestion to use run queue instead of drain queue - drians blkcg part of request_queue for blk-mq thanks, Ming Ming Lei (2): block: run queue before waiting for q_usage_counter becoming zero block: drain blkcg part of request_queue after queue is frozen

[PATCH V2 1/2] block: run queue before waiting for q_usage_counter becoming zero

2017-11-22 Thread Ming Lei
t;mauri...@linux.vnet.ibm.com> Suggested-by: Bart Van Assche <bart.vanass...@wdc.com> Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-mq.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index 11097477eeab..e2b6a57b004d 100644 --- a/b

Re: [PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-22 Thread Ming Lei
On Wed, Nov 22, 2017 at 04:47:48PM +, Bart Van Assche wrote: > On Wed, 2017-11-22 at 13:11 +0800, Ming Lei wrote: > > diff --git a/block/blk-mq.c b/block/blk-mq.c > > index 11097477eeab..3d3797327491 100644 > > --- a/block/blk-mq.c > > +++ b/block/blk-mq.c &

Re: [PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-22 Thread Ming Lei
On Wed, Nov 22, 2017 at 08:04:13AM +0100, Hannes Reinecke wrote: > On 11/22/2017 06:11 AM, Ming Lei wrote: > > Now we track legacy requests with .q_usage_counter in commit 055f6e18e08f > > ("block: Make q_usage_counter also track legacy requests"), but that > > comm

[PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-21 Thread Ming Lei
during IO. This patch fixes the issue by draining requests before waiting for q_usage_counter becoming zero. Fixes: 055f6e18e08f("block: Make q_usage_counter also track legacy requests") Cc: Wen Xiong <wenxi...@us.ibm.com> Signed-off-by: Ming Lei <ming@redhat.com> --- block

Re: [PATCH] bio: ensure __bio_clone_fast copies bi_partno

2017-11-17 Thread Ming Lei
*/ > bio->bi_disk = bio_src->bi_disk; > + bio->bi_partno = bio_src->bi_partno; > bio_set_flag(bio, BIO_CLONED); > bio->bi_opf = bio_src->bi_opf; > bio->bi_write_hint = bio_src->bi_write_hint; Reviewed-by: Ming Lei <ming@redhat.com> -- Ming

[PATCH] block: wake up all tasks blocked in get_request()

2017-11-15 Thread Ming Lei
first. Fixes: 3ef28e83ab157997(block: generic request_queue reference counting) Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 7c54c195e79e..1038706edd87 100644

Re: [PATCH V4] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-15 Thread Ming Lei
On Wed, Nov 15, 2017 at 07:28:00PM +0900, James Bottomley wrote: > On Wed, 2017-11-15 at 18:09 +0800, Ming Lei wrote: > > On Tue, Nov 14, 2017 at 10:14:52AM -0800, James Bottomley wrote: > > > > > > On Tue, 2017-11-14 at 08:55 +0800, Ming Lei wrote

Re: [PATCH V4] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-15 Thread Ming Lei
On Tue, Nov 14, 2017 at 10:14:52AM -0800, James Bottomley wrote: > On Tue, 2017-11-14 at 08:55 +0800, Ming Lei wrote: > > Hi James, > > > > On Mon, Nov 13, 2017 at 10:55:52AM -0800, James Bottomley wrote: > > > > > > On Sat, 2017-11-11 at 10:43 +0800, M

Re: [PATCH V4] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-13 Thread Ming Lei
Hi James, On Mon, Nov 13, 2017 at 10:55:52AM -0800, James Bottomley wrote: > On Sat, 2017-11-11 at 10:43 +0800, Ming Lei wrote: > > On Fri, Nov 10, 2017 at 08:51:58AM -0800, James Bottomley wrote: > > > > > > On Fri, 2017-11-10 at 17:01 +0800, Ming Lei wrote: >

Re: [PATCH V4] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-10 Thread Ming Lei
On Fri, Nov 10, 2017 at 08:51:58AM -0800, James Bottomley wrote: > On Fri, 2017-11-10 at 17:01 +0800, Ming Lei wrote: > > cmd->cmnd can be allocated/freed dynamically in case of > > T10_PI_TYPE2_PROTECTION, > > so we should check it in scsi_show_rq() because this request

Re: [PATCH] blk-mq: respect queue dead via blk_mq_quiesce_queue

2017-11-10 Thread Ming Lei
On Fri, Nov 10, 2017 at 04:30:35PM +, Bart Van Assche wrote: > On Sun, 2017-11-05 at 15:38 +, Bart Van Assche wrote: > > On Sun, 2017-11-05 at 20:10 +0800, Ming Lei wrote: > > > diff --git a/block/blk-core.c b/block/blk-core.c > > > index 048be4aa6024..0b121f29e3

[PATCH V4] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-10 Thread Ming Lei
; Cc: Hannes Reinecke <h...@suse.com> Signed-off-by: Ming Lei <ming@redhat.com> --- V2: - fix typo V3: - prefer to dump data and accept read-after-free - add some comment V4: - read the two variable into local variable first, for avoiding free be

Re: [PATCH] blk-mq: fix issue with shared tag queue re-running

2017-11-09 Thread Ming Lei
On Fri, Nov 10, 2017 at 01:53:18PM +0800, Ming Lei wrote: > On Thu, Nov 09, 2017 at 09:32:58AM -0700, Jens Axboe wrote: > > On 11/09/2017 08:30 AM, Jens Axboe wrote: > > > On 11/09/2017 03:00 AM, Ming Lei wrote: > > >> On Thu, Nov 09, 2017 at 11:41:40AM +0800, Ming

Re: [PATCH] blk-mq: respect queue dead via blk_mq_quiesce_queue

2017-11-09 Thread Ming Lei
On Sun, Nov 05, 2017 at 08:10:08PM +0800, Ming Lei wrote: > blk-mq never respects queue dead, and this may cause use-after-free on > any kind of queue resources. This patch respects the rule by calling > blk_mq_quiesce_queue() when queue is marked as DEAD. > > This patch fixe

Re: [PATCH] blk-mq: fix issue with shared tag queue re-running

2017-11-09 Thread Ming Lei
On Thu, Nov 09, 2017 at 09:32:58AM -0700, Jens Axboe wrote: > On 11/09/2017 08:30 AM, Jens Axboe wrote: > > On 11/09/2017 03:00 AM, Ming Lei wrote: > >> On Thu, Nov 09, 2017 at 11:41:40AM +0800, Ming Lei wrote: > >>> On Wed, Nov 08, 2017 at 03:48:51PM -0700, Jens

Re: [PATCH] blk-mq: fix issue with shared tag queue re-running

2017-11-09 Thread Ming Lei
On Thu, Nov 09, 2017 at 11:41:40AM +0800, Ming Lei wrote: > On Wed, Nov 08, 2017 at 03:48:51PM -0700, Jens Axboe wrote: > > This patch attempts to make the case of hctx re-running on driver tag > > failure more robust. Without this patch, it's pretty easy to trigger a >

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-08 Thread Ming Lei
On Wed, Nov 08, 2017 at 10:57:23AM -0700, Jens Axboe wrote: > On 11/08/2017 09:41 AM, Bart Van Assche wrote: > > On Tue, 2017-11-07 at 20:06 -0700, Jens Axboe wrote: > >> At this point, I have no idea what Bart's setup looks like. Bart, it > >> would be REALLY helpful if you could tell us how you

Re: [PATCH] blk-mq: fix issue with shared tag queue re-running

2017-11-08 Thread Ming Lei
goto free_bitmap; > diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h > index 674641527da7..4ae987c2352c 100644 > --- a/include/linux/blk-mq.h > +++ b/include/linux/blk-mq.h > @@ -35,7 +35,7 @@ struct blk_mq_hw_ctx { > struct blk_mq_ctx **ctxs; > unsigned intnr_ctx; > > - wait_queue_entry_t dispatch_wait; > + wait_queue_entry_t dispatch_wait; > atomic_twait_index; > > struct blk_mq_tags *tags; > @@ -181,8 +181,7 @@ enum { > BLK_MQ_S_STOPPED= 0, > BLK_MQ_S_TAG_ACTIVE = 1, > BLK_MQ_S_SCHED_RESTART = 2, > - BLK_MQ_S_TAG_WAITING= 3, > - BLK_MQ_S_START_ON_RUN = 4, > + BLK_MQ_S_START_ON_RUN = 3, > > BLK_MQ_MAX_DEPTH= 10240, Looks the approach is smart, and effective, since requests are often completed at batch. No regression on scsi test too. Reviewed-by: Ming Lei <ming@redhat.com> -- Ming

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-08 Thread Ming Lei
On Wed, Nov 08, 2017 at 04:41:35PM +, Bart Van Assche wrote: > On Tue, 2017-11-07 at 20:06 -0700, Jens Axboe wrote: > > At this point, I have no idea what Bart's setup looks like. Bart, it > > would be REALLY helpful if you could tell us how you are reproducing > > your hang. I don't know why

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-08 Thread Ming Lei
On Wed, Nov 08, 2017 at 02:20:41PM +0800, Ming Lei wrote: > On Tue, Nov 07, 2017 at 08:17:59PM -0700, Jens Axboe wrote: > > On 11/07/2017 08:12 PM, Ming Lei wrote: > > > On Tue, Nov 07, 2017 at 08:01:48PM -0700, Jens Axboe wrote: > > >> On 11/07/2017 06:03 PM, Ming

[PATCH V3] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-08 Thread Ming Lei
;bart.vanass...@sandisk.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: James Bottomley <james.bottom...@hansenpartnership.com> Cc: Hannes Reinecke <h...@suse.com> Signed-off-by: Ming Lei <ming@redhat.com> --- V2:

Re: [PATCH V2] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-07 Thread Ming Lei
Hi James, On Tue, Nov 07, 2017 at 07:04:23PM -0800, James Bottomley wrote: > On Wed, 2017-11-08 at 09:15 +0800, Ming Lei wrote: > > On Wed, Nov 08, 2017 at 01:06:44AM +, Bart Van Assche wrote: > > > > > > On Wed, 2017-11-08 at 08:59 +0800, Ming Lei wrote: >

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 08:01:48PM -0700, Jens Axboe wrote: > On 11/07/2017 06:03 PM, Ming Lei wrote: > > On Tue, Nov 07, 2017 at 03:06:31PM -0700, Jens Axboe wrote: > >> On 11/07/2017 10:36 AM, Jens Axboe wrote: > >>> On 11/07/2017 10:10 AM, Jens Axboe wrote: >

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 07:55:32PM -0700, Jens Axboe wrote: > On 11/07/2017 05:39 PM, Ming Lei wrote: > > On Tue, Nov 07, 2017 at 04:20:08PM +, Bart Van Assche wrote: > >> On Tue, 2017-11-07 at 10:11 +0800, Ming Lei wrote: > >>> If you can reproduce, please pro

Re: [PATCH] blk-mq: respect queue dead via blk_mq_quiesce_queue

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 10:27:11AM +0800, Ming Lei wrote: > On Mon, Nov 06, 2017 at 04:34:21PM +, Bart Van Assche wrote: > > On Mon, 2017-11-06 at 11:44 +0800, Ming Lei wrote: > > > On Sun, Nov 05, 2017 at 03:38:49PM +, Bart Van Assche wrote: > > > > On Sun, 2

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Wed, Nov 08, 2017 at 08:53:38AM +0800, Ming Lei wrote: > On Tue, Nov 07, 2017 at 05:34:38PM +, Bart Van Assche wrote: > > On Tue, 2017-11-07 at 09:29 -0700, Jens Axboe wrote: > > > On 11/07/2017 09:20 AM, Bart Van Assche wrote: > > > > On Tue, 2017-11-07

Re: [PATCH V2] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-07 Thread Ming Lei
On Wed, Nov 08, 2017 at 01:06:44AM +, Bart Van Assche wrote: > On Wed, 2017-11-08 at 08:59 +0800, Ming Lei wrote: > > On Tue, Nov 07, 2017 at 04:13:48PM +, Bart Van Assche wrote: > > > On Tue, 2017-11-07 at 23:21 +0800, Ming Lei wrote: > > > > diff --git a

[PATCH V2] blk-mq: put driver tag if dispatch budget can't be got

2017-11-07 Thread Ming Lei
We have to put the driver tag if dispatch budget can't be got, otherwise it might cause IO deadlock, especially in case that size of tags is very small. Fixes: de1482974080(blk-mq: introduce .get_budget and .put_budget in blk_mq_ops) Signed-off-by: Ming Lei <ming@redhat.com>

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 03:06:31PM -0700, Jens Axboe wrote: > On 11/07/2017 10:36 AM, Jens Axboe wrote: > > On 11/07/2017 10:10 AM, Jens Axboe wrote: > >> On 11/07/2017 09:29 AM, Jens Axboe wrote: > >>> On 11/07/2017 09:20 AM, Bart Van Assche wrote: > >>>

Re: [PATCH V2] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 04:13:48PM +, Bart Van Assche wrote: > On Tue, 2017-11-07 at 23:21 +0800, Ming Lei wrote: > > cmd->cmnd can be allocated/freed dynamically in case of > > T10_PI_TYPE2_PROTECTION, > > so we can't access it in scsi_show_rq() if 'SCpnt->cmn

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 05:34:38PM +, Bart Van Assche wrote: > On Tue, 2017-11-07 at 09:29 -0700, Jens Axboe wrote: > > On 11/07/2017 09:20 AM, Bart Van Assche wrote: > > > On Tue, 2017-11-07 at 10:11 +0800, Ming Lei wrote: > > > > If you can reproduce, please pro

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 10:34:35PM +, Bart Van Assche wrote: > On Tue, 2017-11-07 at 15:06 -0700, Jens Axboe wrote: > > Just to keep everyone in the loop, this bug is not new to > > for-4.15/block, nor is it new to the current 4.41-rc or 4.13. So it's > > probably different to what Bart is

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 04:20:08PM +, Bart Van Assche wrote: > On Tue, 2017-11-07 at 10:11 +0800, Ming Lei wrote: > > If you can reproduce, please provide me at least the following log > > first: > > > > find /sys/kernel/debug/block -name tags | xargs cat

Re: [PATCH] blk-mq: put driver tag if dispatch budget can't be got

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 09:23:19AM -0700, Jens Axboe wrote: > On 11/07/2017 02:59 AM, Ming Lei wrote: > > We have to put the driver tag if dispatch budget can't be got, otherwise > > it might cause IO deadlock, especially in case that size of tags is very > > small. > &

[PATCH V2] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-07 Thread Ming Lei
_rq()) Cc: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: James Bottomley <james.bottom...@hansenpartnership.com> Cc: Hannes Reinecke <h...@suse.com> Cc: linux-s...@vger.kernel.org

Re: [PATCH] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 03:04:09PM +, John Garry wrote: > Hi Ming, > > On 07/11/2017 14:55, Ming Lei wrote: > > + strcpy(buf, "unknow"); > > typo? Yeah, will fix it in V2, thanks! -- Ming

[PATCH] scsi_debugfs: fix crash in scsi_show_rq()

2017-11-07 Thread Ming Lei
.org Cc: Jens Axboe <ax...@fb.com> Signed-off-by: Ming Lei <ming@redhat.com> --- drivers/scsi/scsi_debugfs.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/scsi_debugfs.c b/drivers/scsi/scsi_debugfs.c index 5e9755008aed..fe3efefc00d3 100644 --- a/driv

Re: [PATCH V8 0/8] block/scsi: safe SCSI quiescing

2017-11-07 Thread Ming Lei
On Tue, Nov 07, 2017 at 01:32:44PM +0100, Oleksandr Natalenko wrote: > Hi Ming, Jens. > > What is the fate of this patchset please? Hi Oleksandr, Recently Bart is working on this patchset, and you may try to see if his V11 works for you: https://marc.info/?t=15094035109=1=2

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-07 Thread Ming Lei
On Mon, Nov 06, 2017 at 07:45:23PM +, Bart Van Assche wrote: > On Sat, 2017-11-04 at 08:19 -0600, Jens Axboe wrote: > > On 11/03/2017 07:55 PM, Ming Lei wrote: > > > It is very expensive to atomic_inc/atomic_dec the host wide counter of > > > host->busy_count, a

[PATCH] blk-mq: put driver tag if dispatch budget can't be got

2017-11-07 Thread Ming Lei
We have to put the driver tag if dispatch budget can't be got, otherwise it might cause IO deadlock, especially in case that size of tags is very small. Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-mq.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a

Re: [PATCH] blk-mq: Revert multiple recent patches

2017-11-06 Thread Ming Lei
On Mon, Nov 06, 2017 at 05:07:31PM -0800, Bart Van Assche wrote: > Since we are close to the merge window and since it is too late for > reworking patches and algorithms that have turned out to be flawed, > revert multiple recent patches. Revert patch "blk-mq: don't handle > TAG_SHARED in restart"

Re: [PATCH] blk-mq: respect queue dead via blk_mq_quiesce_queue

2017-11-06 Thread Ming Lei
On Mon, Nov 06, 2017 at 04:34:21PM +, Bart Van Assche wrote: > On Mon, 2017-11-06 at 11:44 +0800, Ming Lei wrote: > > On Sun, Nov 05, 2017 at 03:38:49PM +, Bart Van Assche wrote: > > > On Sun, 2017-11-05 at 20:10 +0800, Ming Lei wrote: > > > > diff --git

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-06 Thread Ming Lei
On Mon, Nov 06, 2017 at 06:04:42PM +, Bart Van Assche wrote: > On Sat, 2017-11-04 at 09:55 +0800, Ming Lei wrote: > > It is very expensive to atomic_inc/atomic_dec the host wide counter of > > host->busy_count, and it should have been avoided via blk-mq's mechanism > &g

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

2017-11-06 Thread Ming Lei
On Mon, Nov 06, 2017 at 07:45:23PM +, Bart Van Assche wrote: > On Sat, 2017-11-04 at 08:19 -0600, Jens Axboe wrote: > > On 11/03/2017 07:55 PM, Ming Lei wrote: > > > It is very expensive to atomic_inc/atomic_dec the host wide counter of > > > host->busy_count, a

Re: [PATCH v2] blktrace: cleanup/fix registration locking

2017-11-06 Thread Ming Lei
On Mon, Nov 6, 2017 at 12:21 AM, Jens Axboe <ax...@kernel.dk> wrote: > One patch that fixes up a discrepancy in how we lock the queue for > blktrace registration, and one that fixes up how we globally > serialize registering the tracepoints. > Reviewed-by: Ming Lei &l

[PATCH] blk-mq: quiesce queue during switching io sched and updating nr_requests

2017-11-06 Thread Ming Lei
blk_mq_quiesce_queue() after queue is frozen) Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-mq.c | 2 ++ block/elevator.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index c501cbd0de93..88c3e4c89052 100644 --- a/block/blk-mq.c +++ b

Re: [PATCH] blk-mq: respect queue dead via blk_mq_quiesce_queue

2017-11-05 Thread Ming Lei
On Sun, Nov 05, 2017 at 03:38:49PM +, Bart Van Assche wrote: > On Sun, 2017-11-05 at 20:10 +0800, Ming Lei wrote: > > diff --git a/block/blk-core.c b/block/blk-core.c > > index 048be4aa6024..0b121f29e3b1 100644 > > --- a/block/blk-core.c > > +++ b/block/blk-core.c &g

[PATCH] blk-mq: respect queue dead via blk_mq_quiesce_queue

2017-11-05 Thread Ming Lei
172011] Dumping ftrace buffer: [ 42.172011](ftrace buffer empty) [ 42.172011] Kernel Offset: disabled [ 42.172011] ---[ end Kernel panic - not syncing: Fatal exception Cc: sta...@vger.kernel.org Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-core.c | 4 1 fil

[PATCH] blk-mq: don't handle failure in .get_budget

2017-11-04 Thread Ming Lei
. And it isn't enough to simply call blk_mq_end_request() to do that if this request is marked as RQF_DONTPREP. Fixes: 0df21c86bdbf(scsi: implement .get_budget and .put_budget for blk-mq) Signed-off-by: Ming Lei <ming@redhat.com> --- block/blk-mq-sched.c| 14 ++ block/bl

Re: [PATCH V3 0/7] blk-mq: don't allocate driver tag beforehand for flush rq

2017-11-03 Thread Ming Lei
On Thu, Nov 02, 2017 at 11:24:31PM +0800, Ming Lei wrote: > Hi Jens, > > This patchset avoids to allocate driver tag beforehand for flush rq > in case of I/O scheduler, then flush rq isn't treated specially > wrt. get/put driver tag, code gets cleanup much, such as, > reo

<    5   6   7   8   9   10   11   12   13   14   >