[PATCH V10 3/8] sbitmap: introduce __sbitmap_for_each_set()

2017-10-14 Thread Ming Lei
We need to iterate ctx starting from any ctx in round robin way, so introduce this helper. Reviewed-by: Omar Sandoval Cc: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/sbitmap.h | 64 - 1 file changed, 47 insertions(+), 17 deletions

Re: [PATCH V10 0/8] blk-mq-sched: improve sequential I/O performance

2017-10-14 Thread Ming Lei
On Sat, Oct 14, 2017 at 07:38:29PM +0200, Oleksandr Natalenko wrote: > Hi. > > By any chance, could this be backported to 4.14? I'm confused with "SCSI: > allow to pass null rq to scsi_prep_state_check()" since it uses refactored > flags. > > === > if (req && !(req->rq_flags & RQF_PREEMPT)) > =

Re: [PATCH V10 0/8] blk-mq-sched: improve sequential I/O performance

2017-10-14 Thread Ming Lei
On Sat, Oct 14, 2017 at 09:39:21AM -0600, Jens Axboe wrote: > On 10/14/2017 03:22 AM, Ming Lei wrote: > > Hi Jens, > > > > In Red Hat internal storage test wrt. blk-mq scheduler, we found that I/O > > performance is much bad with mq-deadline, especially about seque

Re: [PATCH V7 4/6] blk-mq: introduce .get_budget and .put_budget in blk_mq_ops

2017-10-16 Thread Ming Lei
On Mon, Oct 16, 2017 at 01:30:09PM +0200, Hannes Reinecke wrote: > On 10/13/2017 07:29 PM, Ming Lei wrote: > > On Fri, Oct 13, 2017 at 05:08:52PM +, Bart Van Assche wrote: > >> On Sat, 2017-10-14 at 00:45 +0800, Ming Lei wrote: > >>> On Fri, Oct 13, 2017 at 04:

[PATCH 1/2] SCSI: run idle hctx after delay in scsi_mq_get_budget()

2017-10-16 Thread Ming Lei
If there isn't any outstanding request in this queue, both blk-mq's RESTART and SCSI's builtin RESTART can't work, so we have to deal with this case by running this queue after delay. Fixes: d04b6d97d0a1(scsi: implement .get_budget and .put_budget for blk-mq) Signed-off-by: Mi

[PATCH 0/2] block/SCSI MQ: two RESTART related patches

2017-10-16 Thread Ming Lei
g branch: https://github.com/ming1/linux/commits/blk_mq_improve_restart_V1 Ming Lei (2): SCSI: run idle hctx after delay in scsi_mq_get_budget() blk-mq: don't handle TAG_SHARED in restart block/blk-mq-sched.c| 78 +++-- drivers/scsi/sc

[PATCH 2/2] blk-mq: don't handle TAG_SHARED in restart

2017-10-16 Thread Ming Lei
t-in RESTART is enough to cover itself. So we don't need to pay special attention to TAG_SHARED wrt. restart. Signed-off-by: Ming Lei --- block/blk-mq-sched.c | 78 +++- 1 file changed, 4 insertions(+), 74 deletions(-) diff --git a/block/blk-mq-sc

Re: [PATCH 0/2] block/SCSI MQ: two RESTART related patches

2017-10-16 Thread Ming Lei
On Tue, Oct 17, 2017 at 01:04:16PM +0800, Ming Lei wrote: > Hi Jens, > > The 1st patch runs idle hctx after dealy in scsi_mq_get_budget(), > so that we can keep same behaviour with before, and it can be > thought as a fix. > > The 2nd patch cleans up RESTART, and removes ha

Re: [PATCH V7 4/6] blk-mq: introduce .get_budget and .put_budget in blk_mq_ops

2017-10-17 Thread Ming Lei
On Tue, Oct 17, 2017 at 08:38:01AM +0200, Hannes Reinecke wrote: > On 10/17/2017 03:29 AM, Ming Lei wrote: > > On Mon, Oct 16, 2017 at 01:30:09PM +0200, Hannes Reinecke wrote: > >> On 10/13/2017 07:29 PM, Ming Lei wrote: > >>> On Fri, Oct 13, 2017 at 05:08:52P

Re: [PATCH 0/2] block/SCSI MQ: two RESTART related patches

2017-10-17 Thread Ming Lei
On Tue, Oct 17, 2017 at 04:47:23PM +0100, John Garry wrote: > On 17/10/2017 06:12, Ming Lei wrote: > > On Tue, Oct 17, 2017 at 01:04:16PM +0800, Ming Lei wrote: > > > Hi Jens, > > > > > > The 1st patch runs idle hctx after dealy in scsi_mq_get_budget(), >

[PATCH V3 2/7] blk-flush: don't run queue for requests of bypassing flush

2017-11-02 Thread Ming Lei
blk_insert_flush() should only insert request since run queue always follows it. In case of bypassing flush, we don't need to run queue because every blk_insert_flush() follows one run queue. Signed-off-by: Ming Lei --- block/blk-flush.c | 2 +- 1 file changed, 1 insertion(+), 1 del

[PATCH V3 1/7] blk-mq: put the driver tag of nxt rq before first one is requeued

2017-11-02 Thread Ming Lei
From: Jianchao Wang When free the driver tag of the next rq with I/O scheduler configured, it get the first entry of the list, however, at the moment, the failed rq has been requeued at the head of the list. The rq it gets is the failed rq not the next rq. Free the driver tag of next rq before th

[PATCH V3 4/7] blk-flush: use blk_mq_request_bypass_insert()

2017-11-02 Thread Ming Lei
ts of bypassing flush machinery, just like what the legacy path did. Signed-off-by: Ming Lei --- block/blk-flush.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-flush.c b/block/blk-flush.c index 81bd1a843043..a9773d2075ac 100644 --- a/block/blk-flush.c +++ b/bloc

[PATCH V3 5/7] blk-mq-sched: decide how to handle flush rq via RQF_FLUSH_SEQ

2017-11-02 Thread Ming Lei
st is dispatched to ->dispatch list directly. This is a prepare patch for not preallocating driver tag on flush rq, and don't treat flush rq as special case, just what the legacy path did. Signed-off-by: Ming Lei --- block/blk-mq-sched.c | 29 - 1 file changed,

[PATCH V3 0/7] blk-mq: don't allocate driver tag beforehand for flush rq

2017-11-02 Thread Ming Lei
xt rq before first one is requeued Ming Lei (6): blk-flush: don't run queue for requests of bypassing flush block: pass 'run_queue' to blk_mq_request_bypass_insert blk-flush: use blk_mq_request_bypass_insert() blk-mq-sched: decide how to handle flush rq via RQF_FLUSH_SEQ blk-mq

[PATCH V3 3/7] block: pass 'run_queue' to blk_mq_request_bypass_insert

2017-11-02 Thread Ming Lei
Block flush need this function without running queue, so introduce the parameter. Signed-off-by: Ming Lei --- block/blk-core.c | 2 +- block/blk-mq.c | 5 +++-- block/blk-mq.h | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index

[PATCH V3 6/7] blk-mq: move blk_mq_put_driver_tag*() into blk-mq.h

2017-11-02 Thread Ming Lei
We need this helper to put the driver tag for flush rq, since we will not share tag in the flush request sequence in the following patch in case that I/O scheduler is applied. Signed-off-by: Ming Lei --- block/blk-mq.c | 32 block/blk-mq.h | 33

[PATCH V3 7/7] blk-mq: don't allocate driver tag beforehand for flush rq

2017-11-02 Thread Ming Lei
e driver tag before requeueing. Signed-off-by: Ming Lei --- block/blk-flush.c| 35 ++- block/blk-mq-sched.c | 42 +- block/blk-mq.c | 41 ++--- 3 files changed, 37 insertions(+), 81 de

Re: [PATCH 1/1] [RFC] blk-mq: fix queue stalling on shared hctx restart

2017-10-24 Thread Ming Lei
On Mon, Oct 23, 2017 at 06:12:29PM +0200, Roman Penyaev wrote: > Hi Ming, > > On Fri, Oct 20, 2017 at 3:39 PM, Ming Lei wrote: > > On Wed, Oct 18, 2017 at 12:22:06PM +0200, Roman Pen wrote: > >> Hi all, > >> > >> the patch below fixes queue st

Re: [PATCH V3 0/7] blk-mq: don't allocate driver tag beforehand for flush rq

2017-11-03 Thread Ming Lei
On Thu, Nov 02, 2017 at 11:24:31PM +0800, Ming Lei wrote: > Hi Jens, > > This patchset avoids to allocate driver tag beforehand for flush rq > in case of I/O scheduler, then flush rq isn't treated specially > wrt. get/put driver tag, code gets cleanup much, such as, > reo

Re: [PATCH v3 07/49] bcache: comment on direct access to bvec table

2017-10-19 Thread Ming Lei
On Thu, Aug 10, 2017 at 04:26:03AM -0700, Christoph Hellwig wrote: > I think all this bcache code needs bigger attention. For one > bio_alloc_pages is only used in bcache, so we should move it in there. Looks a good idea. > > Second the way bio_alloc_pages is currently written looks potentiall

Re: [PATCH v3 09/49] block: comment on bio_iov_iter_get_pages()

2017-10-19 Thread Ming Lei
On Thu, Aug 10, 2017 at 04:28:05AM -0700, Christoph Hellwig wrote: > > + * The hacking way of using bvec table as page pointer array is safe > > + * even after multipage bvec is introduced because that space can be > > + * thought as unused by bio_add_page(). > > I'm not sure what value this comme

Re: [PATCH v3 11/49] btrfs: avoid access to .bi_vcnt directly

2017-10-19 Thread Ming Lei
On Thu, Aug 10, 2017 at 04:29:59AM -0700, Christoph Hellwig wrote: > > +static unsigned int get_bio_pages(struct bio *bio) > > +{ > > + unsigned i; > > + struct bio_vec *bv; > > + > > + bio_for_each_segment_all(bv, bio, i) > > + ; > > + > > + return i; > > +} > > s/get_bio_pages/

Re: [PATCH v3 20/49] block: introduce bio_for_each_segment_mp()

2017-10-19 Thread Ming Lei
On Thu, Aug 10, 2017 at 05:11:10AM -0700, Christoph Hellwig wrote: > First: as mentioned in the previous patches I really hate the name > scheme with the _sp and _mp postfixes. > > To be clear and understandable we should always name the versions > that iterate over segments *segment* and the ones

Re: [PATCH v3 41/49] xfs: convert to bio_for_each_segment_all_sp()

2017-10-19 Thread Ming Lei
On Tue, Aug 08, 2017 at 09:32:32AM -0700, Darrick J. Wong wrote: > On Tue, Aug 08, 2017 at 04:45:40PM +0800, Ming Lei wrote: > > Sure would be nice to have a changelog explaining why we're doing this. > > > Cc: "Darrick J. Wong" > > Cc: linux-...@vger

Re: [PATCH v3 37/49] fs/mpage: convert to bio_for_each_segment_all_sp()

2017-10-19 Thread Ming Lei
On Thu, Aug 10, 2017 at 05:16:12AM -0700, Christoph Hellwig wrote: > > struct bio_vec *bv; > > + struct bvec_iter_all bia; > > int i; > > > > - bio_for_each_segment_all(bv, bio, i) { > > + bio_for_each_segment_all_sp(bv, bio, i, bia) { > > struct page *page = bv->bv_page

Re: linux-next: build failure after merge of the block tree

2015-10-06 Thread Ming Lei
On Tue, Oct 6, 2015 at 9:43 AM, Stephen Rothwell wrote: > Hi Jens, > > After merging the block tree, today's linux-next build (arm > multi_v7_defconfig) failed like this: > > drivers/block/loop.c: In function 'lo_rw_aio_complete': > drivers/block/loop.c:474:2: error: too few arguments to function

Re: [PATCH 1/2] block: generic request_queue reference counting

2015-10-06 Thread Ming Lei
On Tue, Oct 6, 2015 at 7:23 AM, Dan Williams wrote: > On Sun, Oct 4, 2015 at 12:52 AM, Ming Lei wrote: >> On Wed, Sep 30, 2015 at 8:41 AM, Dan Williams >> wrote: >>> Allow pmem, and other synchronous/bio-based block drivers, to fallback >> >> Just a bit cur

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

2015-08-28 Thread Ming Lei
On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez wrote: > On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote: >> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds >> wrote: >> > On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood >> > wrote: >> >

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

2015-08-29 Thread Ming Lei
On Sat, 29 Aug 2015 10:50:22 +0200 Arend van Spriel wrote: > On 08/29/2015 09:11 AM, Takashi Iwai wrote: > > On Sat, 29 Aug 2015 06:09:01 +0200, > > Ming Lei wrote: > >> > >> On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez wrote: > >>> On Thu, Aug

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

2015-08-31 Thread Ming Lei
On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel wrote: > On 08/29/2015 12:38 PM, Ming Lei wrote: >> >> On Sat, 29 Aug 2015 10:50:22 +0200 >> Arend van Spriel wrote: >> >>> On 08/29/2015 09:11 AM, Takashi Iwai wrote: >>>> >>>

[PATCH 0/4] block: some misc changes

2015-10-13 Thread Ming Lei
Hi, The 1st three patches are optimizations related with bio splitting. The 4th patch is to mark ctx as pending at batch in flush plug path. Thanks, -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo in

[PATCH 2/4] block: avoid to merge splitted bio

2015-10-13 Thread Ming Lei
The splitted bio has been already too fat to merge, so mark it as NOMERGE. Signed-off-by: Ming Lei --- block/blk-merge.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-merge.c b/block/blk-merge.c index 22293fd..de5716d8 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c

[PATCH 4/4] blk-mq: mark ctx as pending at batch in flush plug path

2015-10-13 Thread Ming Lei
Most of times, flush plug should be the hottest I/O path, so mark ctx as pending after all requests in the list are inserted. Signed-off-by: Ming Lei --- block/blk-mq.c | 18 +- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index

[PATCH 3/4] blk-mq: check bio_mergeable() early before merging

2015-10-13 Thread Ming Lei
It isn't necessary to try to merge the bio which is marked as NOMERGE. Signed-off-by: Ming Lei --- block/blk-mq.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 546b3b8..deb5f4c 100644 --- a/block/blk-mq.c +++ b/block/blk

[PATCH 1/4] block: setup bi_phys_segments after splitting

2015-10-13 Thread Ming Lei
The number of bio->bi_phys_segments is always obtained during bio splitting, so it is natural to setup it just after bio splitting, then we can avoid to compute nr_segment again during merge. Signed-off-by: Ming Lei --- block/blk-merge.c | 29 ++--- 1 file changed,

Re: [PATCH v2] block: flush queued bios when the process blocks

2015-10-14 Thread Ming Lei
sulting in imperfect 'in_generic_make_request' accounting that happens > lazily once the outermost plug completes blk_finish_plug. manifested as > dm-bufio.c:dm_bufio_prefetch's BUG_ON(dm_bufio_in_request()); hitting. Looks this problem should be related with above 'bio_lis

Re: [PATCH 4/4] blk-mq: mark ctx as pending at batch in flush plug path

2015-10-15 Thread Ming Lei
Hi Jeff, On Thu, Oct 15, 2015 at 11:26 PM, Jeff Moyer wrote: > Ming Lei writes: > >> Most of times, flush plug should be the hottest I/O path, >> so mark ctx as pending after all requests in the list are >> inserted. > > Hi, Ming, > > Did you see some perfo

Re: [PATCH 3/4] blk-mq: check bio_mergeable() early before merging

2015-10-15 Thread Ming Lei
On Thu, Oct 15, 2015 at 11:21 PM, Jeff Moyer wrote: > Ming Lei writes: > >> It isn't necessary to try to merge the bio which is marked >> as NOMERGE. >> >> Signed-off-by: Ming Lei >> --- >> block/blk-mq.c | 5 - >> 1 file changed, 4 inse

Re: [PATCH v2] block: flush queued bios when the process blocks

2015-10-15 Thread Ming Lei
On Thu, Oct 15, 2015 at 4:06 PM, Mike Snitzer wrote: > On Wed, Oct 14 2015 at 11:27pm -0400, > Ming Lei wrote: > >> On Sat, Oct 10, 2015 at 3:52 AM, Mike Snitzer wrote: >> > >> > Turns out that this change: >> > http://git.kernel.org/cgit/linux/ker

[PATCH v1 2/6] block: avoid to merge splitted bio

2015-10-17 Thread Ming Lei
The splitted bio has been already too fat to merge, so mark it as NOMERGE. Reviewed-by: Jeff Moyer Signed-off-by: Ming Lei --- block/blk-merge.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-merge.c b/block/blk-merge.c index 22293fd..de5716d8 100644 --- a/block/blk-merge.c

[PATCH v1 3/6] blk-mq: check bio_mergeable() early before merging

2015-10-17 Thread Ming Lei
It isn't necessary to try to merge the bio which is marked as NOMERGE. Signed-off-by: Ming Lei --- block/blk-mq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 546b3b8..d55d022 100644 --- a/block/blk-mq.c +++ b/block/blk

[PATCH v1 0/6] block: some misc changes

2015-10-17 Thread Ming Lei
Hi, The 1st four patches are optimizations related with bio splitting. The 5th patch is one fix for using trace_block_plug(). The 6th patch is to mark ctx as pending at batch in flush plug path. V1: - 3/6: don't check bio_mergeable() in blk_mq_attempt_merge() - 4/6: check bio_me

[PATCH v1 1/6] block: setup bi_phys_segments after splitting

2015-10-17 Thread Ming Lei
The number of bio->bi_phys_segments is always obtained during bio splitting, so it is natural to setup it just after bio splitting, then we can avoid to compute nr_segment again during merge. Reviewed-by: Jeff Moyer Signed-off-by: Ming Lei --- block/blk-merge.c |

[PATCH v1 6/6] blk-mq: mark ctx as pending at batch in flush plug path

2015-10-17 Thread Ming Lei
Most of times, flush plug should be the hottest I/O path, so mark ctx as pending after all requests in the list are inserted. Signed-off-by: Ming Lei --- block/blk-mq.c | 18 +- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index

[PATCH v1 5/6] blk-mq: fix for trace_block_plug()

2015-10-17 Thread Ming Lei
The trace point is for tracing plug event of each request queue instead of each task, so we should check the request count in the plug list from current queue instead of current task. Signed-off-by: Ming Lei --- block/blk-mq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[PATCH v1 4/6] block: check bio_mergeable() early before merging

2015-10-17 Thread Ming Lei
After bio splitting is introduced, one bio can be splitted and it is marked as NOMERGE because it is too fat to be merged, so check bio_mergeable() earlier to avoid to try to merge it unnecessarily. Signed-off-by: Ming Lei --- block/elevator.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion

Re: [PATCH v2] block: flush queued bios when the process blocks

2015-10-17 Thread Ming Lei
On Fri, Oct 16, 2015 at 11:29 PM, Mike Snitzer wrote: > On Thu, Oct 15 2015 at 11:08pm -0400, > Ming Lei wrote: > >> On Thu, Oct 15, 2015 at 4:06 PM, Mike Snitzer wrote: >> > On Wed, Oct 14 2015 at 11:27pm -0400, >> > Ming Lei wrote: >> > >>

Re: [PATCH v3 for-4.4] block: flush queued bios when process blocks to avoid deadlock

2015-10-17 Thread Ming Lei
On Thu, Oct 15, 2015 at 4:47 AM, Mike Snitzer wrote: > From: Mikulas Patocka > > The block layer uses per-process bio list to avoid recursion in > generic_make_request. When generic_make_request is called recursively, > the bio is added to current->bio_list and generic_make_request returns > imm

Re: [PATCH v3 for-4.4] block: flush queued bios when process blocks to avoid deadlock

2015-10-21 Thread Ming Lei
On Wed, Oct 21, 2015 at 4:03 AM, Mikulas Patocka wrote: > > > On Sun, 18 Oct 2015, Ming Lei wrote: > >> On Thu, Oct 15, 2015 at 4:47 AM, Mike Snitzer wrote: >> > From: Mikulas Patocka >> > >> > The block layer uses per-process bio list to avoid

Re: [PATCH v3 for-4.4] block: flush queued bios when process blocks to avoid deadlock

2015-10-21 Thread Ming Lei
On Thu, Oct 22, 2015 at 5:49 AM, Mikulas Patocka wrote: > > > On Thu, 22 Oct 2015, Ming Lei wrote: > >> > Some drivers (dm-snapshot, dm-thin) do acquire a mutex in .make_requests() >> > for every bio. It wouldn't be practical to convert them to not acquir

Re: [PATCH v1 4/6] block: check bio_mergeable() early before merging

2015-10-19 Thread Ming Lei
On Mon, Oct 19, 2015 at 11:27 PM, Jeff Moyer wrote: > Ming Lei writes: > >> After bio splitting is introduced, one bio can be splitted >> and it is marked as NOMERGE because it is too fat to be merged, >> so check bio_mergeable() earlier to avoid to try to merge it >&g

Re: [PATCH v1 5/6] blk-mq: fix for trace_block_plug()

2015-10-19 Thread Ming Lei
On Mon, Oct 19, 2015 at 11:38 PM, Jeff Moyer wrote: > Ming Lei writes: > >> The trace point is for tracing plug event of each request >> queue instead of each task, so we should check the request >> count in the plug list from current queue instead of >> current t

Re: [PATCH v1 5/6] blk-mq: fix for trace_block_plug()

2015-10-19 Thread Ming Lei
On Tue, Oct 20, 2015 at 1:59 AM, Jeff Moyer wrote: > Ming Lei writes: > >> On Mon, Oct 19, 2015 at 11:38 PM, Jeff Moyer wrote: >>> Ming Lei writes: >>> >>>> The trace point is for tracing plug event of each request >>>> queue instead of e

[PATCH v2 5/7] block: check bio_mergeable() early before merging

2015-10-20 Thread Ming Lei
After bio splitting is introduced, one bio can be splitted and it is marked as NOMERGE because it is too fat to be merged, so check bio_mergeable() earlier to avoid to try to merge it unnecessarily. Signed-off-by: Ming Lei --- block/elevator.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion

[PATCH v2 4/7] blk-mq: check bio_mergeable() early before merging

2015-10-20 Thread Ming Lei
It isn't necessary to try to merge the bio which is marked as NOMERGE. Reviewed-by: Jeff Moyer Signed-off-by: Ming Lei --- block/blk-mq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 1796b31..edbc877 100644 --- a/block/blk-mq.c

[PATCH v2 7/7] blk-mq: mark ctx as pending at batch in flush plug path

2015-10-20 Thread Ming Lei
Most of times, flush plug should be the hottest I/O path, so mark ctx as pending after all requests in the list are inserted. Reviewed-by: Jeff Moyer Signed-off-by: Ming Lei --- block/blk-mq.c | 18 +- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/block/blk

[PATCH v2 2/7] block: setup bi_phys_segments after splitting

2015-10-20 Thread Ming Lei
The number of bio->bi_phys_segments is always obtained during bio splitting, so it is natural to setup it just after bio splitting, then we can avoid to compute nr_segment again during merge. Reviewed-by: Jeff Moyer Signed-off-by: Ming Lei --- block/blk-merge.c |

[PATCH v2 3/7] block: avoid to merge splitted bio

2015-10-20 Thread Ming Lei
The splitted bio has been already too fat to merge, so mark it as NOMERGE. Reviewed-by: Jeff Moyer Signed-off-by: Ming Lei --- block/blk-merge.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-merge.c b/block/blk-merge.c index 22293fd..de5716d8 100644 --- a/block/blk-merge.c

[PATCH v2 0/7] block: some misc changes

2015-10-20 Thread Ming Lei
Hi, The 1st patch is one fix for automatic flush plug in case of nomerge queue. The following four patches are optimizations related with bio splitting. The 6th patch is one fix for using trace_block_plug(). The 7th patch is to mark ctx as pending at batch in flush plug path. V2: - Jef

[PATCH v2 6/7] blk-mq: fix for trace_block_plug()

2015-10-20 Thread Ming Lei
The trace point is for tracing plug event of each request queue instead of each task, so we should check the request count in the plug list from current queue instead of current task. Signed-off-by: Ming Lei --- block/blk-mq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[PATCH v2 1/7] block: fix plug list flushing for nomerge queues

2015-10-20 Thread Ming Lei
From: Jeff Moyer Request queues with merging disabled will not flush the plug list after BLK_MAX_REQUEST_COUNT requests have been queued, since the code relies on blk_attempt_plug_merge to compute the request_count. Fix this by computing the number of queued requests even for nomerge queues. Si

Re: blk-mq: takes hours for scsi scanning finish when thousands of LUNs

2015-10-22 Thread Ming Lei
uquest queue should have been initialized completely, that said blk_mq_add_queue_tag_set() should be run in atomic mode of the usage ref counter. Not sure how can you observe the ref counter is running at percpu mode during queue initialization. Care to share what your disk/driver is? I doubt it

Re: blk_mq_register_disk: kobject (00000000009f2dd8): tried to init an initialized object, something is seriously wrong.

2015-10-03 Thread Ming Lei
On Sun, Oct 4, 2015 at 3:33 AM, Meelis Roos wrote: >> This is 4.3.0-rc1 on Sun E220R (dual-CPU sparc64). Sometimes it boots, >> sometimes it fails to boot with looping errors and finally a watchdog >> timeout. This console log from a failure. Config is below. > > I noticed blk-mq related changes i

Re: [PATCH 1/2] block: generic request_queue reference counting

2015-10-04 Thread Ming Lei
t same context > +* need not check that the queue has been frozen (marked dead). > +*/ > + percpu_ref_get(&q->q_usage_counter); > +} > > void blk_rq_timed_out_timer(unsigned long data); > unsigned long blk_rq_timeout(unsigned long timeout); &g

Re: Affinity managed interrupts vs non-managed interrupts

2018-08-29 Thread Ming Lei
Hello Sumit, On Tue, Aug 28, 2018 at 12:04:52PM +0530, Sumit Saxena wrote: > Affinity managed interrupts vs non-managed interrupts > > Hi Thomas, > > We are working on next generation MegaRAID product where requirement is- to > allocate additional 16 MSI-x vectors in addition to number of MSI-x

Re: How to secure erase PCI-E NVME SSD connected via USB3?

2018-07-31 Thread Ming Lei
> > So, how to secure-erase the NVME SSD connected via the JMS583 chip? You may try 'blkdiscard --secure' and see if you are luck. Thanks, Ming Lei

Re: [PATCH] block: neutralize blk_insert_cloned_request IO stall regression (was: Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle)

2018-01-23 Thread Ming Lei
Hi Mike, On Tue, Jan 23, 2018 at 10:22:04AM +0100, Mike Snitzer wrote: > On Thu, Jan 18 2018 at 5:20pm -0500, > Bart Van Assche wrote: > > > On Thu, 2018-01-18 at 17:01 -0500, Mike Snitzer wrote: > > > And yet Laurence cannot reproduce any such lockups with your test... > > > > Hmm ... maybe I

Re: block: neutralize blk_insert_cloned_request IO stall regression (was: Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle)

2018-01-23 Thread Ming Lei
On Tue, Jan 23, 2018 at 8:15 PM, Mike Snitzer wrote: > On Tue, Jan 23 2018 at 5:53am -0500, > Ming Lei wrote: > >> Hi Mike, >> >> On Tue, Jan 23, 2018 at 10:22:04AM +0100, Mike Snitzer wrote: >> > On Thu, Jan 18 2018 at 5:20pm -0500, >> > Bart Van

Re: [PATCH] block: blk-mq-sched: Replace GFP_ATOMIC with GFP_KERNEL in blk_mq_sched_assign_ioc

2018-01-24 Thread Ming Lei
On Wed, Jan 24, 2018 at 08:34:14PM -0700, Jens Axboe wrote: > On 1/24/18 7:46 PM, Jia-Ju Bai wrote: > > The function ioc_create_icq here is not called in atomic context. > > Thus GFP_ATOMIC is not necessary, and it can be replaced with GFP_KERNEL. > > > > This is found by a static analysis tool na

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

2018-01-11 Thread Ming Lei
On Wed, Dec 20, 2017 at 04:47:21PM +0100, Christian Borntraeger wrote: > On 12/18/2017 02:56 PM, Stefan Haberland wrote: > > On 07.12.2017 00:29, Christoph Hellwig wrote: > >> On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: > >> t > commit 11b2025c3326f7096ceb588c3117c7883850

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

2018-01-11 Thread Ming Lei
On Thu, Jan 11, 2018 at 06:46:54PM +0100, Christoph Hellwig wrote: > Thanks for looking into this Ming, I had missed it in the my current > work overload. Can you send the updated series to Jens? OK, I will post it out soon. Thanks, Ming

[PATCH 0/2] blk-mq: support physical CPU hotplug

2018-01-11 Thread Ming Lei
Hi, This two patches support physical CPU hotplug, so that we can make blk-mq scale well when new physical CPU is added or removed, and this use case is normal for VM world. Also this patchset fixes the following warning reported by Christian Borntraeger: https://marc.info/?l=linux-block

[PATCH 2/2] blk-mq: simplify queue mapping & schedule with each possisble CPU

2018-01-11 Thread Ming Lei
ne CPUs for schedule. Reported-by: Christian Borntraeger Tested-by: Christian Borntraeger Tested-by: Stefan Haberland Cc: Thomas Gleixner Signed-off-by: Christoph Hellwig (merged the three into one because any single one may not work, and fix selecting online CPUs for scheduler) Signed-off-by: Ming

[PATCH 1/2] genirq/affinity: assign vectors to all possible CPUs

2018-01-11 Thread Ming Lei
From: Christoph Hellwig Currently we assign managed interrupt vectors to all present CPUs. This works fine for systems were we only online/offline CPUs. But in case of systems that support physical CPU hotplug (or the virtualized version of it) this means the additional CPUs covered for in the

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

2018-01-13 Thread Ming Lei
On Fri, Jan 12, 2018 at 04:55:34PM -0500, Laurence Oberman wrote: > On Fri, 2018-01-12 at 20:57 +, Bart Van Assche wrote: > > On Tue, 2018-01-09 at 08:29 -0800, Tejun Heo wrote: > > > Currently, blk-mq timeout path synchronizes against the usual > > > issue/completion path using a complex schem

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

2018-01-13 Thread Ming Lei
On Fri, Jan 12, 2018 at 04:55:34PM -0500, Laurence Oberman wrote: > On Fri, 2018-01-12 at 20:57 +, Bart Van Assche wrote: > > On Tue, 2018-01-09 at 08:29 -0800, Tejun Heo wrote: > > > Currently, blk-mq timeout path synchronizes against the usual > > > issue/completion path using a complex schem

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

2018-01-13 Thread Ming Lei
On Sat, Jan 13, 2018 at 10:45:14PM +0800, Ming Lei wrote: > On Fri, Jan 12, 2018 at 04:55:34PM -0500, Laurence Oberman wrote: > > On Fri, 2018-01-12 at 20:57 +, Bart Van Assche wrote: > > > On Tue, 2018-01-09 at 08:29 -0800, Tejun Heo wrote: > > > > Currently, bl

Re: v4.15 and I/O hang with BFQ

2018-01-30 Thread Ming Lei
Hi, On Tue, Jan 30, 2018 at 09:05:26AM +0100, Oleksandr Natalenko wrote: > Hi, Paolo, Ivan, Ming et al. > > It looks like I've just encountered the issue Ivan has already described in > [1]. Since I'm able to reproduce it reliably in a VM, I'd like to draw more > attention to it. > > First, I'm

Re: v4.15 and I/O hang with BFQ

2018-01-30 Thread Ming Lei
On Tue, Jan 30, 2018 at 03:30:28PM +0100, Oleksandr Natalenko wrote: > Hi. > ... >systemd-udevd-271 [000] 4.311033: bfq_insert_requests: insert > rq->0 >systemd-udevd-271 [000] ...1 4.311037: blk_mq_do_dispatch_sched: > not get rq, 1 > cfdisk-408 [000]

Re: [PATCH 0/2] genirq/affinity: try to make sure online CPU is assgined to irq vector

2018-02-01 Thread Ming Lei
On Tue, Jan 16, 2018 at 03:22:18PM +, Don Brace wrote: > > -Original Message- > > From: Laurence Oberman [mailto:lober...@redhat.com] > > Sent: Tuesday, January 16, 2018 7:29 AM > > To: Thomas Gleixner ; Ming Lei > > Cc: Christoph Hellwig ; Jens Axboe ; &

Re: [PATCH 0/2] genirq/affinity: try to make sure online CPU is assgined to irq vector

2018-02-01 Thread Ming Lei
On Thu, Feb 01, 2018 at 02:53:35PM +, Don Brace wrote: > > -Original Message- > > From: Ming Lei [mailto:ming@redhat.com] > > Sent: Thursday, February 01, 2018 4:37 AM > > To: Don Brace > > Cc: Laurence Oberman ; Thomas Gleixner > > ; Christoph

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-18 Thread Ming Lei
On Thu, Jan 18, 2018 at 01:11:01PM -0700, Jens Axboe wrote: > On 1/18/18 11:47 AM, Bart Van Assche wrote: > >> This is all very tiresome. > > > > Yes, this is tiresome. It is very annoying to me that others keep > > introducing so many regressions in such important parts of the kernel. > > It is a

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-18 Thread Ming Lei
On Thu, Jan 18, 2018 at 09:02:45PM -0700, Jens Axboe wrote: > On 1/18/18 7:32 PM, Ming Lei wrote: > > On Thu, Jan 18, 2018 at 01:11:01PM -0700, Jens Axboe wrote: > >> On 1/18/18 11:47 AM, Bart Van Assche wrote: > >>>> This is all very tiresome. > >>

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-18 Thread Ming Lei
On Fri, Jan 19, 2018 at 05:09:46AM +, Bart Van Assche wrote: > On Fri, 2018-01-19 at 10:32 +0800, Ming Lei wrote: > > Now most of times both NVMe and SCSI won't return BLK_STS_RESOURCE, and > > it should be DM-only which returns STS_RESOURCE so often. > > That'

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 03:20:13PM +, Bart Van Assche wrote: > On Fri, 2018-01-19 at 15:26 +0800, Ming Lei wrote: > > Please see queue_delayed_work_on(), hctx->run_work is shared by all > > scheduling, once blk_mq_delay_run_hw_queue(100ms) returns, no new > > sche

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 08:24:06AM -0700, Jens Axboe wrote: > On 1/19/18 12:26 AM, Ming Lei wrote: > > On Thu, Jan 18, 2018 at 09:02:45PM -0700, Jens Axboe wrote: > >> On 1/18/18 7:32 PM, Ming Lei wrote: > >>> On Thu, Jan 18, 2018 at 01:11:01PM -0700, Jens Axboe wro

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 08:48:55AM -0700, Jens Axboe wrote: > On 1/19/18 8:40 AM, Ming Lei wrote: > >>>> Where does the dm STS_RESOURCE error usually come from - what's exact > >>>> resource are we running out of? > >>> >

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 09:19:24AM -0700, Jens Axboe wrote: > On 1/19/18 9:05 AM, Ming Lei wrote: > > On Fri, Jan 19, 2018 at 08:48:55AM -0700, Jens Axboe wrote: > >> On 1/19/18 8:40 AM, Ming Lei wrote: > >>>>>> Where does the dm STS_RESOURCE

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 09:27:46AM -0700, Jens Axboe wrote: > On 1/19/18 9:26 AM, Ming Lei wrote: > > On Fri, Jan 19, 2018 at 09:19:24AM -0700, Jens Axboe wrote: > >> On 1/19/18 9:05 AM, Ming Lei wrote: > >>> On Fri, Jan 19, 2018 at 08:48:55AM -0700, Jens Axboe wr

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 09:52:32AM -0700, Jens Axboe wrote: > On 1/19/18 9:47 AM, Mike Snitzer wrote: > > On Fri, Jan 19 2018 at 11:41am -0500, > > Jens Axboe wrote: > > > >> On 1/19/18 9:37 AM, Ming Lei wrote: > >>> On Fri, Jan 19, 2018 at 09:27:46AM -0

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 10:09:11AM -0700, Jens Axboe wrote: > On 1/19/18 10:05 AM, Ming Lei wrote: > > On Fri, Jan 19, 2018 at 09:52:32AM -0700, Jens Axboe wrote: > >> On 1/19/18 9:47 AM, Mike Snitzer wrote: > >>> On Fri, Jan 19 2018 at 11:41am -0500, > >>&g

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 10:38:41AM -0700, Jens Axboe wrote: > On 1/19/18 9:37 AM, Ming Lei wrote: > > On Fri, Jan 19, 2018 at 09:27:46AM -0700, Jens Axboe wrote: > >> On 1/19/18 9:26 AM, Ming Lei wrote: > >>> On Fri, Jan 19, 2018 at 09:19:24AM -0700, Jens Axboe wr

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 10:38:41AM -0700, Jens Axboe wrote: > On 1/19/18 9:37 AM, Ming Lei wrote: > > On Fri, Jan 19, 2018 at 09:27:46AM -0700, Jens Axboe wrote: > >> On 1/19/18 9:26 AM, Ming Lei wrote: > >>> On Fri, Jan 19, 2018 at 09:19:24AM -0700, Jens Axboe wr

Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle

2018-01-19 Thread Ming Lei
On Fri, Jan 19, 2018 at 09:23:35AM -0700, Jens Axboe wrote: > On 1/19/18 9:13 AM, Mike Snitzer wrote: > > On Fri, Jan 19 2018 at 10:48am -0500, > > Jens Axboe wrote: > > > >> On 1/19/18 8:40 AM, Ming Lei wrote: > >>>>>> Where does the dm

Re: [PATCH 2/2] blk-mq: simplify queue mapping & schedule with each possisble CPU

2018-01-26 Thread Ming Lei
Hi Jianchao, On Fri, Jan 19, 2018 at 11:05:35AM +0800, jianchao.wang wrote: > Hi ming > > Sorry for delayed report this. > > On 01/17/2018 05:57 PM, Ming Lei wrote: > > 2) hctx->next_cpu can become offline from online before > > __blk_mq_run_hw_queue > > i

Re: [PATCH 2/2] blk-mq: simplify queue mapping & schedule with each possisble CPU

2018-01-16 Thread Ming Lei
Hi Jianchao, On Wed, Jan 17, 2018 at 10:56:13AM +0800, jianchao.wang wrote: > Hi ming > > Thanks for your patch and kindly response. You are welcome! > > On 01/16/2018 11:32 PM, Ming Lei wrote: > > OK, I got it, and it should have been the only corner case in which > &

Re: [PATCH 2/2] blk-mq: simplify queue mapping & schedule with each possisble CPU

2018-01-16 Thread Ming Lei
Hi Jianchao, On Wed, Jan 17, 2018 at 01:24:23PM +0800, jianchao.wang wrote: > Hi ming > > Thanks for your kindly response. > > On 01/17/2018 11:52 AM, Ming Lei wrote: > >> It is here. > >> __blk_mq_run_hw_queue() > >> > >> WARN_ON(!cpu

Re: [PATCH 2/2] blk-mq: simplify queue mapping & schedule with each possisble CPU

2018-01-17 Thread Ming Lei
Hi Jianchao, On Wed, Jan 17, 2018 at 04:09:11PM +0800, jianchao.wang wrote: > Hi ming > > Thanks for your kindly response. > > On 01/17/2018 02:22 PM, Ming Lei wrote: > > This warning can't be removed completely, for example, the CPU figured > > in blk_mq_h

Re: [PATCH 2/2] blk-mq: simplify queue mapping & schedule with each possisble CPU

2018-01-17 Thread Ming Lei
On Wed, Jan 17, 2018 at 11:07:48AM +0100, Christian Borntraeger wrote: > > > On 01/17/2018 10:57 AM, Ming Lei wrote: > > Hi Jianchao, > > > > On Wed, Jan 17, 2018 at 04:09:11PM +0800, jianchao.wang wrote: > >> Hi ming > >> > >> Thanks for

<    1   2   3   4   5   6   7   8   9   10   >