Re: [PATCH v2 1/1] blk-mq: Inline request status checkers

2019-09-30 Thread Mike Snitzer
On Mon, Sep 30 2019 at 3:53pm -0400, Bart Van Assche wrote: > On 9/30/19 12:43 PM, Pavel Begunkov (Silence) wrote: > > @@ -282,7 +282,7 @@ static bool bt_tags_iter(struct sbitmap *bitmap, > > unsigned int bitnr, void *data) > > * test and set the bit before assining ->rqs[]. > > */ >

[git pull] device mapper fixes for 5.3-rc6

2019-08-23 Thread Mike Snitzer
Hi Linus, More fixes than usual from DM at this stage in a release but... The following changes since commit d45331b00ddb179e291766617259261c112db872: Linux 5.3-rc4 (2019-08-11 13:26:41 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/device-mapp

[git pull] device mapper fixes for 5.3-rc3

2019-08-02 Thread Mike Snitzer
ecent DAX code changes from commit in 5.3 merge. Mike Snitzer (1): dm table: fix various whitespace issues with recent DAX code Pankaj Gupta (1): dm table: fix dax_dev NULL dereference in device_synchronous() drivers/

Re: [PATCH V3 1/2] blk-mq: add callback of .cleanup_rq

2019-07-24 Thread Mike Snitzer
On Tue, Jul 23 2019 at 11:12pm -0400, Ming Lei wrote: > dm-rq needs to free request which has been dispatched and not completed > by underlying queue. However, the underlying queue may have allocated > private data for this request in .queue_rq(), so the request private data > will be leaked in d

Re: [PATCH 1/2] blk-mq: add callback of .cleanup_rq

2019-07-19 Thread Mike Snitzer
On Thu, Jul 18 2019 at 9:35pm -0400, Ming Lei wrote: > On Thu, Jul 18, 2019 at 10:52:01AM -0400, Mike Snitzer wrote: > > On Wed, Jul 17 2019 at 11:25pm -0400, > > Ming Lei wrote: > > > > > dm-rq needs to free request which has been dispatched and not compl

[git pull] device mapper changes for 5.1

2019-03-07 Thread Mike Snitzer
fix typo in name for writeback_wq Jason Cai (Xiang Feng) (1): dm thin: add sanity checks to thin-pool and external snapshot creation Mike Snitzer (5): dm: update dm_process_bio() to split bio if in ->make_request_fn() dm: eliminate 'split_discard_bios' flag from DM t

Re: block: be more careful about status in __bio_chain_endio

2019-02-22 Thread Mike Snitzer
On Fri, Feb 22 2019 at 9:02pm -0500, John Dorminy wrote: > I am perhaps not understanding the intricacies here, or not seeing a > barrier protecting it, so forgive me if I'm off base. I think reading > parent->bi_status here is unsafe. > Consider the following sequence of events on two threads.

Re: block: be more careful about status in __bio_chain_endio

2019-02-22 Thread Mike Snitzer
On Fri, Feb 22 2019 at 5:46pm -0500, Jens Axboe wrote: > On 2/22/19 2:10 PM, Mike Snitzer wrote: > > On Thu, Feb 15 2018 at 4:09am -0500, > > NeilBrown wrote: > > > >> > >> If two bios are chained under the one parent (with bio_chain()) > >> it

Re: block: be more careful about status in __bio_chain_endio

2019-02-22 Thread Mike Snitzer
f (!parent->bi_status) > + if (bio->bi_status) > parent->bi_status = bio->bi_status; > bio_put(bio); > return parent; > -- > 2.14.0.rc0.dirty > Reviewed-by: Mike Snitzer Jens, this one slipped through the crack just over a year ago. It is available in patchwork here: https://patchwork.kernel.org/patch/10220727/

Re: DM: kernel BUG at block/bio.c:1872!

2019-02-20 Thread Mike Snitzer
On Wed, Feb 20 2019 at 10:58pm -0500, Ming Lei wrote: > On Thu, Feb 21, 2019 at 11:24 AM Mike Snitzer wrote: > > > > On Wed, Feb 20 2019 at 8:15pm -0500, > > Ming Lei wrote: > > > > > Hi, > > > > > > Just saw the following kernel panic on

Re: DM: kernel BUG at block/bio.c:1872!

2019-02-20 Thread Mike Snitzer
On Wed, Feb 20 2019 at 8:15pm -0500, Ming Lei wrote: > Hi, > > Just saw the following kernel panic on today's -next by running > generic/081 of xfstests. > > [ 307.026552] run fstests generic/081 at 2019-02-20 17:01:42 > [ 307.647611] [ cut here ] > [ 307.648318] ker

[git pull] device mapper fixes for 5.0 final

2019-02-15 Thread Mike Snitzer
Hi Linus, The following changes since commit d13937116f1e82bf508a6325111b322c30c85eb9: Linux 5.0-rc6 (2019-02-10 14:42:20 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git tags/for-5.0/dm-fixes-3 for you to fetch change

[git pull] device mapper fixes for 5.0-rc6

2019-02-07 Thread Mike Snitzer
;s missing memory barrier before waitqueue_active() calls. - Fix DM core's clone_bio() to work when cloning a subset of a bio with an integrity payload; bio_integrity_trim() wasn't getting called due to bio_trim()'s early return.

Re: block: Fix a WRITE SAME BUG_ON

2019-01-28 Thread Mike Snitzer
On Sat, Jan 26 2019 at 6:17am -0500, John Dorminy wrote: > Hi. I have read a bit of DM code and spent an hour reviewing this... I > didn't get to the point of knowing what the right fix for the problem > is, and I may have a wrong understanding, but I have two thoughts > about the patch: > > I

[git pull] device mapper fixes for 5.0-rc4

2019-01-24 Thread Mike Snitzer
(e.g. dm-stripe, dm-thinp, etc). Joe Thornber (1): dm thin: fix passdown_double_checking_shared_status() Mike Snitzer (4): dm: fix clone_bio() to trigger blk_recount_segments() dm: fix redundant IO accounting for

Re: block: cover another queue enter recursion via BIO_QUEUE_ENTERED

2019-01-22 Thread Mike Snitzer
est_fn is returned. Especially > the same bio can't be submitted from another context. > > Fixes: cd4a4ae4683dc2 ("block: don't use blocking queue entered for recursive > bio submits") > Cc: Tetsuo Handa > Cc: Mike Snitzer > Cc: NeilBrown > Signed-of

Re: [PATCH 3/4] dm: fix missing bio_split() pattern code in __split_and_process_bio()

2019-01-21 Thread Mike Snitzer
On Mon, Jan 21 2019 at 10:17pm -0500, Mike Snitzer wrote: > On Mon, Jan 21 2019 at 9:46pm -0500, > Ming Lei wrote: > > > On Mon, Jan 21, 2019 at 11:02:04AM -0500, Mike Snitzer wrote: > > > On Sun, Jan 20 2019 at 10:21P -0500, > > > Ming Lei wrote: > > &

Re: [PATCH 3/4] dm: fix missing bio_split() pattern code in __split_and_process_bio()

2019-01-21 Thread Mike Snitzer
On Mon, Jan 21 2019 at 9:46pm -0500, Ming Lei wrote: > On Mon, Jan 21, 2019 at 11:02:04AM -0500, Mike Snitzer wrote: > > On Sun, Jan 20 2019 at 10:21P -0500, > > Ming Lei wrote: > > > > > On Sat, Jan 19, 2019 at 01:05:05PM -0500, Mike Snitzer wrote: > >

Re: [PATCH 3/4] dm: fix missing bio_split() pattern code in __split_and_process_bio()

2019-01-21 Thread Mike Snitzer
On Sun, Jan 20 2019 at 10:21P -0500, Ming Lei wrote: > On Sat, Jan 19, 2019 at 01:05:05PM -0500, Mike Snitzer wrote: > > Use the same BIO_QUEUE_ENTERED pattern that was established by commit > > cd4a4ae4683dc ("block: don't use blocking queue entered for recursive >

[PATCH 4/4] dm: fix dm_wq_work() to only use __split_and_process_bio() if appropriate

2019-01-19 Thread Mike Snitzer
Otherwise targets that don't support/expect IO splitting could resubmit bios using code paths with unnecessary IO splitting complexity. Depends-on: 24113d487843 ("dm: avoid indirect call in __dm_make_request") Fixes: 978e51ba38e00 ("dm: optimize bio-based NVMe IO submission&q

[PATCH 3/4] dm: fix missing bio_split() pattern code in __split_and_process_bio()

2019-01-19 Thread Mike Snitzer
it() because it provides useful context about bio splits in blktrace. Depends-on: cd4a4ae4683dc ("block: don't use blocking queue entered for recursive bio submits") Fixes: 18a25da84354 ("dm: ensure bio submission follows a depth-first tree walk") Cc: sta...@vger.kernel

[PATCH 2/4] dm: fix redundant IO accounting for bios that need splitting

2019-01-19 Thread Mike Snitzer
/stat | awk '{ print $7 }' 32768 Fixes: 18a25da84354 ("dm: ensure bio submission follows a depth-first tree walk") Cc: sta...@vger.kernel.org # 4.16+ Reported-by: Bryan Gurney Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 16 1 file changed, 16 insert

[PATCH 0/4] dm: fix various issues with bio splitting code

2019-01-19 Thread Mike Snitzer
recursive pattern. We should probably factor out some helpers that all users could share (e.g. so they don't forget to set BIO_QUEUE_ENTERED or call trace_block_split(), etc). Thanks, Mike Mike Snitzer (4): dm: fix clone_bio() to trigger blk_recount_segments() dm: fix redundant IO acco

[PATCH 1/4] dm: fix clone_bio() to trigger blk_recount_segments()

2019-01-19 Thread Mike Snitzer
DM's clone_bio() now benefits from using bio_trim() by fixing the fact that clone_bio() wasn't clearing BIO_SEG_VALID like bio_trim() does; which triggers blk_recount_segments() via bio_phys_segments(). Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 8 ++-- 1 file changed, 2

[git pull] device mapper changes for 4.21

2018-12-27 Thread Mike Snitzer
dm raid: fix false -EBUSY when handling check/repair message Jaegeuk Kim (1): dm: do not allow readahead to limit IO size Mike Snitzer (3): dm rq: remove unused arguments from rq_completed() dm: remove indirect calls from __send_changing_extent_only() dm rq: cleanup le

Re: dm: don't reuse bio for flushes

2018-12-19 Thread Mike Snitzer
On Wed, Dec 19 2018 at 11:11am -0500, Mike Snitzer wrote: > On Wed, Dec 19 2018 at 10:50am -0500, > Jens Axboe wrote: > > > DM currently has a statically allocated bio that it uses to issue empty > > flushes. It doesn't submit this bio, it just uses it for maintaini

Re: dm: don't reuse bio for flushes

2018-12-19 Thread Mike Snitzer
> embedded bio. > > Fixes: 5cdf2e3fea5e ("blkcg: associate blkg when associating a device") > Reported-by: Ming Lei > Signed-off-by: Jens Axboe Thanks for sorting this one out, definitely wasn't happy with how exposed DM was left with the recent blkg changes. This is clearly better. Acked-by: Mike Snitzer

Re: Upcoming merge window

2018-12-17 Thread Mike Snitzer
252.352669] call_rwsem_down_write_failed+0x13/0x20 > >> [ 252.358601] down_write+0x1b/0x30 > >> [ 252.362781] __generic_file_fsync+0x3e/0xb0 > >> [ 252.367933] ext4_sync_file+0xcc/0x2e0 > >> [ 252.372599] dio_complete+0x1c4/0x210 > >> [ 252.377168]

Re: [PATCH 3/4] blk-mq: deal with shared queue mapping reliably

2018-12-16 Thread Mike Snitzer
atchset (1-3 anyway) I get IO hangs in blkdev_fsync(). With that in mind, Jens needs these fixes (or something comparable) ASAP. Tested-by: Mike Snitzer

[git pull] device mapper fixes for 4.20

2018-12-12 Thread Mike Snitzer
target BIO completion handling Mike Snitzer (4): dm cache metadata: verify cache has blocks in blocks_are_clean_separate_dirty() dm: call blk_queue_split() to impose device limits on bios dm thin: send event about thin-pool state change _after_ making it dm thin: bump targ

Re: xfstests generic/347 was never correct [was: Re: dm: fix inflight IO check]

2018-12-12 Thread Mike Snitzer
On Tue, Dec 11 2018 at 9:58pm -0500, Eric Sandeen wrote: > On 12/11/18 8:34 PM, Mike Snitzer wrote: > > On Mon, Dec 10 2018 at 7:32pm -0500, > > Mike Snitzer wrote: > > > > SO generic/347 needs to account for 2 things: > > 1) issue $XFS_IO_PROG IO in the

xfstests generic/347 was never correct [was: Re: dm: fix inflight IO check]

2018-12-11 Thread Mike Snitzer
On Mon, Dec 10 2018 at 7:32pm -0500, Mike Snitzer wrote: > On Mon, Dec 10 2018 at 5:45pm -0500, > Jens Axboe wrote: > > > After switching to percpu inflight counters, the inflight check > > is totally buggy. It's perfectly valid for some counters to be > &g

Re: for-next hangs on test srp/012

2018-12-11 Thread Mike Snitzer
On Tue, Dec 11 2018 at 7:19pm -0500, Ming Lei wrote: > On Wed, Dec 12, 2018 at 8:04 AM Jens Axboe wrote: > > > > On 12/11/18 3:58 PM, Bart Van Assche wrote: > > > Hi Jens, > > > > > > If I run the following subset of blktests: > > > > > > while :; do ./check -q srp && ./check -q nvmeof-mp; do

Re: for-next hangs on test srp/012

2018-12-11 Thread Mike Snitzer
On Tue, Dec 11 2018 at 6:09pm -0500, Bart Van Assche wrote: > On Tue, 2018-12-11 at 18:05 -0500, Mike Snitzer wrote: > > On Tue, Dec 11 2018 at 5:58pm -0500, > > Bart Van Assche wrote: > > > > > Hi Jens, > > > > > > If I run the followin

Re: for-next hangs on test srp/012

2018-12-11 Thread Mike Snitzer
On Tue, Dec 11 2018 at 5:58pm -0500, Bart Van Assche wrote: > Hi Jens, > > If I run the following subset of blktests: > > while :; do ./check -q srp && ./check -q nvmeof-mp; done > > against today's for-next branch (commit dd2bf2df85a7) then after some > time the following hang is reported:

[PATCH] dm: fix request-based dm's use of dm_wait_for_completion

2018-12-11 Thread Mike Snitzer
the blk-mq or bio-based method accordingly. Fixes: dbd3bbd291 ("dm rq: leverage blk_mq_queue_busy() to check for outstanding IO") Signed-off-by: Mike Snitzer --- drivers/md/dm-rq.c | 6 ++ drivers/md/dm.c| 10 +- 2 files changed, 11 insertions(+), 5 deletions(-) dif

Re: dm: fix inflight IO check

2018-12-10 Thread Mike Snitzer
ite's "resize_io" test doesn't pass. Glad this resolves the xfstest issue but I think more work is needed, so I'll build any additional changes on this fix. Thanks. Acked-by: Mike Snitzer > > --- > > diff --git a/drivers/md/dm.c b/drivers/md/dm.c > ind

Re: dm crypt: fix lost ioprio when queuing crypto bios from task with ioprio

2018-12-07 Thread Mike Snitzer
On Fri, Dec 07 2018 at 2:43pm -0500, Christoph Hellwig wrote: > On Thu, Dec 06, 2018 at 05:15:07PM -0500, Mike Snitzer wrote: > > From: Eric Wheeler > > > > Since dm-crypt queues writes (and sometimes reads) to a different kernel > > thread (workqueue), the bios wi

Re: [PATCH v3] blk-mq: punt failed direct issue to dispatch list

2018-12-07 Thread Mike Snitzer
and is based on a patch from Ming, > but with the list insert case covered as well. > > Fixes: ffe81d45322c ("blk-mq: fix corruption with direct issue") > Cc: sta...@vger.kernel.org > Suggested-by: Ming Lei > Reported-by: Bart Van Assche > Signed-off-by: Jens Axboe Looks good, thanks! Acked-by: Mike Snitzer

Re: [PATCH v2] block/dm: fix handling of busy off direct dispatch path

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 11:06pm -0500, Jens Axboe wrote: > On 12/6/18 8:54 PM, Mike Snitzer wrote: > > On Thu, Dec 06 2018 at 9:49pm -0500, > > Jens Axboe wrote: > > > >> After the direct dispatch corruption fix, we permanently disallow direct > >> di

Re: [PATCH v2] block/dm: fix handling of busy off direct dispatch path

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 9:49pm -0500, Jens Axboe wrote: > After the direct dispatch corruption fix, we permanently disallow direct > dispatch of non read/write requests. This works fine off the normal IO > path, as they will be retried like any other failed direct dispatch > request. But for the

Re: block: fix direct dispatch issue failure for clones

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 8:58pm -0500, Mike Snitzer wrote: > DM is forced to worry about all these details, as covered some in > the header for commit 396eaf21ee17c476e8f66249fb1f4a39003d0ab4, it is > trying to have its cake and eat it too. Gah, obviously meant: DM is _NOT_ trying to

Re: block: fix direct dispatch issue failure for clones

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 8:34pm -0500, Jens Axboe wrote: > On 12/6/18 6:22 PM, jianchao.wang wrote: > > > > > > On 12/7/18 9:13 AM, Jens Axboe wrote: > >> On 12/6/18 6:04 PM, jianchao.wang wrote: > >>> > >>> > >>> On 12/7/18 6:20 AM, Jens Axboe wrote: > After the direct dispatch corruption

Re: block: fix direct dispatch issue failure for clones

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 8:13pm -0500, Jens Axboe wrote: > On 12/6/18 6:04 PM, jianchao.wang wrote: > > > > > > On 12/7/18 6:20 AM, Jens Axboe wrote: > >> After the direct dispatch corruption fix, we permanently disallow direct > >> dispatch of non read/write requests. This works fine off the no

Re: block: fix direct dispatch issue failure for clones

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 8:04pm -0500, jianchao.wang wrote: > > > On 12/7/18 6:20 AM, Jens Axboe wrote: > > After the direct dispatch corruption fix, we permanently disallow direct > > dispatch of non read/write requests. This works fine off the normal IO > > path, as they will be retried like a

Re: block: fix direct dispatch issue failure for clones

2018-12-06 Thread Mike Snitzer
->queue_lock, flags); Not sure what this trailing spin_lock_irqsave(q->queue_lock, flags) is about.. but this looks good. I'll cleanup dm-rq.c to do away with the extra STS_RESOURCE checks for its call to blk_insert_cloned_request() once this lands. Acked-by: Mike Snitzer Thanks.

[PATCH] dm crypt: fix lost ioprio when queuing crypto bios from task with ioprio

2018-12-06 Thread Mike Snitzer
Mikulas Patocka Cc: Alasdair Kergon Cc: Jens Axboe Cc: sta...@vger.kernel.org Signed-off-by: Mike Snitzer --- block/bio.c | 24 drivers/md/dm-crypt.c | 15 +-- include/linux/bio.h | 2 ++ 3 files changed, 39 insertions(+), 2 deletions(-) [Jens

Re: [PATCH v4 0/7] per-cpu in_flight counters for bio-based drivers

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 1:00pm -0500, Mike Snitzer wrote: > On Thu, Dec 06 2018 at 11:41am -0500, > Mike Snitzer wrote: > > > Hey, > > > > This v4 addresses the compile issues on various archs when CONFIG_SMP > > isn't set (by introducing appropriate wrap

Re: [PATCH v4 0/7] per-cpu in_flight counters for bio-based drivers

2018-12-06 Thread Mike Snitzer
On Thu, Dec 06 2018 at 11:41am -0500, Mike Snitzer wrote: > Hey, > > This v4 addresses the compile issues on various archs when CONFIG_SMP > isn't set (by introducing appropriate wrappers in genhd.h) > > Testing with this v4 I was unable to reproduce the issue you

[PATCH v4 0/7] per-cpu in_flight counters for bio-based drivers

2018-12-06 Thread Mike Snitzer
l see problems like that. Thanks, Mike Mike Snitzer (2): dm rq: leverage blk_mq_queue_busy() to check for outstanding IO block: stop passing 'cpu' to all percpu stats methods Mikulas Patocka (5): dm: dont rewrite dm_disk(md)->part0.in_flight block: delete part_round_stats and swit

[PATCH v4 2/7] dm rq: leverage blk_mq_queue_busy() to check for outstanding IO

2018-12-06 Thread Mike Snitzer
Now that request-based dm-multipath only supports blk-mq, make use of the newly introduced blk_mq_queue_busy() to check for outstanding IO -- rather than (ab)using the block core's in_flight counters. Signed-off-by: Mike Snitzer --- drivers/md/dm-rq.c | 9 - 1 file changed, 4 inser

[PATCH v4 6/7] block: return just one value from part_in_flight

2018-12-06 Thread Mike Snitzer
the code, there's no functional change. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/blk-mq.c| 12 +--- block/blk-mq.h| 3 +-- block/genhd.c | 34 -- block/partition-generic.c

[PATCH v4 7/7] dm: remove the pending IO accounting

2018-12-06 Thread Mike Snitzer
From: Mikulas Patocka Remove the "pending" atomic counters, that duplicate block-core's in_flight counters, and update md_in_flight() to look at percpu in_flight counters. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 -- drivers/md

[PATCH v4 4/7] block: delete part_round_stats and switch to less precise counting

2018-12-06 Thread Mike Snitzer
-off-by: Mike Snitzer --- block/bio.c | 24 +++--- block/blk-core.c | 62 +++ block/blk-merge.c | 1 - block/genhd.c | 3 --- block/partition-generic.c | 3 --- include/linux/genhd.h | 3 +-- 6

[PATCH v4 5/7] block: switch to per-cpu in-flight counters

2018-12-06 Thread Mike Snitzer
corrupted due to reentrant interrupt, but the corruption only results in slight counter skew - the in_flight counter must be exact, so it needs local_t. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/genhd.c | 43 +-- include/linux

[PATCH v4 3/7] block: stop passing 'cpu' to all percpu stats methods

2018-12-06 Thread Mike Snitzer
All of part_stat_* and related methods are used with preempt disabled, so there is no need to pass cpu around to allow of them. Just call smp_processor_id() as needed. Suggested-by: Jens Axboe Signed-off-by: Mike Snitzer --- block/bio.c | 16 +--- block/blk-core.c

[PATCH v4 1/7] dm: dont rewrite dm_disk(md)->part0.in_flight

2018-12-06 Thread Mike Snitzer
From: Mikulas Patocka generic_start_io_acct and generic_end_io_acct already update the variable in_flight using atomic operations, so we don't have to overwrite them again. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 4 +--- 1 file changed, 1 inse

[PATCH v3 7/7] dm: remove the pending IO accounting

2018-12-05 Thread Mike Snitzer
From: Mikulas Patocka Remove the "pending" atomic counters, that duplicate block-core's in_flight counters, and update md_in_flight() to look at percpu in_flight counters. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 -- drivers/md

[PATCH v3 4/7] block: delete part_round_stats and switch to less precise counting

2018-12-05 Thread Mike Snitzer
-off-by: Mike Snitzer --- block/bio.c | 24 +-- block/blk-core.c | 62 +++ block/blk-merge.c | 1 - block/genhd.c | 3 -- block/partition-generic.c | 3 -- include/linux/genhd.h | 3 +- 6 files changed

[PATCH v3 0/7] per-cpu in_flight counters for bio-based drivers

2018-12-05 Thread Mike Snitzer
Hi, This v3 adds "block: stop passing 'cpu' to all percpu stats methods" before switching over to percpu counters. Thanks, Mike Mike Snitzer (2): dm rq: leverage blk_mq_queue_busy() to check for outstanding IO block: stop passing 'cpu' to all percpu stats meth

[PATCH v3 2/7] dm rq: leverage blk_mq_queue_busy() to check for outstanding IO

2018-12-05 Thread Mike Snitzer
Now that request-based dm-multipath only supports blk-mq, make use of the newly introduced blk_mq_queue_busy() to check for outstanding IO -- rather than (ab)using the block core's in_flight counters. Signed-off-by: Mike Snitzer --- drivers/md/dm-rq.c | 9 - 1 file changed, 4 inser

[PATCH v3 3/7] block: stop passing 'cpu' to all percpu stats methods

2018-12-05 Thread Mike Snitzer
All of part_stat_* and related methods are used with preempt disabled, so there is no need to pass cpu around to allow of them. Just call smp_processor_id() as needed. Suggested-by: Jens Axboe Signed-off-by: Mike Snitzer --- block/bio.c | 16 +--- block/blk-core.c

[PATCH v3 5/7] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
corrupted due to reentrant interrupt, but the corruption only results in slight counter skew - the in_flight counter must be exact, so it needs local_t. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/genhd.c | 49 ++- include/linux

[PATCH v3 1/7] dm: dont rewrite dm_disk(md)->part0.in_flight

2018-12-05 Thread Mike Snitzer
From: Mikulas Patocka generic_start_io_acct and generic_end_io_acct already update the variable in_flight using atomic operations, so we don't have to overwrite them again. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 4 +--- 1 file changed, 1 inse

[PATCH v3 6/7] block: return just one value from part_in_flight

2018-12-05 Thread Mike Snitzer
the code, there's no functional change. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/blk-mq.c| 12 +--- block/blk-mq.h| 3 +-- block/genhd.c | 32 +++- block/partition-generic.c | 6 +++--- in

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
On Wed, Dec 05 2018 at 1:04pm -0500, Jens Axboe wrote: > On 12/5/18 11:03 AM, Mike Snitzer wrote: > > On Wed, Dec 05 2018 at 12:54pm -0500, > > Jens Axboe wrote: > > > >> On 12/5/18 10:49 AM, Mike Snitzer wrote: > >>> On Wed, Dec 05 2018

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
On Wed, Dec 05 2018 at 12:54pm -0500, Jens Axboe wrote: > On 12/5/18 10:49 AM, Mike Snitzer wrote: > > On Wed, Dec 05 2018 at 12:30pm -0500, > > Jens Axboe wrote: > > > >> There's also no need to pass in the cpu, if we're not running with > &g

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
On Wed, Dec 05 2018 at 12:30pm -0500, Jens Axboe wrote: > On 11/30/18 3:22 PM, Mike Snitzer wrote: > > diff --git a/block/genhd.c b/block/genhd.c > > index cdf174d7d329..d4c9dd65def6 100644 > > --- a/block/genhd.c > > +++ b/block/genhd.c > > @@ -45,53 +45,76 @@ s

Re: [PATCH 05/14] dm: set the static flush bio device on demand

2018-12-04 Thread Mike Snitzer
er bios. > > Signed-off-by: Dennis Zhou > Cc: Alasdair Kergon > Cc: Mike Snitzer > --- > block/bio.c | 1 + > drivers/md/dm.c | 12 +++- > 2 files changed, 12 insertions(+), 1 deletion(-) > > diff --git a/block/bio.c b/block/bio.c > index 452b8e79b9

[PATCH v2 0/6] per-cpu in_flight counters for bio-based drivers

2018-11-30 Thread Mike Snitzer
that it was doing, see: https://www.redhat.com/archives/dm-devel/2018-November/msg00415.html Happy to iterate on this patchset further as needed, all review/suggestions are very much appreciated. Thanks, Mike Mike Snitzer (1): dm rq: leverage blk_mq_queue_busy() to check for outstanding IO

[PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-11-30 Thread Mike Snitzer
corrupted due to reentrant interrupt, but the corruption only results in slight counter skew - the in_flight counter must be exact, so it needs local_t. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/bio.c | 4 ++-- block/blk-core.c | 4 ++-- block/blk-merge.c

[PATCH v2 6/6] dm: remove the pending IO accounting

2018-11-30 Thread Mike Snitzer
From: Mikulas Patocka Remove the "pending" atomic counters, that duplicate block-core's in_flight counters, and update md_in_flight() to look at percpu in_flight counters. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 -- drivers/md

[PATCH v2 1/6] dm: dont rewrite dm_disk(md)->part0.in_flight

2018-11-30 Thread Mike Snitzer
From: Mikulas Patocka generic_start_io_acct and generic_end_io_acct already update the variable in_flight using atomic operations, so we don't have to overwrite them again. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 4 +--- 1 file changed, 1 inse

[PATCH v2 5/6] block: return just one value from part_in_flight

2018-11-30 Thread Mike Snitzer
the code, there's no functional change. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/blk-mq.c| 12 +--- block/blk-mq.h| 3 +-- block/genhd.c | 32 +++- block/partition-generic.c | 6 +++--- in

[PATCH v2 3/6] block: delete part_round_stats and switch to less precise counting

2018-11-30 Thread Mike Snitzer
-off-by: Mike Snitzer --- block/bio.c | 24 +++--- block/blk-core.c | 63 +++ block/blk-merge.c | 1 - block/genhd.c | 4 --- block/partition-generic.c | 4 --- include/linux/genhd.h | 3 +-- 6

[PATCH v2 2/6] dm rq: leverage blk_mq_queue_busy() to check for outstanding IO

2018-11-30 Thread Mike Snitzer
Now that request-based dm-multipath only supports blk-mq, make use of the newly introduced blk_mq_queue_busy() to check for outstanding IO -- rather than (ab)using the block core's in_flight counters. Signed-off-by: Mike Snitzer --- drivers/md/dm-rq.c | 9 - 1 file changed, 4 inser

Re: [PATCH 2/3] block: switch to per-cpu in-flight counters

2018-11-30 Thread Mike Snitzer
On Fri, Nov 30 2018 at 4:52pm -0500, Jens Axboe wrote: > On 11/30/18 2:44 PM, Mike Snitzer wrote: > > On Tue, Nov 27 2018 at 7:42pm -0500, > > Mikulas Patocka wrote: > > > >> Now when part_round_stats is gone, we can switch to per-cpu in-flight > >&g

Re: [PATCH 1/3] block: delete part_round_stats and switch to less precise counting

2018-11-30 Thread Mike Snitzer
On Fri, Nov 30 2018 at 2:42pm -0500, Mike Snitzer wrote: > On Tue, Nov 27 2018 at 7:42pm -0500, > Mikulas Patocka wrote: > > > We want to convert to per-cpu in_flight counters. > > > > The function part_round_stats needs the in_flight counter every jiffy, it >

Re: [PATCH 2/3] block: switch to per-cpu in-flight counters

2018-11-30 Thread Mike Snitzer
On Tue, Nov 27 2018 at 7:42pm -0500, Mikulas Patocka wrote: > Now when part_round_stats is gone, we can switch to per-cpu in-flight > counters. > > We use the local-atomic type local_t, so that if part_inc_in_flight or > part_dec_in_flight is reentrantly called from an interrupt, the value will

Re: [PATCH 0/3] per-cpu in_flight counters for bio-based drivers

2018-11-30 Thread Mike Snitzer
On Fri, Nov 30 2018 at 10:50am -0500, Mike Snitzer wrote: > On Fri, Nov 30 2018 at 9:43am -0500, > Mike Snitzer wrote: > > > On Tue, Nov 27 2018 at 7:42pm -0500, > > Mikulas Patocka wrote: > > > > > These are the patches for per-cpu in_flight counters.

Re: [PATCH 1/3] block: delete part_round_stats and switch to less precise counting

2018-11-30 Thread Mike Snitzer
On Tue, Nov 27 2018 at 7:42pm -0500, Mikulas Patocka wrote: > We want to convert to per-cpu in_flight counters. > > The function part_round_stats needs the in_flight counter every jiffy, it > would be too costly to sum all the percpu variables every jiffy, so it > must be deleted. part_round_st

Re: [PATCH 0/3] per-cpu in_flight counters for bio-based drivers

2018-11-30 Thread Mike Snitzer
On Fri, Nov 30 2018 at 9:43am -0500, Mike Snitzer wrote: > On Tue, Nov 27 2018 at 7:42pm -0500, > Mikulas Patocka wrote: > > > These are the patches for per-cpu in_flight counters. > > Do you have updated before vs after performance results for these > changes? &

Re: [PATCH 0/3] per-cpu in_flight counters for bio-based drivers

2018-11-30 Thread Mike Snitzer
On Tue, Nov 27 2018 at 7:42pm -0500, Mikulas Patocka wrote: > These are the patches for per-cpu in_flight counters. Do you have updated before vs after performance results for these changes? I'd imagine they are comparable to your previous run (though that run included some other DM changes th

Re: [PATCH 3/3] block: return just one value from part_in_flight

2018-11-29 Thread Mike Snitzer
On Tue, Nov 27 2018 at 7:42pm -0500, Mikulas Patocka wrote: > The previous patches deleted all the code that needed the second value > returned from part_in_flight - now the kernel only uses the first value. > > Consequently, part_in_flight (and blk_mq_in_flight) may be changed so that > it onl

Re: [PATCH 2/3] block: switch to per-cpu in-flight counters

2018-11-29 Thread Mike Snitzer
On Thu, Nov 29 2018 at 5:05pm -0500, Mikulas Patocka wrote: > > > On Thu, 29 Nov 2018, Mike Snitzer wrote: > > > On Tue, Nov 27 2018 at 7:42pm -0500, > > Mikulas Patocka wrote: > > > > > Now when part_round_stats is gone, we can switch to per-cpu in

Re: [PATCH 2/3] block: switch to per-cpu in-flight counters

2018-11-29 Thread Mike Snitzer
On Tue, Nov 27 2018 at 7:42pm -0500, Mikulas Patocka wrote: > Now when part_round_stats is gone, we can switch to per-cpu in-flight > counters. > > We use the local-atomic type local_t, so that if part_inc_in_flight or > part_dec_in_flight is reentrantly called from an interrupt, the value will

Re: nvme: allow ANA support to be independent of native multipathing

2018-11-20 Thread Mike Snitzer
On Tue, Nov 20 2018 at 4:42am -0500, Christoph Hellwig wrote: > On Mon, Nov 19, 2018 at 09:56:50AM -0500, Mike Snitzer wrote: > > SO: will you be taking my v2 patch for 4.21 or not? > > No. This isn't how a Linux maintainer engages in technical discussion. You _clearly_

Re: nvme: allow ANA support to be independent of native multipathing

2018-11-19 Thread Mike Snitzer
On Mon, Nov 19 2018 at 4:39am -0500, Christoph Hellwig wrote: > On Fri, Nov 16, 2018 at 02:28:02PM -0500, Mike Snitzer wrote: > > You rejected the idea of allowing fine-grained control over whether > > native NVMe multipathing is enabled or not on a per-namespace basis. > &g

Re: nvme: allow ANA support to be independent of native multipathing

2018-11-16 Thread Mike Snitzer
On Fri, Nov 16 2018 at 5:17am -0500, Christoph Hellwig wrote: > On Fri, Nov 16, 2018 at 11:06:32AM +0100, Hannes Reinecke wrote: > > Ok, so would you be happy with making ANA support configurable? > > I've looked a bit over the whole situation, and what I think we need > to do is: > > a) warn

[PATCH v2] nvme: allow ANA support to be independent of native multipathing

2018-11-16 Thread Mike Snitzer
module param to allow ANA to be disabled via nvme_core.ana=N. Also, emit warning if ANA is enabled but native multipathing isn't. And as always, if embedded NVMe users do not want any performance overhead associated with ANA or native NVMe multipathing they can disable CONFIG_NVME_MULTIPATH.

Re: nvme: allow ANA support to be independent of native multipathing

2018-11-16 Thread Mike Snitzer
On Fri, Nov 16 2018 at 4:14am -0500, Christoph Hellwig wrote: > On Thu, Nov 15, 2018 at 12:46:05PM -0500, Mike Snitzer wrote: > > Whether or not ANA is present is a choice of the target implementation; > > the host (and whether it supports multipathing) has _zero_ influence on &g

Re: nvme: allow ANA support to be independent of native multipathing

2018-11-16 Thread Mike Snitzer
On Fri, Nov 16 2018 at 2:25am -0500, Hannes Reinecke wrote: > On 11/15/18 6:46 PM, Mike Snitzer wrote: > >Whether or not ANA is present is a choice of the target implementation; > >the host (and whether it supports multipathing) has _zero_ influence on > >this. If the targ

Re: [PATCH 3/3] block: use a driver-specific handler for the "inflight" value

2018-11-16 Thread Mike Snitzer
On Fri, Nov 16 2018 at 4:11am -0500, Christoph Hellwig wrote: > On Fri, Nov 16, 2018 at 01:04:19AM +0100, Mikulas Patocka wrote: > > Device mapper was converted to percpu inflight counters. In order to > > display the correct values in the "inflight" sysfs file and in > > /proc/diskstats, we nee

Re: [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Mike Snitzer
splitting multi-page bvec into segments, the max segment > > limit may be reached, so the bio split need to be considered under > > this situation too. > > > > Cc: Dave Chinner > > Cc: Kent Overstreet > > Cc: Mike Snitzer > > Cc: dm-de...@redhat.co

[PATCH] nvme: allow ANA support to be independent of native multipathing

2018-11-15 Thread Mike Snitzer
associated with ANA or native NVMe multipathing they can disable CONFIG_NVME_MULTIPATH. Signed-off-by: Mike Snitzer --- drivers/nvme/host/core.c | 10 + drivers/nvme/host/multipath.c | 49 +-- drivers/nvme/host/nvme.h | 4 3 fi

Re: multipath-tools: add ANA support for NVMe device

2018-11-14 Thread Mike Snitzer
On Wed, Nov 14 2018 at 1:51pm -0500, Hannes Reinecke wrote: > On 11/14/18 6:47 PM, Mike Snitzer wrote: > > On Wed, Nov 14 2018 at 2:49am -0500, > > Hannes Reinecke wrote: > > > >> On 11/14/18 6:38 AM, Mike Snitzer wrote: > >>> On Tue, Nov 13 2018 a

Re: multipath-tools: add ANA support for NVMe device

2018-11-14 Thread Mike Snitzer
On Wed, Nov 14 2018 at 2:49am -0500, Hannes Reinecke wrote: > On 11/14/18 6:38 AM, Mike Snitzer wrote: > >On Tue, Nov 13 2018 at 1:00pm -0500, > >Mike Snitzer wrote: > > > >>[1]: > >>http://lists.infradead.org/pipermail/linux-nvme/2018-November/0207

Re: [patch 5/5] block: use a driver-specific handler for the "inflight" value

2018-11-14 Thread Mike Snitzer
On Wed, Nov 14 2018 at 10:18am -0500, Christoph Hellwig wrote: > On Thu, Nov 08, 2018 at 12:07:01PM -0500, Mike Snitzer wrote: > > Discussed doing that with Jens and reported as much here: > > > > https://www.redhat.com/archives/dm-devel/2018-November/msg00068.html &

Re: multipath-tools: add ANA support for NVMe device

2018-11-13 Thread Mike Snitzer
On Tue, Nov 13 2018 at 1:00pm -0500, Mike Snitzer wrote: > On Tue, Nov 13 2018 at 11:18am -0500, > Keith Busch wrote: > > > On Mon, Nov 12, 2018 at 04:53:23PM -0500, Mike Snitzer wrote: > > > On Mon, Nov 12 2018 at 11:23am -0500, > > > Martin Wilck

Re: multipath-tools: add ANA support for NVMe device

2018-11-13 Thread Mike Snitzer
On Tue, Nov 13 2018 at 11:18am -0500, Keith Busch wrote: > On Mon, Nov 12, 2018 at 04:53:23PM -0500, Mike Snitzer wrote: > > On Mon, Nov 12 2018 at 11:23am -0500, > > Martin Wilck wrote: > > > > > Hello Lijie, > > > > > > On Thu, 2018-11-08 at 1

  1   2   3   4   5   >