Re: [dm-devel] [PATCH v2] dm: Check for device sector overflow if CONFIG_LBDAF is not set

2018-11-15 Thread Mikulas Patocka
On Wed, 7 Nov 2018, Milan Broz wrote: > Reference to a device in device-mapper table contains offset in sectors. > > If the sector_t is 32bit integer (CONFIG_LBDAF is not set), then > several device-mapper targets can overflow this offset and validity > check is then performed on a wrong

[dm-devel] [PATCH] nvme: allow ANA support to be independent of native multipathing

2018-11-15 Thread Mike Snitzer
Whether or not ANA is present is a choice of the target implementation; the host (and whether it supports multipathing) has _zero_ influence on this. If the target declares a path as 'inaccessible' the path _is_ inaccessible to the host. As such, ANA support should be functional even if native

Re: [dm-devel] [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Mike Snitzer
On Thu, Nov 15 2018 at 3:20pm -0500, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > > First it is more efficient to use bio_for_each_bvec() in both > > blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how > > many multi-page bvecs there are

[dm-devel] [PATCH 0/3] device mapper percpu counter patches

2018-11-15 Thread Mikulas Patocka
Hi These are the device mapper percpu counter patches. They are on the top of https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/log/?h=dm-4.21 Mikulas -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

[dm-devel] [PATCH 1/3] dm: move dm_stats_account_io before generic_end_io_acct

2018-11-15 Thread Mikulas Patocka
Make sure that the statistics are not updated while the device is suspended. So, we move statistics update before generic_end_io_acct. Signed-off-by: Mikulas Patocka --- drivers/md/dm.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) Index: linux-dm/drivers/md/dm.c

[dm-devel] [PATCH 3/3] block: use a driver-specific handler for the "inflight" value

2018-11-15 Thread Mikulas Patocka
Device mapper was converted to percpu inflight counters. In order to display the correct values in the "inflight" sysfs file and in /proc/diskstats, we need a custom callback that sums the percpu counters. The function part_round_stats calculates the number of in-flight I/Os every jiffy and uses

Re: [dm-devel] [patch 0/5] device mapper percpu patches

2018-11-15 Thread Mikulas Patocka
On Wed, 7 Nov 2018, Jens Axboe wrote: > On 11/7/18 3:47 PM, Mikulas Patocka wrote: > > > > I'd like to know - which kernel part needs to sum the percpu IO counters > > frequently? > > > > My impression was that the counters need to be summed only when the user > > is reading the files in

Re: [dm-devel] [RFC] dm-bow working prototype

2018-11-15 Thread Mikulas Patocka
On Mon, 29 Oct 2018, Paul Lawrence wrote: > > > The snapshot target could be hacked so that it remembers space trimmed > > with REQ_OP_DISCARD and won't reallocate these blocks. > > > > But I suspect that running discard over the whole device would degrade > > performance more than copying

[dm-devel] [PATCH 2/3] dm: use percpu counters

2018-11-15 Thread Mikulas Patocka
Use percpu inflight counters to avoid cache line bouncing and improve performance. Signed-off-by: Mikulas Patocka --- drivers/md/dm-core.h |5 + drivers/md/dm.c | 50 ++ 2 files changed, 39 insertions(+), 16 deletions(-) Index:

Re: [dm-devel] [PATCH] nvme: allow ANA support to be independent of native multipathing

2018-11-15 Thread Hannes Reinecke
On 11/15/18 6:46 PM, Mike Snitzer wrote: Whether or not ANA is present is a choice of the target implementation; the host (and whether it supports multipathing) has _zero_ influence on this. If the target declares a path as 'inaccessible' the path _is_ inaccessible to the host. As such, ANA

[dm-devel] [PATCH V10 00/19] block: support multi-page bvec

2018-11-15 Thread Ming Lei
Hi, This patchset brings multi-page bvec into block layer: 1) what is multi-page bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multi-page bvec introduced?

[dm-devel] [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Ming Lei
First it is more efficient to use bio_for_each_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_bvec() is used, the bvec may need to be splitted because its length can be very longer than

[dm-devel] [PATCH V10 04/19] block: use bio_for_each_bvec() to map sg

2018-11-15 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua

[dm-devel] [PATCH V10 06/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-15 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use bvec_last_segment() to truncate the bio. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc:

[dm-devel] [PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-15 Thread Ming Lei
Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to increase BIO_MAX_PAGES for it. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc:

[dm-devel] [PATCH V10 07/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-15 Thread Ming Lei
Preparing for supporting multi-page bvec. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org Cc: David Sterba Cc:

[dm-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-15 Thread Ming Lei
There are still cases in which we need to use bio_bvecs() for get the number of multi-page segment, so introduce it. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc:

[dm-devel] [PATCH V10 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-15 Thread Ming Lei
iov_iter is implemented with bvec itererator, so it is safe to pass multipage bvec to it, and this way is much more efficient than passing one page in each bvec. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org

[dm-devel] [PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-15 Thread Ming Lei
Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), physical segment number is mainly figured out in blk_queue_split() for fast path, and the flag of BIO_SEG_VALID is set there too. Now only blk_recount_segments() and blk_recalc_rq_segments() use this flag. Basically

[dm-devel] [PATCH V10 13/19] iomap & xfs: only account for new added page

2018-11-15 Thread Ming Lei
After multi-page is enabled, one new page may be merged to a segment even though it is a new added page. This patch deals with this issue by post-check in case of merge, and only a freshly new added page need to be dealt with for iomap & xfs. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike

[dm-devel] [PATCH V10 14/19] block: enable multipage bvecs

2018-11-15 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Now any request queue which supports queue cluster will see multi-page bvecs. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc:

[dm-devel] [PATCH V10 17/19] block: don't use bio->bi_vcnt to figure out segment number

2018-11-15 Thread Ming Lei
It is wrong to use bio->bi_vcnt to figure out how many segments there are in the bio even though CLONED flag isn't set on this bio, because this bio may be splitted or advanced. So always use bio_segments() in blk_recount_segments(), and it shouldn't cause any performance loss now because the

[dm-devel] [PATCH V10 19/19] block: kill BLK_MQ_F_SG_MERGE

2018-11-15 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org Cc: David

[dm-devel] [PATCH V10 16/19] block: document usage of bio iterator helpers

2018-11-15 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua

[dm-devel] [PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-15 Thread Ming Lei
bch_bio_alloc_pages() is always called on one new bio, so it is safe to access the bvec table directly. Given it is the only kind of this case, open code the bvec table access since bio_for_each_segment_all() will be changed to support for iterating over multipage bvec. Cc: Dave Chinner Cc: Kent

[dm-devel] [PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-15 Thread Ming Lei
BTRFS is the only user of this helper, so move this helper into BTRFS, and implement it via bio_for_each_segment_all(), since bio->bi_vcnt may not equal to number of pages after multipage bvec is enabled. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc:

[dm-devel] [PATCH V10 01/19] block: introduce multi-page page bvec helpers

2018-11-15 Thread Ming Lei
This patch introduces helpers of 'mp_bvec_iter_*' for multipage bvec support. The introduced helpers treate one bvec as real multi-page segment, which may include more than one pages. The existed helpers of bvec_iter_* are interfaces for supporting current bvec iterator which is thought as

[dm-devel] [PATCH V10 02/19] block: introduce bio_for_each_bvec()

2018-11-15 Thread Ming Lei
This helper is used for iterating over multi-page bvec for bio split & merge code. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org

[dm-devel] [PATCH V10 05/19] block: introduce bvec_last_segment()

2018-11-15 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: