[Cluster-devel] [PATCH V14 15/18] block: always define BIO_MAX_PAGES as 256

2019-01-21 Thread Ming Lei
Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to increase BIO_MAX_PAGES for it. CONFIG_THP_SWAP needs to split one THP into normal pages and adds them all to one bio. With multipage-bvec, it just takes one bvec to hold them all. Reviewed-by: Omar Sandoval Reviewed-by:

[Cluster-devel] [PATCH V14 16/18] block: document usage of bio iterator helpers

2019-01-21 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- Documentation/block/biovecs.txt | 25

[Cluster-devel] [PATCH V14 17/18] block: kill QUEUE_FLAG_NO_SG_MERGE

2019-01-21 Thread Ming Lei
Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), physical segment number is mainly figured out in blk_queue_split() for fast path, and the flag of BIO_SEG_VALID is set there too. Now only blk_recount_segments() and blk_recalc_rq_segments() use this flag. Basically

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Christoph Hellwig
On Mon, Jan 21, 2019 at 04:17:47PM +0800, Ming Lei wrote: > V14: > - drop patch(patch 4 in V13) for renaming bvec helpers, as suggested by > Jens > - use mp_bvec_* as multi-page bvec helper name WTF? Where is this coming from? mp is just a nightmare of a name, and I also didn't see

[Cluster-devel] [PATCH V14 18/18] block: kill BLK_MQ_F_SG_MERGE

2019-01-21 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 - drivers/block/loop.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/rbd.c

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Christoph Hellwig
On Mon, Jan 21, 2019 at 04:37:12PM +0800, Ming Lei wrote: > On Mon, Jan 21, 2019 at 09:22:46AM +0100, Christoph Hellwig wrote: > > On Mon, Jan 21, 2019 at 04:17:47PM +0800, Ming Lei wrote: > > > V14: > > > - drop patch(patch 4 in V13) for renaming bvec helpers, as suggested by > > > Jens > > >

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Ming Lei
On Mon, Jan 21, 2019 at 09:22:46AM +0100, Christoph Hellwig wrote: > On Mon, Jan 21, 2019 at 04:17:47PM +0800, Ming Lei wrote: > > V14: > > - drop patch(patch 4 in V13) for renaming bvec helpers, as suggested by > > Jens > > - use mp_bvec_* as multi-page bvec helper name > > WTF? Where

[Cluster-devel] [PATCH V14 13/18] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2019-01-21 Thread Ming Lei
This patch introduces one extra iterator variable to bio_for_each_segment_all(), then we can allow bio_for_each_segment_all() to iterate over multi-page bvec. Given it is just one mechannical & simple change on all bio_for_each_segment_all() users, this patch does tree-wide change in one single

[Cluster-devel] [PATCH V14 09/18] fs/buffer.c: use bvec iterator to truncate the bio

2019-01-21 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use mp_bvec_last_segment() to truncate the bio. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- fs/buffer.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-)

[Cluster-devel] [PATCH V14 14/18] block: enable multipage bvecs

2019-01-21 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/bio.c | 22 +++--- fs/iomap.c | 4 ++-- fs/xfs/xfs_aops.c | 4 ++-- include/linux/bio.h | 2 +- 4 files changed, 20 insertions(+), 12

[Cluster-devel] [PATCH V14 12/18] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2019-01-21 Thread Ming Lei
bch_bio_alloc_pages() is always called on one new bio, so it is safe to access the bvec table directly. Given it is the only kind of this case, open code the bvec table access since bio_for_each_segment_all() will be changed to support for iterating over multipage bvec. Acked-by: Coly Li

[Cluster-devel] [PATCH V14 11/18] block: loop: pass multi-page bvec to iov_iter

2019-01-21 Thread Ming Lei
iov_iter is implemented on bvec itererator helpers, so it is safe to pass multi-page bvec to it, and this way is much more efficient than passing one page in each bvec. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- drivers/block/loop.c | 20

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Ming Lei
On Mon, Jan 21, 2019 at 09:38:10AM +0100, Christoph Hellwig wrote: > On Mon, Jan 21, 2019 at 04:37:12PM +0800, Ming Lei wrote: > > On Mon, Jan 21, 2019 at 09:22:46AM +0100, Christoph Hellwig wrote: > > > On Mon, Jan 21, 2019 at 04:17:47PM +0800, Ming Lei wrote: > > > > V14: > > > > - drop

[Cluster-devel] [PATCH V14 08/18] block: introduce mp_bvec_last_segment()

2019-01-21 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 22 ++ 1 file changed, 22 insertions(+) diff --git

[Cluster-devel] [PATCH V14 07/18] block: use bio_for_each_mp_bvec() to map sg

2019-01-21 Thread Ming Lei
It is more efficient to use bio_for_each_mp_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/blk-merge.c | 70

[Cluster-devel] [PATCH V14 06/18] block: use bio_for_each_mp_bvec() to compute multi-page bvec count

2019-01-21 Thread Ming Lei
First it is more efficient to use bio_for_each_mp_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_mp_bvec() is used, the bvec may need to be splitted because its length can be very longer

[Cluster-devel] [PATCH V14 10/18] btrfs: use mp_bvec_last_segment to get bio's last page

2019-01-21 Thread Ming Lei
Preparing for supporting multi-page bvec. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- fs/btrfs/extent_io.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index dc8ba3ee515d..986ef49b0269 100644 ---

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Sagi Grimberg
V14: - drop patch(patch 4 in V13) for renaming bvec helpers, as suggested by Jens - use mp_bvec_* as multi-page bvec helper name - fix one build issue, which is caused by missing one converion of bio_for_each_segment_all in fs/gfs2 - fix one 32bit ARCH

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Ming Lei
On Mon, Jan 21, 2019 at 01:43:21AM -0800, Sagi Grimberg wrote: > > > V14: > > - drop patch(patch 4 in V13) for renaming bvec helpers, as suggested by > > Jens > > - use mp_bvec_* as multi-page bvec helper name > > - fix one build issue, which is caused by missing one converion of > >

[Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Ming Lei
Hi, This patchset brings multi-page bvec into block layer: 1) what is multi-page bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multi-page bvec introduced?

[Cluster-devel] [PATCH V14 02/18] block: don't use bio->bi_vcnt to figure out segment number

2019-01-21 Thread Ming Lei
It is wrong to use bio->bi_vcnt to figure out how many segments there are in the bio even though CLONED flag isn't set on this bio, because this bio may be splitted or advanced. So always use bio_segments() in blk_recount_segments(), and it shouldn't cause any performance loss now because the

[Cluster-devel] [PATCH V14 01/18] btrfs: look at bi_size for repair decisions

2019-01-21 Thread Ming Lei
From: Christoph Hellwig bio_readpage_error currently uses bi_vcnt to decide if it is worth retrying an I/O. But the vector count is mostly an implementation artifact - it really should figure out if there is more than a single sector worth retrying. Use bi_size for that and shift by

[Cluster-devel] [PATCH V14 05/18] block: introduce bio_for_each_mp_bvec() and rq_for_each_mp_bvec()

2019-01-21 Thread Ming Lei
bio_for_each_mp_bvec() is used for iterating over multi-page bvec for bio split & merge code. rq_for_each_mp_bvec() can be used for drivers which may handle the multi-page bvec directly, so far loop is one perfect use case. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval

[Cluster-devel] [PATCH V14 03/18] block: remove bvec_iter_rewind()

2019-01-21 Thread Ming Lei
Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes bio_rewind_iter(), then no one uses bvec_iter_rewind() any more, so remove it. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bvec.h | 24 1 file

[Cluster-devel] [PATCH V14 04/18] block: introduce multi-page bvec helpers

2019-01-21 Thread Ming Lei
This patch introduces helpers of 'mp_bvec_iter_*' for multi-page bvec support. The introduced helpers treate one bvec as real multi-page segment, which may include more than one pages. The existed helpers of bvec_iter_* are interfaces for supporting current bvec iterator which is thought as