Re: [Cluster-devel] [PATCH V15 14/18] block: enable multipage bvecs

2019-02-27 Thread Ming Lei
On Wed, Feb 27, 2019 at 08:47:09PM +, Jon Hunter wrote: > > On 21/02/2019 08:42, Marek Szyprowski wrote: > > Dear All, > > > > On 2019-02-15 12:13, Ming Lei wrote: > >> This patch pulls the trigger for multi-page bvecs. > >> > >> Revie

Re: [Cluster-devel] [PATCH V15 14/18] block: enable multipage bvecs

2019-02-22 Thread Ming Lei
On Thu, Feb 21, 2019 at 11:22:39AM +0100, Marek Szyprowski wrote: > Hi Ming, > > On 2019-02-21 11:16, Ming Lei wrote: > > On Thu, Feb 21, 2019 at 11:08:19AM +0100, Marek Szyprowski wrote: > >> On 2019-02-21 10:57, Ming Lei wrote: > >>> On Thu, Feb 21, 2019 a

Re: [Cluster-devel] [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Ming Lei
On Thu, Feb 21, 2019 at 11:08:19AM +0100, Marek Szyprowski wrote: > Hi Ming, > > On 2019-02-21 10:57, Ming Lei wrote: > > On Thu, Feb 21, 2019 at 09:42:59AM +0100, Marek Szyprowski wrote: > >> On 2019-02-15 12:13, Ming Lei wrote: > >>> This patch pu

Re: [Cluster-devel] [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Ming Lei
On Thu, Feb 21, 2019 at 09:42:59AM +0100, Marek Szyprowski wrote: > Dear All, > > On 2019-02-15 12:13, Ming Lei wrote: > > This patch pulls the trigger for multi-page bvecs. > > > > Reviewed-by: Omar Sandoval > > Signed-off-by: Ming Lei > > Since Lin

Re: [Cluster-devel] [dm-devel] [PATCH V15 00/18] block: support multi-page bvec

2019-02-19 Thread Ming Lei
On Tue, Feb 19, 2019 at 08:28:19AM -0800, Bart Van Assche wrote: > On Sun, 2019-02-17 at 21:11 +0800, Ming Lei wrote: > > The following patch should fix this issue: > > > > > > diff --git a/block/blk-merge.c b/block/blk-merge.c > > index bed065904677..066b66

Re: [Cluster-devel] [dm-devel] [PATCH V15 00/18] block: support multi-page bvec

2019-02-17 Thread Ming Lei
On Sun, Feb 17, 2019 at 09:13:32PM +0800, Ming Lei wrote: > On Fri, Feb 15, 2019 at 10:59:47AM -0700, Jens Axboe wrote: > > On 2/15/19 10:14 AM, Bart Van Assche wrote: > > > On Fri, 2019-02-15 at 08:49 -0700, Jens Axboe wrote: > > >> On 2/15/19 4:13 AM, Ming Lei wrote

Re: [Cluster-devel] [dm-devel] [PATCH V15 00/18] block: support multi-page bvec

2019-02-17 Thread Ming Lei
On Fri, Feb 15, 2019 at 10:59:47AM -0700, Jens Axboe wrote: > On 2/15/19 10:14 AM, Bart Van Assche wrote: > > On Fri, 2019-02-15 at 08:49 -0700, Jens Axboe wrote: > >> On 2/15/19 4:13 AM, Ming Lei wrote: > >>> This patchset brings multi-page bvec into block layer:

[Cluster-devel] [PATCH V15 18/18] block: kill BLK_MQ_F_SG_MERGE

2019-02-15 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 - drivers/block/loop.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/rbd.c

[Cluster-devel] [PATCH V15 15/18] block: always define BIO_MAX_PAGES as 256

2019-02-15 Thread Ming Lei
-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bio.h | 8 1 file changed, 8 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index 9f77adcfde82..bdd11d4c2f05 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -34,15 +34,7 @@ #define BIO_BUG_ON #endif

[Cluster-devel] [PATCH V15 17/18] block: kill QUEUE_FLAG_NO_SG_MERGE

2019-02-15 Thread Ming Lei
Signed-off-by: Ming Lei --- block/blk-merge.c | 31 ++- block/blk-mq-debugfs.c | 1 - block/blk-mq.c | 3 --- drivers/md/dm-table.c | 13 - include/linux/blkdev.h | 1 - 5 files changed, 6 insertions(+), 43 deletions(-) diff --git a/block/blk

[Cluster-devel] [PATCH V15 16/18] block: document usage of bio iterator helpers

2019-02-15 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- Documentation/block/biovecs.txt | 25

[Cluster-devel] [PATCH V15 14/18] block: enable multipage bvecs

2019-02-15 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/bio.c | 22 +++--- fs/iomap.c | 4 ++-- fs/xfs/xfs_aops.c | 4 ++-- include/linux/bio.h | 2 +- 4 files changed, 20 insertions(+), 12 deletions

[Cluster-devel] [PATCH V15 11/18] block: loop: pass multi-page bvec to iov_iter

2019-02-15 Thread Ming Lei
iov_iter is implemented on bvec itererator helpers, so it is safe to pass multi-page bvec to it, and this way is much more efficient than passing one page in each bvec. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- drivers/block/loop.c | 20

[Cluster-devel] [PATCH V15 10/18] btrfs: use mp_bvec_last_segment to get bio's last page

2019-02-15 Thread Ming Lei
Preparing for supporting multi-page bvec. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- fs/btrfs/extent_io.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index dc8ba3ee515d..986ef49b0269 100644 --- a/fs/btrfs

[Cluster-devel] [PATCH V15 13/18] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2019-02-15 Thread Ming Lei
ngle patch, so that we can avoid to use a temporary helper for this conversion. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/bio.c | 27 ++- block/bounce.c| 6 -- drivers/md/bc

[Cluster-devel] [PATCH V15 06/18] block: use bio_for_each_bvec() to compute multi-page bvec count

2019-02-15 Thread Ming Lei
-off-by: Ming Lei --- block/blk-merge.c | 103 +++--- 1 file changed, 83 insertions(+), 20 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index f85d878f313d..4ef56b2d2aa5 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -161,6

[Cluster-devel] [PATCH V15 12/18] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2019-02-15 Thread Ming Lei
Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- drivers/md/bcache/util.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/md/bcache/util.c b/drivers/md/bcache/util.c index 20eddeac1531..62fb917f7a4f 100644 --- a/drivers/md/bcache

[Cluster-devel] [PATCH V15 09/18] fs/buffer.c: use bvec iterator to truncate the bio

2019-02-15 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use mp_bvec_last_segment() to truncate the bio. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- fs/buffer.c | 5 - 1 file changed, 4 insertions(+), 1 deletion

[Cluster-devel] [PATCH V15 07/18] block: use bio_for_each_bvec() to map sg

2019-02-15 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/blk-merge.c | 70

[Cluster-devel] [PATCH V15 08/18] block: introduce mp_bvec_last_segment()

2019-02-15 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 22 ++ 1 file changed, 22 insertions(+) diff --git

[Cluster-devel] [PATCH V15 05/18] block: introduce bio_for_each_bvec() and rq_for_each_bvec()

2019-02-15 Thread Ming Lei
-off-by: Ming Lei --- include/linux/bio.h| 10 ++ include/linux/blkdev.h | 4 2 files changed, 14 insertions(+) diff --git a/include/linux/bio.h b/include/linux/bio.h index 72b4f7be2106..7ef8a7505c0a 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -156,6 +156,16 @@ st

[Cluster-devel] [PATCH V15 04/18] block: introduce multi-page bvec helpers

2019-02-15 Thread Ming Lei
ept not changed - returned(copied) single-page bvec is built in flight by bvec helpers from the stored multi-page bvec Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 30 +++--- 1 file changed, 27 insertions(+)

[Cluster-devel] [PATCH V15 03/18] block: remove bvec_iter_rewind()

2019-02-15 Thread Ming Lei
Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes bio_rewind_iter(), then no one uses bvec_iter_rewind() any more, so remove it. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bvec.h | 24 --

[Cluster-devel] [PATCH V15 00/18] block: support multi-page bvec

2019-02-15 Thread Ming Lei
6], http://marc.info/?t=14982021534=1=2 Christoph Hellwig (1): btrfs: look at bi_size for repair decisions Ming Lei (17): block: don't use bio->bi_vcnt to figure out segment number block: remove bvec_iter_rewind() block: introduce multi-page bvec helpers block: introduce bio_fo

[Cluster-devel] [PATCH V15 01/18] btrfs: look at bi_size for repair decisions

2019-02-15 Thread Ming Lei
From: Christoph Hellwig bio_readpage_error currently uses bi_vcnt to decide if it is worth retrying an I/O. But the vector count is mostly an implementation artifact - it really should figure out if there is more than a single sector worth retrying. Use bi_size for that and shift by

[Cluster-devel] [PATCH V15 02/18] block: don't use bio->bi_vcnt to figure out segment number

2019-02-15 Thread Ming Lei
t if it isn't less than max segments") Signed-off-by: Ming Lei --- block/blk-merge.c | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 71e9ac03f621..f85d878f313d 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Ming Lei
On Mon, Jan 21, 2019 at 01:43:21AM -0800, Sagi Grimberg wrote: > > > V14: > > - drop patch(patch 4 in V13) for renaming bvec helpers, as suggested by > > Jens > > - use mp_bvec_* as multi-page bvec helper name > > - fix one build issue, which is caused by missing one converion of > >

Re: [Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Ming Lei
On Mon, Jan 21, 2019 at 09:38:10AM +0100, Christoph Hellwig wrote: > On Mon, Jan 21, 2019 at 04:37:12PM +0800, Ming Lei wrote: > > On Mon, Jan 21, 2019 at 09:22:46AM +0100, Christoph Hellwig wrote: > > > On Mon, Jan 21, 2019 at 04:17:47PM +0800, Ming Lei wrote: > > > &g

[Cluster-devel] [PATCH V14 17/18] block: kill QUEUE_FLAG_NO_SG_MERGE

2019-01-21 Thread Ming Lei
Signed-off-by: Ming Lei --- block/blk-merge.c | 31 ++- block/blk-mq-debugfs.c | 1 - block/blk-mq.c | 3 --- drivers/md/dm-table.c | 13 - include/linux/blkdev.h | 1 - 5 files changed, 6 insertions(+), 43 deletions(-) diff --git a/block/blk

[Cluster-devel] [PATCH V14 18/18] block: kill BLK_MQ_F_SG_MERGE

2019-01-21 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 - drivers/block/loop.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/rbd.c

[Cluster-devel] [PATCH V14 16/18] block: document usage of bio iterator helpers

2019-01-21 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- Documentation/block/biovecs.txt | 25

[Cluster-devel] [PATCH V14 15/18] block: always define BIO_MAX_PAGES as 256

2019-01-21 Thread Ming Lei
-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bio.h | 8 1 file changed, 8 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index af288f6e8ab0..1d279a6ae737 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -34,15 +34,7 @@ #define BIO_BUG_ON #endif

[Cluster-devel] [PATCH V14 12/18] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2019-01-21 Thread Ming Lei
Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- drivers/md/bcache/util.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/md/bcache/util.c b/drivers/md/bcache/util.c index 20eddeac1531..62fb917f7a4f 100644 --- a/drivers/md/bcache

[Cluster-devel] [PATCH V14 11/18] block: loop: pass multi-page bvec to iov_iter

2019-01-21 Thread Ming Lei
iov_iter is implemented on bvec itererator helpers, so it is safe to pass multi-page bvec to it, and this way is much more efficient than passing one page in each bvec. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- drivers/block/loop.c | 20

[Cluster-devel] [PATCH V14 09/18] fs/buffer.c: use bvec iterator to truncate the bio

2019-01-21 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use mp_bvec_last_segment() to truncate the bio. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- fs/buffer.c | 5 - 1 file changed, 4 insertions(+), 1 deletion

[Cluster-devel] [PATCH V14 14/18] block: enable multipage bvecs

2019-01-21 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/bio.c | 22 +++--- fs/iomap.c | 4 ++-- fs/xfs/xfs_aops.c | 4 ++-- include/linux/bio.h | 2 +- 4 files changed, 20 insertions(+), 12 deletions

[Cluster-devel] [PATCH V14 13/18] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2019-01-21 Thread Ming Lei
ngle patch, so that we can avoid to use a temporary helper for this conversion. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/bio.c | 27 ++- block/bounce.c| 6 -- drivers/md/bc

[Cluster-devel] [PATCH V14 10/18] btrfs: use mp_bvec_last_segment to get bio's last page

2019-01-21 Thread Ming Lei
Preparing for supporting multi-page bvec. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- fs/btrfs/extent_io.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index dc8ba3ee515d..986ef49b0269 100644 --- a/fs/btrfs

[Cluster-devel] [PATCH V14 07/18] block: use bio_for_each_mp_bvec() to map sg

2019-01-21 Thread Ming Lei
It is more efficient to use bio_for_each_mp_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/blk-merge.c | 70

[Cluster-devel] [PATCH V14 06/18] block: use bio_for_each_mp_bvec() to compute multi-page bvec count

2019-01-21 Thread Ming Lei
Signed-off-by: Ming Lei --- block/blk-merge.c | 103 +++--- 1 file changed, 83 insertions(+), 20 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index f85d878f313d..2dfc30d8bc77 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c

[Cluster-devel] [PATCH V14 08/18] block: introduce mp_bvec_last_segment()

2019-01-21 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 22 ++ 1 file changed, 22 insertions(+) diff --git

[Cluster-devel] [PATCH V14 04/18] block: introduce multi-page bvec helpers

2019-01-21 Thread Ming Lei
ept not changed - returned(copied) single-page bvec is built in flight by bvec helpers from the stored multi-page bvec Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 30 +++--- 1 file changed, 27 insertions(+)

[Cluster-devel] [PATCH V14 03/18] block: remove bvec_iter_rewind()

2019-01-21 Thread Ming Lei
Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes bio_rewind_iter(), then no one uses bvec_iter_rewind() any more, so remove it. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bvec.h | 24 --

[Cluster-devel] [PATCH V14 05/18] block: introduce bio_for_each_mp_bvec() and rq_for_each_mp_bvec()

2019-01-21 Thread Ming Lei
-off-by: Ming Lei --- include/linux/bio.h| 10 ++ include/linux/blkdev.h | 4 2 files changed, 14 insertions(+) diff --git a/include/linux/bio.h b/include/linux/bio.h index 72b4f7be2106..730288145568 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -156,6 +15

[Cluster-devel] [PATCH V14 02/18] block: don't use bio->bi_vcnt to figure out segment number

2019-01-21 Thread Ming Lei
t if it isn't less than max segments") Signed-off-by: Ming Lei --- block/blk-merge.c | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 71e9ac03f621..f85d878f313d 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@

[Cluster-devel] [PATCH V14 00/18] block: support multi-page bvec

2019-01-21 Thread Ming Lei
4], http://marc.info/?l=linux-mm=147745525801433=2 [5], http://marc.info/?t=14956948457=1=2 [6], http://marc.info/?t=14982021534=1=2 Christoph Hellwig (1): btrfs: look at bi_size for repair decisions Ming Lei (17): block: don't use bio->bi_vcnt to figure out segment number bl

[Cluster-devel] [PATCH V14 01/18] btrfs: look at bi_size for repair decisions

2019-01-21 Thread Ming Lei
From: Christoph Hellwig bio_readpage_error currently uses bi_vcnt to decide if it is worth retrying an I/O. But the vector count is mostly an implementation artifact - it really should figure out if there is more than a single sector worth retrying. Use bi_size for that and shift by

[Cluster-devel] [PATCH V13 17/19] block: document usage of bio iterator helpers

2019-01-11 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- Documentation/block/biovecs.txt | 25

[Cluster-devel] [PATCH V13 16/19] block: always define BIO_MAX_PAGES as 256

2019-01-11 Thread Ming Lei
-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bio.h | 8 1 file changed, 8 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index 1ece9f30294b..54ef81f11f83 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -34,15 +34,7 @@ #define BIO_BUG_ON #endif

[Cluster-devel] [PATCH V13 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2019-01-11 Thread Ming Lei
Signed-off-by: Ming Lei --- block/blk-merge.c | 31 ++- block/blk-mq-debugfs.c | 1 - block/blk-mq.c | 3 --- drivers/md/dm-table.c | 13 - include/linux/blkdev.h | 1 - 5 files changed, 6 insertions(+), 43 deletions(-) diff --git a/block/blk

[Cluster-devel] [PATCH V13 15/19] block: enable multipage bvecs

2019-01-11 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/bio.c | 22 +++--- fs/iomap.c | 4 ++-- fs/xfs/xfs_aops.c | 4 ++-- include/linux/bio.h | 2 +- 4 files changed, 20 insertions(+), 12 deletions

[Cluster-devel] [PATCH V13 19/19] block: kill BLK_MQ_F_SG_MERGE

2019-01-11 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 - drivers/block/loop.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/rbd.c

[Cluster-devel] [PATCH V13 14/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2019-01-11 Thread Ming Lei
ngle patch, so that we can avoid to use a temporary helper for this conversion. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/bio.c | 27 ++- block/bounce.c| 6 -- drivers/md/bc

[Cluster-devel] [PATCH V13 13/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2019-01-11 Thread Ming Lei
Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- drivers/md/bcache/util.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/md/bcache/util.c b/drivers/md/bcache/util.c index 20eddeac1531..62fb917f7a4f 100644 --- a/drivers/md/bcache

[Cluster-devel] [PATCH V13 12/19] block: loop: pass multi-page bvec to iov_iter

2019-01-11 Thread Ming Lei
iov_iter is implemented on bvec itererator helpers, so it is safe to pass multi-page bvec to it, and this way is much more efficient than passing one page in each bvec. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- drivers/block/loop.c | 20

[Cluster-devel] [PATCH V13 11/19] btrfs: use bvec_last_segment to get bio's last page

2019-01-11 Thread Ming Lei
Preparing for supporting multi-page bvec. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- fs/btrfs/extent_io.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index dc8ba3ee515d..c092f88700bd 100644 --- a/fs/btrfs

[Cluster-devel] [PATCH V13 09/19] block: introduce bvec_last_segment()

2019-01-11 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 22 ++ 1 file changed, 22 insertions(+) diff --git

[Cluster-devel] [PATCH V13 08/19] block: use bio_for_each_bvec() to map sg

2019-01-11 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/blk-merge.c | 70

[Cluster-devel] [PATCH V13 10/19] fs/buffer.c: use bvec iterator to truncate the bio

2019-01-11 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use bvec_last_segment() to truncate the bio. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- fs/buffer.c | 5 - 1 file changed, 4 insertions(+), 1 deletion

[Cluster-devel] [PATCH V13 05/19] block: introduce multi-page bvec helpers

2019-01-11 Thread Ming Lei
ept not changed - returned(copied) single-page bvec is built in flight by bvec helpers from the stored multi-page bvec Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 25 - 1 file changed, 20 insertions(+)

[Cluster-devel] [PATCH V13 06/19] block: introduce bio_for_each_bvec() and rq_for_each_bvec()

2019-01-11 Thread Ming Lei
-off-by: Ming Lei --- include/linux/bio.h| 10 ++ include/linux/blkdev.h | 4 include/linux/bvec.h | 7 +++ 3 files changed, 21 insertions(+) diff --git a/include/linux/bio.h b/include/linux/bio.h index 16a65361535f..06888d45beb4 100644 --- a/include/linux/bio.h +++ b/include/l

[Cluster-devel] [PATCH V13 07/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2019-01-11 Thread Ming Lei
-off-by: Ming Lei --- block/blk-merge.c | 99 --- 1 file changed, 79 insertions(+), 20 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index f85d878f313d..abe1c89c1253 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -161,6

[Cluster-devel] [PATCH V13 03/19] block: remove bvec_iter_rewind()

2019-01-11 Thread Ming Lei
Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes bio_rewind_iter(), then no one uses bvec_iter_rewind() any more, so remove it. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bvec.h | 24 --

[Cluster-devel] [PATCH V13 04/19] block: rename bvec helpers

2019-01-11 Thread Ming Lei
. bvec_iter_advance() isn't renamed becasue this helper is always operated on real bvec even though multi-page bvec is supported. Acked-by: Miguel Ojeda Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Suggested-by: Christoph Hellwig Signed-off-by: Ming Lei --- .clang-format | 2

[Cluster-devel] [PATCH V13 02/19] block: don't use bio->bi_vcnt to figure out segment number

2019-01-11 Thread Ming Lei
t if it isn't less than max segments") Signed-off-by: Ming Lei --- block/blk-merge.c | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 71e9ac03f621..f85d878f313d 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@

[Cluster-devel] [PATCH V13 00/19] block: support multi-page bvec

2019-01-11 Thread Ming Lei
5], http://marc.info/?t=14956948457=1=2 [6], http://marc.info/?t=14982021534=1=2 Christoph Hellwig (1): btrfs: look at bi_size for repair decisions Ming Lei (18): block: don't use bio->bi_vcnt to figure out segment number block: remove bvec_iter_rewind() block: rename bvec help

[Cluster-devel] [PATCH V13 01/19] btrfs: look at bi_size for repair decisions

2019-01-11 Thread Ming Lei
From: Christoph Hellwig bio_readpage_error currently uses bi_vcnt to decide if it is worth retrying an I/O. But the vector count is mostly an implementation artifact - it really should figure out if there is more than a single sector worth retrying. Use bi_size for that and shift by

Re: [Cluster-devel] [PATCH V12 00/20] block: support multi-page bvec

2018-11-28 Thread Ming Lei
On Wed, Nov 28, 2018 at 07:20:51PM -0700, Jens Axboe wrote: > On 11/28/18 6:30 PM, Ming Lei wrote: > >> I'm going back and forth on those one a bit. Any concerns with > >> pushing this to 4.22? > > > > My only one concern is about the warning of > > "b

Re: [Cluster-devel] [PATCH V12 00/20] block: support multi-page bvec

2018-11-28 Thread Ming Lei
On Wed, Nov 28, 2018 at 06:44:00AM -0700, Jens Axboe wrote: > On 11/25/18 7:17 PM, Ming Lei wrote: > > Hi, > > > > This patchset brings multi-page bvec into block layer: > > > > 1) what is multi-page bvec? > > > > Multipage bvecs means that

Re: [Cluster-devel] [PATCH V12 16/20] block: enable multipage bvecs

2018-11-26 Thread Ming Lei
On Mon, Nov 26, 2018 at 01:58:42PM +0100, Christoph Hellwig wrote: > > + phys_addr_t vec_end_addr = page_to_phys(bv->bv_page) + > > + bv->bv_offset + bv->bv_len; > > The name is a little confusing, as the real end addr would be -1. Maybe > throw the -1 in here, and

[Cluster-devel] [PATCH V12 20/20] block: kill BLK_MQ_F_SG_MERGE

2018-11-25 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 - drivers/block/loop.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/rbd.c

[Cluster-devel] [PATCH V12 19/20] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-25 Thread Ming Lei
Signed-off-by: Ming Lei --- block/blk-merge.c | 31 ++- block/blk-mq-debugfs.c | 1 - block/blk-mq.c | 3 --- drivers/md/dm-table.c | 13 - include/linux/blkdev.h | 1 - 5 files changed, 6 insertions(+), 43 deletions(-) diff --git a/block/blk

[Cluster-devel] [PATCH V12 18/20] block: document usage of bio iterator helpers

2018-11-25 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Signed-off-by: Ming Lei --- Documentation/block/biovecs.txt | 25 + 1 file changed, 25 insertions(+) diff --git

[Cluster-devel] [PATCH V12 17/20] block: always define BIO_MAX_PAGES as 256

2018-11-25 Thread Ming Lei
-by: Ming Lei --- include/linux/bio.h | 8 1 file changed, 8 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index 5505f74aef8b..7be48c55b14a 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -34,15 +34,7 @@ #define BIO_BUG_ON #endif -#ifdef CONFIG_THP_SWAP

[Cluster-devel] [PATCH V12 16/20] block: enable multipage bvecs

2018-11-25 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Signed-off-by: Ming Lei --- block/bio.c | 22 +++--- fs/iomap.c | 4 ++-- fs/xfs/xfs_aops.c | 4 ++-- include/linux/bio.h | 2 +- 4 files changed, 20 insertions(+), 12 deletions(-) diff --git a/block/bio.c

[Cluster-devel] [PATCH V12 12/20] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-25 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use bvec_last_segment() to truncate the bio. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- fs/buffer.c | 5 - 1 file changed, 4 insertions(+), 1 deletion

[Cluster-devel] [PATCH V12 14/20] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-25 Thread Ming Lei
Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- drivers/md/bcache/util.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/md/bcache/util.c b/drivers/md/bcache/util.c index 20eddeac1531..62fb917f7a4f 100644 --- a/drivers/md/bcache/util.c +++ b/drivers/md/bcache

[Cluster-devel] [PATCH V12 15/20] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-25 Thread Ming Lei
ngle patch, so that we can avoid to use a temporary helper for this conversion. Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/bio.c | 27 ++- block/bounce.c| 6 -- drivers/md/bcache/btree.c

[Cluster-devel] [PATCH V12 13/20] block: loop: pass multi-page bvec to iov_iter

2018-11-25 Thread Ming Lei
iov_iter is implemented on bvec itererator helpers, so it is safe to pass multi-page bvec to it, and this way is much more efficient than passing one page in each bvec. Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- drivers/block/loop.c | 20 ++-- 1 file changed, 10

[Cluster-devel] [PATCH V12 11/20] block: introduce bvec_last_segment()

2018-11-25 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 22 ++ 1 file changed, 22 insertions(+) diff --git

[Cluster-devel] [PATCH V12 09/20] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-25 Thread Ming Lei
max segment size, so we have to split the big bvec into several segments. Thirdly when splitting multi-page bvec into segments, the max segment limit may be reached, so the bio split need to be considered under this situation too. Signed-off-by: Ming Lei --- block/blk-merge.c | 100

[Cluster-devel] [PATCH V12 10/20] block: use bio_for_each_bvec() to map sg

2018-11-25 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-merge.c | 70 +++ 1 file

[Cluster-devel] [PATCH V12 08/20] block: introduce bio_for_each_bvec() and rq_for_each_bvec()

2018-11-25 Thread Ming Lei
bio_for_each_bvec() is used for iterating over multi-page bvec for bio split & merge code. rq_for_each_bvec() can be used for drivers which may handle the multi-page bvec directly, so far loop is one perfect use case. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/b

[Cluster-devel] [PATCH V12 02/20] btrfs: look at bi_size for repair decisions

2018-11-25 Thread Ming Lei
From: Christoph Hellwig bio_readpage_error currently uses bi_vcnt to decide if it is worth retrying an I/O. But the vector count is mostly an implementation artifact - it really should figure out if there is more than a single sector worth retrying. Use bi_size for that and shift by

[Cluster-devel] [PATCH V12 06/20] block: rename bvec helpers

2018-11-25 Thread Ming Lei
. bvec_iter_advance() isn't renamed becasue this helper is always operated on real bvec even though multi-page bvec is supported. Suggested-by: Christoph Hellwig Signed-off-by: Ming Lei --- .clang-format | 2 +- drivers/md/dm-integrity.c | 2 +- drivers/md/dm-io.c | 4

[Cluster-devel] [PATCH V12 07/20] block: introduce multi-page bvec helpers

2018-11-25 Thread Ming Lei
ept not changed - returned(copied) single-page bvec is built in flight by bvec helpers from the stored multi-page bvec Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 25 - 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/include/li

[Cluster-devel] [PATCH V12 05/20] block: remove bvec_iter_rewind()

2018-11-25 Thread Ming Lei
Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes bio_rewind_iter(), then no one uses bvec_iter_rewind() any more, so remove it. Signed-off-by: Ming Lei --- include/linux/bvec.h | 24 1 file changed, 24 deletions(-) diff --git a/include/linu

[Cluster-devel] [PATCH V12 04/20] block: don't use bio->bi_vcnt to figure out segment number

2018-11-25 Thread Ming Lei
segments") Signed-off-by: Ming Lei --- block/blk-merge.c | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index e69d8f8ba819..51ec6ca56a0a 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -367,13 +367,7 @@ void blk_re

[Cluster-devel] [PATCH V12 03/20] block: remove the "cluster" flag

2018-11-25 Thread Ming Lei
also affect by setting the maximum segment size and the segment boundary. Signed-off-by: Christoph Hellwig Replace virt boundary with segment boundary limit. Signed-off-by: Ming Lei --- block/blk-merge.c | 20 block/blk-settings.c| 3 --- block/blk-sysfs.c

[Cluster-devel] [PATCH V12 01/20] btrfs: remove various bio_offset arguments

2018-11-25 Thread Ming Lei
From: Christoph Hellwig The btrfs write path passes a bio_offset argument through some deep callchains including async offloading. In the end this is easily calculatable using page_offset plus the bvec offset for the first page in the bio, and only actually used by by a single function. Just

[Cluster-devel] [PATCH V12 00/20] block: support multi-page bvec

2018-11-25 Thread Ming Lei
r repair decisions block: remove the "cluster" flag Ming Lei (17): block: don't use bio->bi_vcnt to figure out segment number block: remove bvec_iter_rewind() block: rename bvec helpers block: introduce multi-page bvec helpers block: introduce bio_for_each_bvec() and rq_for_eac

Re: [Cluster-devel] [PATCH V11 15/19] block: enable multipage bvecs

2018-11-23 Thread Ming Lei
On Wed, Nov 21, 2018 at 05:12:06PM +0100, Christoph Hellwig wrote: > On Wed, Nov 21, 2018 at 11:48:13PM +0800, Ming Lei wrote: > > I guess the correct check should be: > > > > end_addr = vec_addr + bv->bv_offset + bv->bv_len; &g

Re: [Cluster-devel] [PATCH V11 12/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-22 Thread Ming Lei
On Thu, Nov 22, 2018 at 12:03:15PM +0100, Christoph Hellwig wrote: > > +/* used for chunk_for_each_segment */ > > +static inline void bvec_next_segment(const struct bio_vec *bvec, > > +struct bvec_iter_all *iter_all) > > FYI, chunk_for_each_segment doesn't exist

Re: [Cluster-devel] [PATCH V11 07/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-22 Thread Ming Lei
On Thu, Nov 22, 2018 at 11:58:49AM +0100, Christoph Hellwig wrote: > Btw, given that this is the last user of bvec_last_segment after my > other patches I think we should kill bvec_last_segment and do something > like this here: > > > diff --git a/fs/buffer.c b/fs/buffer.c > index

Re: [Cluster-devel] [PATCH V11 03/19] block: introduce bio_for_each_bvec()

2018-11-22 Thread Ming Lei
On Thu, Nov 22, 2018 at 11:30:33AM +0100, Christoph Hellwig wrote: > Btw, this patch instead of the plain rever might make it a little > more clear what is going on by skipping the confusing helper altogher > and operating on the raw bvec array: > > > diff --git a/include/linux/bio.h

Re: [Cluster-devel] [PATCH V11 14/19] block: handle non-cluster bio out of blk_bio_segment_split

2018-11-22 Thread Ming Lei
On Thu, Nov 22, 2018 at 11:41:50AM +0100, Christoph Hellwig wrote: > On Thu, Nov 22, 2018 at 06:32:09PM +0800, Ming Lei wrote: > > On Thu, Nov 22, 2018 at 11:04:28AM +0100, Christoph Hellwig wrote: > > > On Thu, Nov 22, 2018 at 05:33:00PM +0800, Ming Lei wrote: > > > >

Re: [Cluster-devel] [PATCH V11 14/19] block: handle non-cluster bio out of blk_bio_segment_split

2018-11-22 Thread Ming Lei
On Thu, Nov 22, 2018 at 11:04:28AM +0100, Christoph Hellwig wrote: > On Thu, Nov 22, 2018 at 05:33:00PM +0800, Ming Lei wrote: > > However, using virt boundary limit on non-cluster seems over-kill, > > because the bio will be over-split(each small bvec may be split as one bio) >

Re: [Cluster-devel] [PATCH V11 14/19] block: handle non-cluster bio out of blk_bio_segment_split

2018-11-22 Thread Ming Lei
On Thu, Nov 22, 2018 at 11:04:28AM +0100, Christoph Hellwig wrote: > On Thu, Nov 22, 2018 at 05:33:00PM +0800, Ming Lei wrote: > > However, using virt boundary limit on non-cluster seems over-kill, > > because the bio will be over-split(each small bvec may be split as one bio) >

Re: [Cluster-devel] [PATCH V11 03/19] block: introduce bio_for_each_bvec()

2018-11-22 Thread Ming Lei
On Wed, Nov 21, 2018 at 06:12:17PM +0100, Christoph Hellwig wrote: > On Wed, Nov 21, 2018 at 05:10:25PM +0100, Christoph Hellwig wrote: > > No - I think we can always use the code without any segment in > > bvec_iter_advance. Because bvec_iter_advance only operates on the > > iteractor, the

Re: [Cluster-devel] [PATCH V11 14/19] block: handle non-cluster bio out of blk_bio_segment_split

2018-11-22 Thread Ming Lei
On Wed, Nov 21, 2018 at 06:46:21PM +0100, Christoph Hellwig wrote: > Actually.. > > I think we can kill this code entirely. If we look at what the > clustering setting is really about it is to avoid ever merging a > segement that spans a page boundary. And we should be able to do > that with

  1   2   >