[Cluster-devel] [PATCH V13 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2019-01-11 Thread Ming Lei
Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), physical segment number is mainly figured out in blk_queue_split() for fast path, and the flag of BIO_SEG_VALID is set there too. Now only blk_recount_segments() and blk_recalc_rq_segments() use this flag. Basically

[Cluster-devel] [PATCH V13 19/19] block: kill BLK_MQ_F_SG_MERGE

2019-01-11 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 - drivers/block/loop.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/rbd.c

[Cluster-devel] [PATCH V13 15/19] block: enable multipage bvecs

2019-01-11 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/bio.c | 22 +++--- fs/iomap.c | 4 ++-- fs/xfs/xfs_aops.c | 4 ++-- include/linux/bio.h | 2 +- 4 files changed, 20 insertions(+), 12

[Cluster-devel] [PATCH V13 16/19] block: always define BIO_MAX_PAGES as 256

2019-01-11 Thread Ming Lei
Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to increase BIO_MAX_PAGES for it. CONFIG_THP_SWAP needs to split one THP into normal pages and adds them all to one bio. With multipage-bvec, it just takes one bvec to hold them all. Reviewed-by: Omar Sandoval Reviewed-by:

[Cluster-devel] [PATCH V13 14/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2019-01-11 Thread Ming Lei
This patch introduces one extra iterator variable to bio_for_each_segment_all(), then we can allow bio_for_each_segment_all() to iterate over multi-page bvec. Given it is just one mechannical & simple change on all bio_for_each_segment_all() users, this patch does tree-wide change in one single

[Cluster-devel] [PATCH V13 13/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2019-01-11 Thread Ming Lei
bch_bio_alloc_pages() is always called on one new bio, so it is safe to access the bvec table directly. Given it is the only kind of this case, open code the bvec table access since bio_for_each_segment_all() will be changed to support for iterating over multipage bvec. Acked-by: Coly Li

[Cluster-devel] [PATCH V13 01/19] btrfs: look at bi_size for repair decisions

2019-01-11 Thread Ming Lei
From: Christoph Hellwig bio_readpage_error currently uses bi_vcnt to decide if it is worth retrying an I/O. But the vector count is mostly an implementation artifact - it really should figure out if there is more than a single sector worth retrying. Use bi_size for that and shift by

[Cluster-devel] [PATCH V13 00/19] block: support multi-page bvec

2019-01-11 Thread Ming Lei
Hi, This patchset brings multi-page bvec into block layer: 1) what is multi-page bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multi-page bvec introduced?

[Cluster-devel] [PATCH V13 02/19] block: don't use bio->bi_vcnt to figure out segment number

2019-01-11 Thread Ming Lei
It is wrong to use bio->bi_vcnt to figure out how many segments there are in the bio even though CLONED flag isn't set on this bio, because this bio may be splitted or advanced. So always use bio_segments() in blk_recount_segments(), and it shouldn't cause any performance loss now because the

[Cluster-devel] [PATCH V13 04/19] block: rename bvec helpers

2019-01-11 Thread Ming Lei
We will support multi-page bvec soon, and have to deal with single-page vs multi-page bvec. This patch follows Christoph's suggestion to rename all the following helpers: for_each_bvec bvec_iter_bvec bvec_iter_len bvec_iter_page bvec_iter_offset into:

[Cluster-devel] [PATCH V13 06/19] block: introduce bio_for_each_bvec() and rq_for_each_bvec()

2019-01-11 Thread Ming Lei
bio_for_each_bvec() is used for iterating over multi-page bvec for bio split & merge code. rq_for_each_bvec() can be used for drivers which may handle the multi-page bvec directly, so far loop is one perfect use case. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by:

[Cluster-devel] [PATCH V13 05/19] block: introduce multi-page bvec helpers

2019-01-11 Thread Ming Lei
This patch introduces helpers of 'bvec_iter_*' for multi-page bvec support. The introduced helpers treate one bvec as real multi-page segment, which may include more than one pages. The existed helpers of bvec_iter_* are interfaces for supporting current bvec iterator which is thought as

[Cluster-devel] [PATCH V13 03/19] block: remove bvec_iter_rewind()

2019-01-11 Thread Ming Lei
Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes bio_rewind_iter(), then no one uses bvec_iter_rewind() any more, so remove it. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/bvec.h | 24 1 file

[Cluster-devel] [PATCH V13 07/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2019-01-11 Thread Ming Lei
First it is more efficient to use bio_for_each_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_bvec() is used, the bvec may need to be splitted because its length can be very longer than

[Cluster-devel] [PATCH V13 17/19] block: document usage of bio iterator helpers

2019-01-11 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- Documentation/block/biovecs.txt | 25

[Cluster-devel] [PATCH V13 10/19] fs/buffer.c: use bvec iterator to truncate the bio

2019-01-11 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use bvec_last_segment() to truncate the bio. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- fs/buffer.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-)

[Cluster-devel] [PATCH V13 08/19] block: use bio_for_each_bvec() to map sg

2019-01-11 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/blk-merge.c | 70

[Cluster-devel] [PATCH V13 09/19] block: introduce bvec_last_segment()

2019-01-11 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 22 ++ 1 file changed, 22 insertions(+) diff --git

[Cluster-devel] [PATCH V13 11/19] btrfs: use bvec_last_segment to get bio's last page

2019-01-11 Thread Ming Lei
Preparing for supporting multi-page bvec. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- fs/btrfs/extent_io.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index dc8ba3ee515d..c092f88700bd 100644 ---

[Cluster-devel] [PATCH V13 12/19] block: loop: pass multi-page bvec to iov_iter

2019-01-11 Thread Ming Lei
iov_iter is implemented on bvec itererator helpers, so it is safe to pass multi-page bvec to it, and this way is much more efficient than passing one page in each bvec. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- drivers/block/loop.c | 20

[Cluster-devel] [PATCH 05/11] gfs2: Only use struct gfs2_rbm for bitmap manipulations

2019-01-11 Thread Andreas Gruenbacher
GFS2 uses struct gfs2_rbm to represent a filesystem block number as a bit position within a resource group. This representation is used in the bitmap manipulation code to prevent excessive conversions between block numbers and bit positions, but also in struct gfs2_blkreserv which is part of

[Cluster-devel] [PATCH 08/11] gfs2: Add buffer head to struct gfs2_rgrpd

2019-01-11 Thread Andreas Gruenbacher
Prepare for removing the buffer head from struct gfs2_bitmap by adding a buffer head to struct gfs2_rgrpd for the resource group header. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/incore.h | 1 + fs/gfs2/rgrp.c | 46 +++--- 2 files changed, 28

[Cluster-devel] [PATCH 11/11] gfs2: Skip gfs2_metatype_check for cached buffers

2019-01-11 Thread Andreas Gruenbacher
When reading in buffers from disk, set a new BH_Verify buffer head flag. In gfs2_metatype_check, skip the check if BH_Verify is cleared and clear the flag when checking. That way, we'll only check the metatype once when reading buffers from disk, and not when the buffers were already in the page

[Cluster-devel] [PATCH 07/11] gfs2: Minor gfs2_rgrp_send_discards cleanup

2019-01-11 Thread Andreas Gruenbacher
Pass the resource group and bitmap index to gfs2_rgrp_send_discards separately: one of the next patches will remove bi_bh from struct gfs2_bitmap, so we'll need to get it from the resource group in the future. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/lops.c | 3 ++- fs/gfs2/rgrp.c | 11

[Cluster-devel] [PATCH 04/11] gfs2: Minor gfs2_free_extlen cleanup

2019-01-11 Thread Andreas Gruenbacher
Limit the n_unaligned argument of gfs2_unaligned_extlen to the number of blocks to scan so that the len argument can never underrun. Slightly simplify the unaligned block logic in gfs2_free_extlen. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/rgrp.c | 30 +- 1

[Cluster-devel] [PATCH 09/11] gfs2: Read bitmap buffers on demand

2019-01-11 Thread Andreas Gruenbacher
Before this patch, when locking a resource group, gfs2 would read in the resource group header and all the bitmap buffers of the resource group. Those buffers would then be locked into memory until the resource group is unlocked, which will happen when the filesystem is unmounted or when

[Cluster-devel] [PATCH 10/11] gfs2: Clean up assertion, consistency check, and error reporting functions

2019-01-11 Thread Andreas Gruenbacher
Instead of passing the __func__, __FILE__, and __LINE__ pre-processor macros to each of those functions, print the location of the caller via: printk(%pS", (void *)_RET_IP_). This gives enough context information to locate where in the code an error occurred, and reduces the code size by about

[Cluster-devel] [PATCH 00/11] Read bitmap buffers on demand

2019-01-11 Thread Andreas Gruenbacher
Here's a bunch of patches that have accumulated here lately. The most interesting one is likely "gfs2: Read bitmap buffers on demand" which should make filesystems with large resource groups more efficient. Thanks, Andreas Andreas Gruenbacher (11): gfs2: Fix setting GBF_FULL in gfs2_rbm_find

[Cluster-devel] [PATCH 02/11] gfs2: Rename minext => minlen

2019-01-11 Thread Andreas Gruenbacher
Rename the minext parameters which represent extent lengths to minlen to avoid confusion with maxext, which are actual extents. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/rgrp.c | 29 +++-- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git

[Cluster-devel] [PATCH 03/11] gfs2: Don't use struct gfs2_rbm in struct gfs2_extent

2019-01-11 Thread Andreas Gruenbacher
Don't use struct gfs2_rbm in struct gfs2_extent: the extent will always be in the same resource group, so there is no need to track the resource group separately. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/rgrp.c | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-) diff --git

[Cluster-devel] [PATCH 06/11] gfs2: Minor gfs2_adjust_reservation and gfs2_alloc_extent cleanups

2019-01-11 Thread Andreas Gruenbacher
Pass the resource group and starting position to gfs2_adjust_reservation directly instead of passing an rbm. With this change, gfs2_alloc_blocks no longer needs the rbm after calling gfs2_alloc_extent, so gfs2_alloc_extent can modify the rbm instead of creating a copy. Signed-off-by: Andreas

[Cluster-devel] [PATCH 01/11] gfs2: Fix setting GBF_FULL in gfs2_rbm_find

2019-01-11 Thread Andreas Gruenbacher
In gfs2_rbm_find, set the GBF_FULL flag whenever there are no free blocks in an entire bitmap. That requires that gfs2_bitfit scans the entire bitmap (offset == 0), but doesn't depend on initial_offset. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/rgrp.c | 13 + 1 file changed, 5