Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Sagi Grimberg
The only user in your final tree seems to be the loop driver, and even that one only uses the helper for read/write bios. I think something like this would be much simpler in the end: The recently submitted nvme-tcp host driver should also be a user of this. Does it make sense to keep it as

Re: [Cluster-devel] [PATCH 0/2] gfs2: improvements to recovery and withdraw process

2018-11-20 Thread Steven Whitehouse
Hi, On 08/11/18 20:25, Bob Peterson wrote: Hi, This is a first draft of a two-patch set to fix some of the nasty journal recovery problems I've found lately. The problems have to do with file system corruption caused when recovery replays a journal after the resource group blocks have been

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Ming Lei
On Tue, Nov 20, 2018 at 12:11:35PM -0800, Sagi Grimberg wrote: > > > > > The only user in your final tree seems to be the loop driver, and > > > > even that one only uses the helper for read/write bios. > > > > > > > > I think something like this would be much simpler in the end: > > > > > >

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Sagi Grimberg
Yeah, that is the most common example, given merge is enabled in most of cases. If the driver or device doesn't care merge, you can disable it and always get single bio request, then the bio's bvec table can be reused for send(). Does bvec_iter span bvecs with your patches? I didn't see that

[Cluster-devel] FYI: dlm.service possibly silenty overwritten with DisplayLink driver installer

2018-11-20 Thread Jan Pokorný
Accidentally, when searching for something systemd related, dlm.service caught my eye, and surprisingly, it was rather in a HW support in Linux SW enablement context. Briefly looking into the Ubuntu driver that allegedly contained that file (or recipe to create it, actually), I've realized the

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Sagi Grimberg
I would like to avoid growing bvec tables and keep everything preallocated. Plus, a bvec_iter operates on a bvec which means we'll need a table there as well... Not liking it so far... In case of bios in one request, we can't know how many bvecs there are except for calling rq_bvecs(), so it

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Sagi Grimberg
Wait, I see that the bvec is still a single array per bio. When you said a table I thought you meant a 2-dimentional array... I mean a new 1-d table A has to be created for multiple bios in one rq, and build it in the following way rq_for_each_bvec(tmp, rq, rq_iter)

[Cluster-devel] [PATCH V11 04/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-20 Thread Ming Lei
First it is more efficient to use bio_for_each_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_bvec() is used, the bvec may need to be splitted because its length can be very longer than

[Cluster-devel] [PATCH V11 06/19] block: introduce bvec_last_segment()

2018-11-20 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bvec.h | 22 ++ 1 file changed, 22 insertions(+) diff --git

[Cluster-devel] [PATCH V11 05/19] block: use bio_for_each_bvec() to map sg

2018-11-20 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-merge.c | 68 +++ 1 file

[Cluster-devel] [PATCH V11 03/19] block: introduce bio_for_each_bvec()

2018-11-20 Thread Ming Lei
This helper is used for iterating over multi-page bvec for bio split & merge code. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/bio.h | 25 ++--- include/linux/bvec.h | 36 +--- 2 files changed, 51 insertions(+), 10

[Cluster-devel] [PATCH V11 07/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-20 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use bvec_last_segment() to truncate the bio. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- fs/buffer.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-)

[Cluster-devel] [PATCH V11 08/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-20 Thread Ming Lei
Preparing for supporting multi-page bvec. Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- fs/btrfs/extent_io.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index d228f706ff3e..5d5965297e7e 100644 ---

[Cluster-devel] [PATCH V11 02/19] block: introduce multi-page bvec helpers

2018-11-20 Thread Ming Lei
This patch introduces helpers of 'segment_iter_*' for multipage bvec support. The introduced helpers treate one bvec as real multi-page segment, which may include more than one pages. The existed helpers of bvec_iter_* are interfaces for supporting current bvec iterator which is thought as

[Cluster-devel] [PATCH V11 00/19] block: support multi-page bvec

2018-11-20 Thread Ming Lei
Hi, This patchset brings multi-page bvec into block layer: 1) what is multi-page bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multi-page bvec introduced?

[Cluster-devel] [PATCH V11 01/19] block: don't use bio->bi_vcnt to figure out segment number

2018-11-20 Thread Ming Lei
It is wrong to use bio->bi_vcnt to figure out how many segments there are in the bio even though CLONED flag isn't set on this bio, because this bio may be splitted or advanced. So always use bio_segments() in blk_recount_segments(), and it shouldn't cause any performance loss now because the

[Cluster-devel] [PATCH V11 19/19] block: kill BLK_MQ_F_SG_MERGE

2018-11-20 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-mq-debugfs.c | 1 - drivers/block/loop.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/rbd.c

[Cluster-devel] [PATCH V11 17/19] block: document usage of bio iterator helpers

2018-11-20 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Signed-off-by: Ming Lei --- Documentation/block/biovecs.txt | 24 1 file changed, 24 insertions(+) diff --git

[Cluster-devel] [PATCH V11 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-20 Thread Ming Lei
Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), physical segment number is mainly figured out in blk_queue_split() for fast path, and the flag of BIO_SEG_VALID is set there too. Now only blk_recount_segments() and blk_recalc_rq_segments() use this flag. Basically

[Cluster-devel] [PATCH V11 16/19] block: always define BIO_MAX_PAGES as 256

2018-11-20 Thread Ming Lei
Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to increase BIO_MAX_PAGES for it. CONFIG_THP_SWAP needs to split one THP into normal pages and adds them all to one bio. With multipage-bvec, it just takes one bvec to hold them all. Reviewed-by: Christoph Hellwig Signed-off-by:

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Ming Lei
On Tue, Nov 20, 2018 at 07:20:45PM -0800, Sagi Grimberg wrote: > > > Not sure I understand the 'blocking' problem in this case. > > > > We can build a bvec table from this req, and send them all > > in send(), > > I would like to avoid growing bvec tables and keep everything > preallocated.

Re: [Cluster-devel] [PATCH v1 0/4] fs: fix race between llseek SEEK_END and write

2018-11-20 Thread Al Viro
On Wed, Nov 21, 2018 at 11:43:56AM +0900, Eiichi Tsukata wrote: > Some file systems (including ext4, xfs, ramfs ...) have the following > problem as I've described in the commit message of the 1/4 patch. > > The commit ef3d0fd27e90 ("vfs: do (nearly) lockless generic_file_llseek") > removed

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Sagi Grimberg
Not sure I understand the 'blocking' problem in this case. We can build a bvec table from this req, and send them all in send(), I would like to avoid growing bvec tables and keep everything preallocated. Plus, a bvec_iter operates on a bvec which means we'll need a table there as well...

[Cluster-devel] [PATCH V11 15/19] block: enable multipage bvecs

2018-11-20 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Signed-off-by: Ming Lei --- block/bio.c | 32 +++- fs/iomap.c| 2 +- fs/xfs/xfs_aops.c | 2 +- 3 files changed, 29 insertions(+), 7 deletions(-) diff --git a/block/bio.c b/block/bio.c index

[Cluster-devel] [PATCH V11 12/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-20 Thread Ming Lei
This patch introduces one extra iterator variable to bio_for_each_segment_all(), then we can allow bio_for_each_segment_all() to iterate over multi-page bvec. Given it is just one mechannical & simple change on all bio_for_each_segment_all() users, this patch does tree-wide change in one single

[Cluster-devel] [PATCH V11 14/19] block: handle non-cluster bio out of blk_bio_segment_split

2018-11-20 Thread Ming Lei
We will enable multi-page bvec soon, but non-cluster queue can't handle the multi-page bvec at all. This patch borrows bounce's idea to clone new single-page bio for non-cluster queue, and moves its handling out of blk_bio_segment_split(). Signed-off-by: Ming Lei --- block/Makefile | 3

[Cluster-devel] [PATCH V11 13/19] block: move bounce_clone_bio into bio.c

2018-11-20 Thread Ming Lei
We will reuse bounce_clone_bio() for cloning bio in case of !blk_queue_cluster(q), so move this helper into bio.c and rename it as bio_clone_bioset(). No function change. Signed-off-by: Ming Lei --- block/bio.c| 69 + block/blk.h|

[Cluster-devel] [PATCH V11 09/19] btrfs: move bio_pages_all() to btrfs

2018-11-20 Thread Ming Lei
BTRFS is the only user of this helper, so move this helper into BTRFS, and implement it via bio_for_each_segment_all(), since bio->bi_vcnt may not equal to number of pages after multipage bvec is enabled. Signed-off-by: Ming Lei --- fs/btrfs/extent_io.c | 14 +- include/linux/bio.h

[Cluster-devel] [PATCH V11 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-20 Thread Ming Lei
bch_bio_alloc_pages() is always called on one new bio, so it is safe to access the bvec table directly. Given it is the only kind of this case, open code the bvec table access since bio_for_each_segment_all() will be changed to support for iterating over multipage bvec. Acked-by: Coly Li

[Cluster-devel] [PATCH V11 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-20 Thread Ming Lei
iov_iter is implemented on bvec itererator helpers, so it is safe to pass multi-page bvec to it, and this way is much more efficient than passing one page in each bvec. Signed-off-by: Ming Lei --- drivers/block/loop.c | 20 ++-- include/linux/blkdev.h | 4 2 files

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Ming Lei
On Tue, Nov 20, 2018 at 08:42:04PM -0800, Sagi Grimberg wrote: > > > > Yeah, that is the most common example, given merge is enabled > > > in most of cases. If the driver or device doesn't care merge, > > > you can disable it and always get single bio request, then the > > > bio's bvec table can

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Sagi Grimberg
The only user in your final tree seems to be the loop driver, and even that one only uses the helper for read/write bios. I think something like this would be much simpler in the end: The recently submitted nvme-tcp host driver should also be a user of this. Does it make sense to keep it as

Re: [Cluster-devel] [PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-20 Thread Huang, Ying
Ming Lei writes: > On Thu, Nov 15, 2018 at 05:59:36PM -0800, Omar Sandoval wrote: >> On Thu, Nov 15, 2018 at 04:53:02PM +0800, Ming Lei wrote: >> > Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to >> > increase BIO_MAX_PAGES for it. >> >> You mentioned to it in the cover

Re: [Cluster-devel] gfs2: Remove vestigial bd_ops (version 2)

2018-11-20 Thread Steven Whitehouse
On 20/11/18 14:37, Bob Peterson wrote: Hi, Here is a new and improved version of the patch I posted on 16 November. Since the field is no longer needed, neither are the function parameters used to allocate a bd. --- Field bd_ops was set but never used, so I removed it, and all code

[Cluster-devel] gfs2: Remove vestigial bd_ops (version 2)

2018-11-20 Thread Bob Peterson
Hi, Here is a new and improved version of the patch I posted on 16 November. Since the field is no longer needed, neither are the function parameters used to allocate a bd. --- Field bd_ops was set but never used, so I removed it, and all code supporting it. Signed-off-by: Bob Peterson ---

Re: [Cluster-devel] [PATCH 02/13] GFS2: Make do_xmote determine its own gh parameter

2018-11-20 Thread Steven Whitehouse
Hi, On 19/11/18 21:06, Bob Peterson wrote: Hi Steve, - Original Message - On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. Before this patch, do_xmote was called with a gh parameter, but only for promotes, not demotes. This patch

Re: [Cluster-devel] [PATCH 03/13] GFS2: Eliminate a goto in finish_xmote

2018-11-20 Thread Steven Whitehouse
Hi, On 19/11/18 21:26, Bob Peterson wrote: Hi, - Original Message - On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. This patch eliminates a goto in function finish_xmote so we can begin unraveling the cryptic logic with later

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-20 Thread Christoph Hellwig
On Mon, Nov 19, 2018 at 04:49:27PM -0800, Sagi Grimberg wrote: > >> The only user in your final tree seems to be the loop driver, and >> even that one only uses the helper for read/write bios. >> >> I think something like this would be much simpler in the end: > > The recently submitted nvme-tcp