Re: [Cluster-devel] [PATCH 03/13] GFS2: Eliminate a goto in finish_xmote

2018-11-19 Thread Bob Peterson
Hi, - Original Message - > > > On 19/11/18 13:29, Bob Peterson wrote: > > This is another baby step toward a better glock state machine. > > This patch eliminates a goto in function finish_xmote so we can > > begin unraveling the cryptic logic with later patches. > > > > Signed-off-by:

Re: [Cluster-devel] [PATCH V10 04/19] block: use bio_for_each_bvec() to map sg

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:33:14PM +0100, Christoph Hellwig wrote: > > + if (!*sg) > > + return sglist; > > + else { > > No need for an else after an early return. OK, good catch! Thanks, Ming

Re: [Cluster-devel] [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 12:20:28PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > > First it is more efficient to use bio_for_each_bvec() in both > > blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how > > many multi-page bvecs there are

Re: [Cluster-devel] [PATCH V10 07/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:37:10PM +0100, Christoph Hellwig wrote: > On Thu, Nov 15, 2018 at 04:52:54PM +0800, Ming Lei wrote: > > index 2955a4ea2fa8..161e14b8b180 100644 > > --- a/fs/btrfs/compression.c > > +++ b/fs/btrfs/compression.c > > @@ -400,8 +400,11 @@ blk_status_t

Re: [Cluster-devel] [PATCH V10 05/19] block: introduce bvec_last_segment()

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 03:23:56PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:52PM +0800, Ming Lei wrote: > > BTRFS and guard_bio_eod() need to get the last singlepage segment > > from one multipage bvec, so introduce this helper to make them happy. > > > > Cc: Dave Chinner > >

Re: [Cluster-devel] [PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 04:23:56PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:55PM +0800, Ming Lei wrote: > > BTRFS is the only user of this helper, so move this helper into > > BTRFS, and implement it via bio_for_each_segment_all(), since > > bio->bi_vcnt may not equal to number

Re: [Cluster-devel] [PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-19 Thread Christoph Hellwig
On Mon, Nov 19, 2018 at 04:19:24PM +0800, Ming Lei wrote: > On Fri, Nov 16, 2018 at 02:38:45PM +0100, Christoph Hellwig wrote: > > On Thu, Nov 15, 2018 at 04:52:55PM +0800, Ming Lei wrote: > > > BTRFS is the only user of this helper, so move this helper into > > > BTRFS, and implement it via

Re: [Cluster-devel] [PATCH V10 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 04:40:22PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:57PM +0800, Ming Lei wrote: > > iov_iter is implemented with bvec itererator, so it is safe to pass > > multipage bvec to it, and this way is much more efficient than > > passing one page in each bvec. >

Re: [Cluster-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:45:41PM +0100, Christoph Hellwig wrote: > On Thu, Nov 15, 2018 at 04:52:56PM +0800, Ming Lei wrote: > > There are still cases in which we need to use bio_bvecs() for get the > > number of multi-page segment, so introduce it. > > The only user in your final tree seems to

Re: [Cluster-devel] [PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:38:45PM +0100, Christoph Hellwig wrote: > On Thu, Nov 15, 2018 at 04:52:55PM +0800, Ming Lei wrote: > > BTRFS is the only user of this helper, so move this helper into > > BTRFS, and implement it via bio_for_each_segment_all(), since > > bio->bi_vcnt may not equal to

Re: [Cluster-devel] [PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 04:44:02PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:58PM +0800, Ming Lei wrote: > > bch_bio_alloc_pages() is always called on one new bio, so it is safe > > to access the bvec table directly. Given it is the only kind of this > > case, open code the bvec

Re: [Cluster-devel] [PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:46:45PM +0100, Christoph Hellwig wrote: > > - bio_for_each_segment_all(bv, bio, i) { > > + for (i = 0, bv = bio->bi_io_vec; i < bio->bi_vcnt; bv++) { > > This really needs a comment. Otherwise it looks fine to me. OK, will do it in next version. Thanks, Ming

Re: [Cluster-devel] [PATCH V10 13/19] iomap & xfs: only account for new added page

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 05:46:58PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:53:00PM +0800, Ming Lei wrote: > > After multi-page is enabled, one new page may be merged to a segment > > even though it is a new added page. > > > > This patch deals with this issue by post-check in

Re: [Cluster-devel] [PATCH V10 13/19] iomap & xfs: only account for new added page

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:49:36PM +0100, Christoph Hellwig wrote: > I'd much rather have __bio_try_merge_page only do merges in > the same page, and have a new __bio_try_merge_segment that does > multi-page merges. This will keep the accounting a lot simpler. Looks this way is clever, will do

Re: [Cluster-devel] [PATCH V10 12/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 01:42:52PM +0100, David Sterba wrote: > On Thu, Nov 15, 2018 at 04:52:59PM +0800, Ming Lei wrote: > > diff --git a/block/blk-zoned.c b/block/blk-zoned.c > > index 13ba2011a306..789b09ae402a 100644 > > --- a/block/blk-zoned.c > > +++ b/block/blk-zoned.c > > @@ -123,6 +123,7

Re: [Cluster-devel] [PATCH V10 12/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 05:22:45PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:59PM +0800, Ming Lei wrote: > > This patch introduces one extra iterator variable to > > bio_for_each_segment_all(), > > then we can allow bio_for_each_segment_all() to iterate over multi-page > >

Re: [Cluster-devel] [PATCH V10 14/19] block: enable multipage bvecs

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 05:56:27PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:53:01PM +0800, Ming Lei wrote: > > This patch pulls the trigger for multi-page bvecs. > > > > Now any request queue which supports queue cluster will see multi-page > > bvecs. > > > > Cc: Dave Chinner >

Re: [Cluster-devel] [PATCH V10 14/19] block: enable multipage bvecs

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:53:08PM +0100, Christoph Hellwig wrote: > > - > > - if (page == bv->bv_page && off == bv->bv_offset + bv->bv_len) { > > - bv->bv_len += len; > > - bio->bi_iter.bi_size += len; > > - return true; > > -

Re: [Cluster-devel] [PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 05:59:36PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:53:02PM +0800, Ming Lei wrote: > > Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to > > increase BIO_MAX_PAGES for it. > > You mentioned to it in the cover letter, but this needs more

Re: [Cluster-devel] [PATCH V10 17/19] block: don't use bio->bi_vcnt to figure out segment number

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 06:11:40PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:53:04PM +0800, Ming Lei wrote: > > It is wrong to use bio->bi_vcnt to figure out how many segments > > there are in the bio even though CLONED flag isn't set on this bio, > > because this bio may be

Re: [Cluster-devel] [PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-19 Thread Ming Lei
On Thu, Nov 15, 2018 at 06:18:11PM -0800, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:53:05PM +0800, Ming Lei wrote: > > Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after > > splitting"), > > physical segment number is mainly figured out in blk_queue_split() for > > fast

Re: [Cluster-devel] [PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-19 Thread Ming Lei
On Fri, Nov 16, 2018 at 02:58:03PM +0100, Christoph Hellwig wrote: > On Thu, Nov 15, 2018 at 04:53:05PM +0800, Ming Lei wrote: > > Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after > > splitting"), > > physical segment number is mainly figured out in blk_queue_split() for > > fast

[Cluster-devel] [PATCH 03/13] GFS2: Eliminate a goto in finish_xmote

2018-11-19 Thread Bob Peterson
This is another baby step toward a better glock state machine. This patch eliminates a goto in function finish_xmote so we can begin unraveling the cryptic logic with later patches. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 11 +-- 1 file changed, 5 insertions(+), 6 deletions(-)

[Cluster-devel] [PATCH 01/13] GFS2: Remove gotos from function run_queue

2018-11-19 Thread Bob Peterson
This patch removes all the gotos from function run_queue and inlines the code where necessary. This is a small step toward unravelling the logic to reorganize the glock state machine. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 38 ++ 1 file changed, 18

[Cluster-devel] [PATCH 02/13] GFS2: Make do_xmote determine its own gh parameter

2018-11-19 Thread Bob Peterson
This is another baby step toward a better glock state machine. Before this patch, do_xmote was called with a gh parameter, but only for promotes, not demotes. This patch allows do_xmote to determine the gh autonomously. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 12 ++-- 1 file

[Cluster-devel] [PATCH 12/13] GFS2: Add new GL_ST_UNLOCK state to reduce calls to the __ version

2018-11-19 Thread Bob Peterson
Before this patch, the truncate code called __state_machine but only did the unlock of GLF_UNLOCK. This patch adds a new state GL_ST_UNLOCK does the same thing, thus allowing it to call the regular state_machine function. This reduces the calls to the helper. Signed-off-by: Bob Peterson ---

Re: [Cluster-devel] [PATCH 03/13] GFS2: Eliminate a goto in finish_xmote

2018-11-19 Thread Steven Whitehouse
On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. This patch eliminates a goto in function finish_xmote so we can begin unraveling the cryptic logic with later patches. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 11 +-- 1

Re: [Cluster-devel] [PATCH 02/13] GFS2: Make do_xmote determine its own gh parameter

2018-11-19 Thread Steven Whitehouse
On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. Before this patch, do_xmote was called with a gh parameter, but only for promotes, not demotes. This patch allows do_xmote to determine the gh autonomously. Signed-off-by: Bob Peterson ---

[Cluster-devel] [PATCH 05/13] GFS2: Add do_xmote states to state machine

2018-11-19 Thread Bob Peterson
This patch adds two new states to the glock state machine. The first is a normal call for gl_target. The second is an abnormal call for cases in which dlm was unable to grant our request. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 37 - fs/gfs2/glock.h

[Cluster-devel] [PATCH 04/13] GFS2: Baby step toward a real state machine: finish_xmote

2018-11-19 Thread Bob Peterson
This patch adds a new function state_machine and some hooks to call it. For this early version, we've only got two states: idle and finish_xmote. Later, many more will be added. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 74 +++-

[Cluster-devel] [PATCH 00/13] Radical Reform of glock state machine (take 2)

2018-11-19 Thread Bob Peterson
This is a second go at my "Radical Reform of glock state machine" patch set. It's been rebased to the latest git tree, and had a lot more testing (from the actual for-next tree rather than testing the ports). This includes performance testing using iozone. The patches are unchanged from my

[Cluster-devel] [PATCH 06/13] GFS2: Make do_xmote not call the state machine again

2018-11-19 Thread Bob Peterson
Before this patch, the state machine could call do_xmote which, in turn, could call back into the state machine. This patch unravels the logic so instead it sends back an -EAGAIN return code, which signals the state machine to loop under the new state. Signed-off-by: Bob Peterson ---

[Cluster-devel] [PATCH 08/13] GFS2: Add a new GL_ST_PROMOTE state to glock state machine

2018-11-19 Thread Bob Peterson
Before this patch, function run_queue did a bunch of logic when glocks are being promoted. This patch moves the logic to the glock state machine and simplifies run_queue further. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 46 +++--- fs/gfs2/glock.h

[Cluster-devel] [PATCH 11/13] GFS2: Reduce glock_work_func to a single call to state_machine

2018-11-19 Thread Bob Peterson
Before this patch, function glock_work_func would call into the state machine for GL_FINISH_XMOTE, then GL_RUN, plus some work related to dropping references and requeueing itself. This patch moves all that functionality to a new GL_WORK state. This reduces glock_work_func to a single call to the

[Cluster-devel] [PATCH 07/13] GFS2: Add blocking and non-blocking demote to state machine

2018-11-19 Thread Bob Peterson
Before this patch, function run_queue would do special processing before calling the state machine for the blocking and non-blocking demote-in-progress cases. This function rolls those functions into the state machine, which will allow us to eventually simplify with later patches. Signed-off-by:

[Cluster-devel] [PATCH 10/13] GFS2: Reduce redundancy in GL_ST_DEMOTE_NONBLOCK state

2018-11-19 Thread Bob Peterson
This patch just simplifies the path through the GL_ST_DEMOTE_NONBLOCK state in the state machine. Regardless of whether a holder is found, it still needs to clear the GLF_LOCK bit. But we need to be careful to do it check for a holder before we release GLF_LOCK. Signed-off-by: Bob Peterson ---

[Cluster-devel] [PATCH 09/13] GFS2: Replace run_queue with new GL_ST_RUN state in state machine

2018-11-19 Thread Bob Peterson
Now we replace the run_queue function with a new state in the glock state machine which does the same thing. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 60 + fs/gfs2/glock.h | 1 + 2 files changed, 27 insertions(+), 34 deletions(-) diff

[Cluster-devel] [PATCH 13/13] GFS2: Optimization of GL_ST_UNLOCK state

2018-11-19 Thread Bob Peterson
The previous patch added a new GL_ST_UNLOCK state that simply cleared the GLF_LOCK bit, then went into the RUN state. Since the RUN state immediately sets the GLF_LOCK again and does a few other things, we can optimize it by putting GL_ST_UNLOCK in the middle of the GL_ST_RUN state. This avoids

Re: [Cluster-devel] [PATCH 02/13] GFS2: Make do_xmote determine its own gh parameter

2018-11-19 Thread Bob Peterson
Hi Steve, - Original Message - > > > On 19/11/18 13:29, Bob Peterson wrote: > > This is another baby step toward a better glock state machine. > > Before this patch, do_xmote was called with a gh parameter, but > > only for promotes, not demotes. This patch allows do_xmote to > >