Re: [dm-devel] [PATCH V6 4/5] blk-mq-sched: improve dispatching from sw queue

2017-10-11 Thread Omar Sandoval
This still didn't address Jens' concern about using q->queue_depth as the heuristic for whether to do the full sw queue flush or one-by-one dispatch. The EWMA approach is a bit too complex for now, can you please try the heuristic of whether the driver ever returned BLK_STS_RESOURCE? > Reviewed-by: Om

Re: [dm-devel] [PATCH V6 5/5] blk-mq-sched: don't dequeue request until all in ->dispatch are flushed

2017-10-11 Thread Omar Sandoval
ake sure that request isn't dequeued until ->dispatch is > flushed. > > Reviewed-by: Bart Van Assche <bart.vanass...@wdc.com> > Reviewed-by: Christoph Hellwig <h...@lst.de> I think this will do for now. Reviewed-by: Omar Sandoval <osan...@fb.com> > Signed-off

Re: [dm-devel] [PATCH V6 1/5] blk-mq-sched: fix scheduler bad performance

2017-10-11 Thread Omar Sandoval
) when mq-deadline/kyber > is used by not taking requests if hw queue is busy. > > Reviewed-by: Bart Van Assche <bart.vanass...@wdc.com> > Reviewed-by: Christoph Hellwig <h...@lst.de> Reviewed-by: Omar Sandoval <osan...@fb.com> > Signed-off-by: Ming Lei <m

Re: [dm-devel] [PATCH V6 3/5] sbitmap: introduce __sbitmap_for_each_set()

2017-10-11 Thread Omar Sandoval
On Mon, Oct 09, 2017 at 07:24:22PM +0800, Ming Lei wrote: > We need to iterate ctx starting from any ctx in round robin > way, so introduce this helper. > > Cc: Omar Sandoval <osan...@fb.com> Reviewed-by: Omar Sandoval <osan...@fb.com> > Signed-off-by: Mi

Re: [dm-devel] [PATCH V10 01/19] block: introduce multi-page page bvec helpers

2018-11-16 Thread Omar Sandoval
el.org > Cc: Darrick J. Wong > Cc: linux-...@vger.kernel.org > Cc: Gao Xiang > Cc: Christoph Hellwig > Cc: Theodore Ts'o > Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de

Re: [dm-devel] [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > First it is more efficient to use bio_for_each_bvec() in both > blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how > many multi-page bvecs there are in the bio. > > Secondly once bio_for_each_bvec() is used, the bvec

Re: [dm-devel] [PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:55PM +0800, Ming Lei wrote: > BTRFS is the only user of this helper, so move this helper into > BTRFS, and implement it via bio_for_each_segment_all(), since > bio->bi_vcnt may not equal to number of pages after multipage bvec > is enabled. Shouldn't you also get rid

Re: [dm-devel] [PATCH V10 17/19] block: don't use bio->bi_vcnt to figure out segment number

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:04PM +0800, Ming Lei wrote: > It is wrong to use bio->bi_vcnt to figure out how many segments > there are in the bio even though CLONED flag isn't set on this bio, > because this bio may be splitted or advanced. > > So always use bio_segments() in

Re: [dm-devel] [PATCH V10 14/19] block: enable multipage bvecs

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:01PM +0800, Ming Lei wrote: > This patch pulls the trigger for multi-page bvecs. > > Now any request queue which supports queue cluster will see multi-page > bvecs. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-devel@redhat.com > Cc:

Re: [dm-devel] [PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:05PM +0800, Ming Lei wrote: > Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), > physical segment number is mainly figured out in blk_queue_split() for > fast path, and the flag of BIO_SEG_VALID is set there too. > > Now only

Re: [dm-devel] [PATCH V10 07/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-16 Thread Omar Sandoval
rg > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > fs/btrfs/compression.c | 5 - > fs/btrfs/extent_io.c | 5 +++-- > 2 files

Re: [dm-devel] [PATCH V10 16/19] block: document usage of bio iterator helpers

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:03PM +0800, Ming Lei wrote: > Now multi-page bvec is supported, some helpers may return page by > page, meantime some may return segment by segment, this patch > documents the usage. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc:

Re: [dm-devel] [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-16 Thread Omar Sandoval
c: Christoph Hellwig > Cc: Theodore Ts'o > Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > incl

Re: [dm-devel] [PATCH V10 05/19] block: introduce bvec_last_segment()

2018-11-16 Thread Omar Sandoval
g > Cc: Gao Xiang > Cc: Christoph Hellwig > Cc: Theodore Ts'o > Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-by: Omar Sandoval Minor com

Re: [dm-devel] [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:05:10PM -0500, Mike Snitzer wrote: > On Thu, Nov 15 2018 at 3:20pm -0500, > Omar Sandoval wrote: > > > On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > > > First it is more efficient to use bio_for_each_bvec() in both >

Re: [dm-devel] [PATCH V10 02/19] block: introduce bio_for_each_bvec()

2018-11-16 Thread Omar Sandoval
c: Theodore Ts'o > Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-by: Omar Sandoval One comment below. > Signed-off-by: Ming Lei > --- > i

Re: [dm-devel] [PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:58PM +0800, Ming Lei wrote: > bch_bio_alloc_pages() is always called on one new bio, so it is safe > to access the bvec table directly. Given it is the only kind of this > case, open code the bvec table access since bio_for_each_segment_all() > will be changed to

Re: [dm-devel] [PATCH V10 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-16 Thread Omar Sandoval
Cc: linux-...@vger.kernel.org > Cc: Gao Xiang > Cc: Christoph Hellwig > Cc: Theodore Ts'o > Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-by: Omar Sandoval

Re: [dm-devel] [PATCH V10 13/19] iomap & xfs: only account for new added page

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:00PM +0800, Ming Lei wrote: > After multi-page is enabled, one new page may be merged to a segment > even though it is a new added page. > > This patch deals with this issue by post-check in case of merge, and > only a freshly new added page need to be dealt with for

Re: [dm-devel] [PATCH V10 04/19] block: use bio_for_each_bvec() to map sg

2018-11-16 Thread Omar Sandoval
..@vger.kernel.org > Cc: Gao Xiang > Cc: Christoph Hellwig > Cc: Theodore Ts'o > Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-

Re: [dm-devel] [PATCH V10 06/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-16 Thread Omar Sandoval
Gao Xiang > Cc: Christoph Hellwig > Cc: Theodore Ts'o > Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei

Re: [dm-devel] [PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-16 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:02PM +0800, Ming Lei wrote: > Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to > increase BIO_MAX_PAGES for it. You mentioned to it in the cover letter, but this needs more explanation in the commit message. Why did CONFIG_THP_SWAP require > 256?

Re: [dm-devel] [PATCH V10 19/19] block: kill BLK_MQ_F_SG_MERGE

2018-11-21 Thread Omar Sandoval
Cc: linux-e...@vger.kernel.org > Cc: Coly Li > Cc: linux-bca...@vger.kernel.org > Cc: Boaz Harrosh > Cc: Bob Peterson > Cc: cluster-de...@redhat.com Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > block/blk-mq-debugfs.c | 1 - > drivers/block

Re: [dm-devel] [PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-21 Thread Omar Sandoval
On Fri, Nov 16, 2018 at 02:59:22PM +0100, Christoph Hellwig wrote: > On Thu, Nov 15, 2018 at 06:18:11PM -0800, Omar Sandoval wrote: > > This commit message wasn't very clear. Is it the case that > > QUEUE_FLAG_NO_SG_MERGE is no longer set by any drivers? > > I think he wants

Re: [dm-devel] [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-21 Thread Omar Sandoval
On Fri, Nov 16, 2018 at 10:19:56AM +0100, Christoph Hellwig wrote: > On Thu, Nov 15, 2018 at 02:18:47PM -0800, Omar Sandoval wrote: > > My only reason to prefer unsigned int is consistency. unsigned int is > > much more common in the kernel: > > > > $ ag --cc

Re: [dm-devel] [PATCH V12 18/20] block: document usage of bio iterator helpers

2018-11-27 Thread Omar Sandoval
On Mon, Nov 26, 2018 at 10:17:18AM +0800, Ming Lei wrote: > Now multi-page bvec is supported, some helpers may return page by > page, meantime some may return segment by segment, this patch > documents the usage. Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > D

Re: [dm-devel] [PATCH V12 16/20] block: enable multipage bvecs

2018-11-27 Thread Omar Sandoval
On Mon, Nov 26, 2018 at 10:17:16AM +0800, Ming Lei wrote: > This patch pulls the trigger for multi-page bvecs. Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > block/bio.c | 22 +++--- > fs/iomap.c | 4 ++-- > fs/xfs

Re: [dm-devel] [PATCH V12 09/20] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-27 Thread Omar Sandoval
it need to be considered under > this situation too. Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > block/blk-merge.c | 100 > +++--- > 1 file changed, 80 insertions(+), 20 deletions(-) > > diff --git a/bloc

Re: [dm-devel] [PATCH V12 17/20] block: always define BIO_MAX_PAGES as 256

2018-11-27 Thread Omar Sandoval
takes one bvec to > hold them all. > > Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > include/linux/bio.h | 8 > 1 file changed, 8 deletions(-) > > diff --git a/include/linux/bio.h b/include/linux/bio.h

Re: [dm-devel] [PATCH V12 04/20] block: don't use bio->bi_vcnt to figure out segment number

2018-11-27 Thread Omar Sandoval
s use bio_segments() in blk_recount_segments(), and it shouldn't > cause any performance loss now because the physical segment number is figured > out in blk_queue_split() and BIO_SEG_VALID is set meantime since > bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"). > > Reviewe

Re: [dm-devel] [PATCH V12 06/20] block: rename bvec helpers

2018-11-27 Thread Omar Sandoval
ly deal with single-page > bvec, or called segment. We will introduce helpers named with 'bvec' > for multi-page bvec. > > bvec_iter_advance() isn't renamed becasue this helper is always operated > on real bvec even though multi-page bvec is supported. > > Suggested-by:

Re: [dm-devel] [PATCH V12 13/20] block: loop: pass multi-page bvec to iov_iter

2018-11-27 Thread Omar Sandoval
On Mon, Nov 26, 2018 at 10:17:13AM +0800, Ming Lei wrote: > iov_iter is implemented on bvec itererator helpers, so it is safe to pass > multi-page bvec to it, and this way is much more efficient than passing one > page in each bvec. > > Reviewed-by: Christoph Hellwig Reviewed-by

Re: [dm-devel] [PATCH V12 05/20] block: remove bvec_iter_rewind()

2018-11-27 Thread Omar Sandoval
On Mon, Nov 26, 2018 at 10:17:05AM +0800, Ming Lei wrote: > Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes > bio_rewind_iter(), then no one uses bvec_iter_rewind() any more, > so remove it. Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --

Re: [dm-devel] [PATCH V12 01/20] btrfs: remove various bio_offset arguments

2018-11-27 Thread Omar Sandoval
On Mon, Nov 26, 2018 at 10:17:01AM +0800, Ming Lei wrote: > From: Christoph Hellwig > > The btrfs write path passes a bio_offset argument through some deep > callchains including async offloading. In the end this is easily > calculatable using page_offset plus the bvec offset for the first >

Re: [dm-devel] [PATCH V12 15/20] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-27 Thread Omar Sandoval
imple change on all > bio_for_each_segment_all() > users, this patch does tree-wide change in one single patch, so that we can > avoid to use a temporary helper for this conversion. > > Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- &

Re: [dm-devel] [PATCH V12 14/20] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-27 Thread Omar Sandoval
l be changed to support for iterating over multipage bvec. > > Acked-by: Coly Li > Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval > Signed-off-by: Ming Lei > --- > drivers/md/bcache/util.c | 6 +- > 1 file changed, 5 insertions(+), 1 deletion(-) > >

Re: [dm-devel] [PATCH V12 02/20] btrfs: look at bi_size for repair decisions

2018-11-27 Thread Omar Sandoval
re is more than a > single sector worth retrying. Use bi_size for that and shift by > PAGE_SHIFT. This really should be blocks/sectors, but given that > btrfs doesn't support a sector size different from the PAGE_SIZE > using the page size keeps the changes to a minimum. > > Reviewed-by:

Re: [dm-devel] [PATCH V12 03/20] block: remove the "cluster" flag

2018-11-27 Thread Omar Sandoval
ect to the LLDD is that > it limits each segments to be inside a single page, which we can > also affect by setting the maximum segment size and the segment > boundary. Reviewed-by: Omar Sandoval One comment typo below. > Signed-off-by: Christoph Hellwig > > Replace virt bou