Re: [PATCH blktests 2/2] block/011: Perform PCI reset while doing IO

2017-06-26 Thread Johannes Thumshirn
On Mon, Jun 26, 2017 at 02:27:24PM -0700, Omar Sandoval wrote: [...] > > + > > + while kill -0 $! 2>/dev/null; do > > + echo 0 > "/sys/bus/pci/devices/${pdev}/enable" > > + sleep .2 > > + echo 1 > "/sys/bus/pci/devices/${pdev}/enable" > > Test looks good, but one qu

Re: [PATCH v2 16/51] block: bounce: avoid direct access to bvec table

2017-06-26 Thread Matthew Wilcox
On Mon, Jun 26, 2017 at 08:09:59PM +0800, Ming Lei wrote: > bio_for_each_segment_all(bvec, bio, i) { > - org_vec = bio_orig->bi_io_vec + i + start; > - > - if (bvec->bv_page == org_vec->bv_page) > - continue; > + orig_vec = bio_iter_iove

Re: [PATCH BUGFIX V2] block, bfq: update wr_busy_queues if needed on a queue split

2017-06-26 Thread Paolo Valente
> Il giorno 19 giu 2017, alle ore 13:43, Paolo Valente > ha scritto: > > This commit fixes a bug triggered by a non-trivial sequence of > events. These events are briefly described in the next two > paragraphs. The impatiens, or those who are familiar with queue > merging and splitting, can jum

Re: [block] 044f1daaaa kernel BUG at block/blk-mq.c:415!

2017-06-26 Thread Jens Axboe
On 06/26/2017 09:53 PM, Fengguang Wu wrote: > Greetings, > > Sorry the report is a bit late, this bug looks still active: > First I've heard about it! > [ 36.037364] sector 2, nr/cnr 0/2 > [ 36.037367] bio 88001af84e80, biotail 88001af84e80, len 0 > [ 36.037367] bio 880

[PATCH blktests v2] loop/002: Regression testing for loop device flush

2017-06-26 Thread James Wang
Add a regression testing for loop device. when an unbound device be close that take too long time. kernel will consume serveral orders of magnitude more wall time than it does for a mounted device. Signed-off-by: James Wang --- tests/loop/002 | 63

Re: [PATCH blktests] loop/002: Regression testing for loop device flush

2017-06-26 Thread James Wang
On 06/27/2017 02:58 AM, Omar Sandoval wrote: > Hi, James, thanks for sending this in. Sorry for the delay, I've been > out of the office for a couple of weeks. A few comments below. > > On Thu, Jun 08, 2017 at 08:28:12PM +0800, James Wang wrote: >> Add a regression testing for loop device. when a

Re: [PATCH 00/20] LightNVM: pblk patches for 4.13

2017-06-26 Thread Jens Axboe
On Mon, Jun 26 2017, Javier González wrote: > Hi Matias, > > Here you have the pblk patchset for this window. > > Apart from small fixes for LightNVM core and pblk, there are three > relevant changes: > > - Metadata for each line is no longer issued on a separate workqueue, >but instead, al

Re: [PATCH blktests 2/2] block/011: Perform PCI reset while doing IO

2017-06-26 Thread Jens Axboe
On 06/26/2017 03:29 PM, Omar Sandoval wrote: > On Mon, Jun 26, 2017 at 08:25:36AM -0600, Jens Axboe wrote: >> On 06/26/2017 08:06 AM, Johannes Thumshirn wrote: >>> On Fri, Jun 23, 2017 at 09:36:14AM -0600, Jens Axboe wrote: On 06/23/2017 08:29 AM, Johannes Thumshirn wrote: > From: Omar San

Re: [PATCH blktests 1/2] rc: add helpers to handle PCI test devices

2017-06-26 Thread Omar Sandoval
On Fri, Jun 23, 2017 at 04:29:50PM +0200, Johannes Thumshirn wrote: > Add two helpers to check whether a device is attached via PCI and to get the > PCI device from a TEST_DEV > > Signed-off-by: Johannes Thumshirn > --- > common/rc | 15 +++ > 1 file changed, 15 insertions(+) > > di

Re: [PATCH blktests 2/2] block/011: Perform PCI reset while doing IO

2017-06-26 Thread Omar Sandoval
On Mon, Jun 26, 2017 at 08:25:36AM -0600, Jens Axboe wrote: > On 06/26/2017 08:06 AM, Johannes Thumshirn wrote: > > On Fri, Jun 23, 2017 at 09:36:14AM -0600, Jens Axboe wrote: > >> On 06/23/2017 08:29 AM, Johannes Thumshirn wrote: > >>> From: Omar Sandoval > >>> > >>> This test-case performs I/O w

Re: [PATCH blktests 2/2] block/011: Perform PCI reset while doing IO

2017-06-26 Thread Omar Sandoval
On Fri, Jun 23, 2017 at 04:29:51PM +0200, Johannes Thumshirn wrote: > From: Omar Sandoval > > This test-case performs I/O with fio while doing PCI disable/enable > cycles. > > In the results we don't care for I/O errors but for hiccups in dmesg only. > > Signed-off-by: Johannes Thumshirn > ---

Re: [GIT PULL] New test and a fixup for blktests

2017-06-26 Thread Jens Axboe
On 06/26/2017 03:04 PM, Omar Sandoval wrote: > On Mon, Jun 26, 2017 at 03:02:55PM -0600, Jens Axboe wrote: >> On 06/26/2017 02:45 PM, Omar Sandoval wrote: >>> On Wed, Jun 21, 2017 at 09:22:47AM -0600, Jens Axboe wrote: Hi Omar, Two changes here: - block/006 wants to use the

Re: [GIT PULL] New test and a fixup for blktests

2017-06-26 Thread Omar Sandoval
On Mon, Jun 26, 2017 at 03:02:55PM -0600, Jens Axboe wrote: > On 06/26/2017 02:45 PM, Omar Sandoval wrote: > > On Wed, Jun 21, 2017 at 09:22:47AM -0600, Jens Axboe wrote: > >> Hi Omar, > >> > >> Two changes here: > >> > >> - block/006 wants to use the iops results, but it isn't using _fio_perf. > >

Re: [GIT PULL] New test and a fixup for blktests

2017-06-26 Thread Jens Axboe
On 06/26/2017 02:45 PM, Omar Sandoval wrote: > On Wed, Jun 21, 2017 at 09:22:47AM -0600, Jens Axboe wrote: >> Hi Omar, >> >> Two changes here: >> >> - block/006 wants to use the iops results, but it isn't using _fio_perf. >> - Add block/010 to test for shared vs non-shared tags performance. >> >> >

Re: [PATCH] sd: add support for TCG OPAL self encrypting disks

2017-06-26 Thread Tejun Heo
On Mon, Jun 26, 2017 at 12:43:27PM -0400, Martin K. Petersen wrote: > > Christoph, > > > ping? > > Looks good to me. I'll queue it up for 4.13 as soon as Linus has pulled > in the ata bits. I can route it through libata tree w/ your ack if that's more convenient. Thanks. -- tejun

Re: [GIT PULL] New test and a fixup for blktests

2017-06-26 Thread Omar Sandoval
On Wed, Jun 21, 2017 at 09:22:47AM -0600, Jens Axboe wrote: > Hi Omar, > > Two changes here: > > - block/006 wants to use the iops results, but it isn't using _fio_perf. > - Add block/010 to test for shared vs non-shared tags performance. > > > https://github.com/axboe/blktests.git axboe > >

Re: [PATCH blktests] sg/001: don't require scsi_debug

2017-06-26 Thread Omar Sandoval
On Fri, Jun 23, 2017 at 10:14:48AM +0200, Johannes Thumshirn wrote: > Don't require scsi_debug but check if TEST_DEV is a SCSI device and use it > instead. > > Signed-off-by: Johannes Thumshirn > --- > tests/sg/001 | 8 +++- > 1 file changed, 3 insertions(+), 5 deletions(-) > > diff --git a

Re: [PATCH 9/9] nvme: add support for streams and directives

2017-06-26 Thread Jens Axboe
On 06/26/2017 01:36 PM, Andreas Dilger wrote: > On Jun 26, 2017, at 7:56 AM, Jens Axboe wrote: >> >> On 06/26/2017 03:59 AM, Christoph Hellwig wrote: >>> Looks mostly good, >>> >>> but two nit-picks: >>> >>> - can we keep a module option to disable streams, or in fact for >>> now maybe to explic

Re: [PATCH 9/9] nvme: add support for streams and directives

2017-06-26 Thread Andreas Dilger
On Jun 26, 2017, at 7:56 AM, Jens Axboe wrote: > > On 06/26/2017 03:59 AM, Christoph Hellwig wrote: >> Looks mostly good, >> >> but two nit-picks: >> >> - can we keep a module option to disable streams, or in fact for >> now maybe to explicitly enable it? I expect this to be interesting >>

Re: [PATCH blktests] loop/002: Regression testing for loop device flush

2017-06-26 Thread Omar Sandoval
Hi, James, thanks for sending this in. Sorry for the delay, I've been out of the office for a couple of weeks. A few comments below. On Thu, Jun 08, 2017 at 08:28:12PM +0800, James Wang wrote: > Add a regression testing for loop device. when an unbound device > be close that take too long time. ke

Re: [PATCH v7 21/22] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-26 Thread Darrick J. Wong
On Mon, Jun 26, 2017 at 01:58:32PM -0400, jlay...@redhat.com wrote: > On Mon, 2017-06-26 at 08:22 -0700, Darrick J. Wong wrote: > > On Fri, Jun 16, 2017 at 03:34:26PM -0400, Jeff Layton wrote: > > > Just check and advance the data errseq_t in struct file before > > > before returning from fsync on

Re: [PATCH v2 14/51] btrfs: avoid to access bvec table directly for a cloned bio

2017-06-26 Thread Liu Bo
On Mon, Jun 26, 2017 at 08:09:57PM +0800, Ming Lei wrote: > Commit 17347cec15f919901c90(Btrfs: change how we iterate bios in endio) > mentioned that for dio the submitted bio may be fast cloned, we > can't access the bvec table directly for a cloned bio, so use > bio_get_first_bvec() to retrieve th

Re: [PATCH 9/9] nvme: add support for streams and directives

2017-06-26 Thread Jens Axboe
On 06/26/2017 11:52 AM, Martin K. Petersen wrote: > > Christoph, > >> - can we keep a module option to disable streams, or in fact for >>now maybe to explicitly enable it? I expect this to be interesting >>at least for the first devices that implement it. Also given that >>it needs

Re: [PATCH v7 21/22] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-26 Thread jlayton
On Mon, 2017-06-26 at 08:22 -0700, Darrick J. Wong wrote: > On Fri, Jun 16, 2017 at 03:34:26PM -0400, Jeff Layton wrote: > > Just check and advance the data errseq_t in struct file before > > before returning from fsync on normal files. Internal filemap_* > > callers are left as-is. > > > > Signed

Re: [PATCH 9/9] nvme: add support for streams and directives

2017-06-26 Thread Martin K. Petersen
Christoph, > - can we keep a module option to disable streams, or in fact for >now maybe to explicitly enable it? I expect this to be interesting >at least for the first devices that implement it. Also given that >it needs to be explicitly enabled I would expect some overhead of >

Re: [PATCH v2 00/51] block: support multipage bvec

2017-06-26 Thread Jens Axboe
On 06/26/2017 06:09 AM, Ming Lei wrote: > Hi, > > This patchset brings multipage bvec into block layer: > > 1) what is multipage bvec? > > Multipage bvecs means that one 'struct bio_bvec' can hold > multiple pages which are physically contiguous instead > of one single page used in linux kernel

Re: [PATCH v2 00/51] block: support multipage bvec

2017-06-26 Thread David Sterba
On Mon, Jun 26, 2017 at 08:09:43PM +0800, Ming Lei wrote: > btrfs: avoid access to .bi_vcnt directly > btrfs: avoid to access bvec table directly for a cloned bio > btrfs: comment on direct access bvec table > btrfs: use bvec_get_last_page to get bio's last page > fs/btrfs: convert to bio

Re: [PATCH] sd: add support for TCG OPAL self encrypting disks

2017-06-26 Thread Martin K. Petersen
Christoph, > ping? Looks good to me. I'll queue it up for 4.13 as soon as Linus has pulled in the ata bits. -- Martin K. Petersen Oracle Linux Engineering

Re: [PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

2017-06-26 Thread Jens Axboe
On 06/26/2017 10:09 AM, Darrick J. Wong wrote: > On Mon, Jun 26, 2017 at 07:55:27AM -0600, Jens Axboe wrote: >> On 06/26/2017 03:51 AM, Christoph Hellwig wrote: >>> Please document the userspace API (added linux-api and linux-man >>> to CC for sugestions), especially including the odd effects of th

Re: [PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

2017-06-26 Thread Darrick J. Wong
On Mon, Jun 26, 2017 at 07:55:27AM -0600, Jens Axboe wrote: > On 06/26/2017 03:51 AM, Christoph Hellwig wrote: > > Please document the userspace API (added linux-api and linux-man > > to CC for sugestions), especially including the odd effects of the > > per-inode settings. > > Of course, I'll sen

[PATCH 7/9] xfs: add support for passing in write hints for buffered writes

2017-06-26 Thread Jens Axboe
Reviewed-by: Andreas Dilger Reviewed-by: Martin K. Petersen Reviewed-by: Darrick J. Wong Signed-off-by: Jens Axboe --- fs/xfs/xfs_aops.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 76b6f988e2fa..ceb124bd8f80 100644 --- a/fs/xfs/xfs_aops.c +

[PATCH 2/9] block: add support for write hints in a bio

2017-06-26 Thread Jens Axboe
No functional changes in this patch, we just set aside 3 bits in the bio/request flags, which can be used to hold a WRITE_LIFE_* life time hint. Ensure that we don't merge requests that have different life time hints assigned to them. Reviewed-by: Martin K. Petersen Reviewed-by: Christoph Hellwi

[PATCH 9/9] nvme: add support for streams and directives

2017-06-26 Thread Jens Axboe
This adds support for Directives in NVMe, particular for the Streams directive. Support for Directives is a new feature in NVMe 1.3. It allows a user to pass in information about where to store the data, so that it the device can do so most effiently. If an application is managing and writing data

[PATCH 8/9] btrfs: add support for passing in write hints for buffered writes

2017-06-26 Thread Jens Axboe
Reviewed-by: Andreas Dilger Signed-off-by: Chris Mason Reviewed-by: Martin K. Petersen Signed-off-by: Jens Axboe --- fs/btrfs/extent_io.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 19eedf2e630b..fde09c6005fc 100644 --- a/fs/btrfs/exten

[PATCH 5/9] fs: add support for buffered writeback to pass down write hints

2017-06-26 Thread Jens Axboe
Reviewed-by: Andreas Dilger Reviewed-by: Martin K. Petersen Signed-off-by: Jens Axboe --- fs/buffer.c | 14 +- fs/mpage.c | 1 + 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 306b720f7383..307b508c9d60 100644 --- a/fs/buffer.c +++

[PATCH 4/9] fs: add O_DIRECT support for sending down write life time hints

2017-06-26 Thread Jens Axboe
Reviewed-by: Andreas Dilger Reviewed-by: Martin K. Petersen Signed-off-by: Jens Axboe --- fs/block_dev.c | 2 ++ fs/direct-io.c | 2 ++ fs/iomap.c | 5 - 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index dd91c99e9ba0..30e1fb65c2fa 100644

[PATCHSET v10] Add support for write life time hints

2017-06-26 Thread Jens Axboe
A new iteration of this patchset, previously known as write streams. As before, this patchset aims at enabling applications split up writes into separate streams, based on the perceived life time of the data written. This is useful for a variety of reasons: - For NVMe, this feature is ratified and

[PATCH 6/9] ext4: add support for passing in write hints for buffered writes

2017-06-26 Thread Jens Axboe
Reviewed-by: Andreas Dilger Reviewed-by: Martin K. Petersen Signed-off-by: Jens Axboe --- fs/ext4/page-io.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index 930ca0fc9a0f..02e5a7b8d60b 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -350

[PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

2017-06-26 Thread Jens Axboe
Define a set of write life time hints: RWH_WRITE_LIFE_NOT_SET No hint information set RWH_WRITE_LIFE_NONE No hints about write life time RWH_WRITE_LIFE_SHORTData written has a short life time RWH_WRITE_LIFE_MEDIUM Data written has a medium life time RWH_WRITE_LIFE_LONG Data written

[PATCH 3/9] blk-mq: expose write hints through debugfs

2017-06-26 Thread Jens Axboe
Useful to verify that things are working the way they should. Reading the file will return number of kb written with each write hint. Writing the file will reset the statistics. No care is taken to ensure that we don't race on updates. Drivers will write to q->write_hints[] if they handle a given

Re: [PATCH v7 21/22] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-26 Thread Darrick J. Wong
On Fri, Jun 16, 2017 at 03:34:26PM -0400, Jeff Layton wrote: > Just check and advance the data errseq_t in struct file before > before returning from fsync on normal files. Internal filemap_* > callers are left as-is. > > Signed-off-by: Jeff Layton > --- > fs/xfs/xfs_file.c | 15 +++

Re: [PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

2017-06-26 Thread Jeff Layton
On Sat, 2017-06-24 at 09:16 -0400, Jeff Layton wrote: > On Sat, 2017-06-24 at 04:59 -0700, Christoph Hellwig wrote: > > On Tue, Jun 20, 2017 at 01:44:44PM -0400, Jeff Layton wrote: > > > In order to query for errors with errseq_t, you need a previously- > > > sampled point from which to check. When

Re: [PATCH blktests 2/2] block/011: Perform PCI reset while doing IO

2017-06-26 Thread Jens Axboe
On 06/26/2017 08:06 AM, Johannes Thumshirn wrote: > On Fri, Jun 23, 2017 at 09:36:14AM -0600, Jens Axboe wrote: >> On 06/23/2017 08:29 AM, Johannes Thumshirn wrote: >>> From: Omar Sandoval >>> >>> This test-case performs I/O with fio while doing PCI disable/enable >>> cycles. >>> >>> In the result

Re: [PATCH 7/9] xfs: add support for passing in write hints for buffered writes

2017-06-26 Thread Jens Axboe
On 06/26/2017 03:56 AM, Christoph Hellwig wrote: >> +ioend->io_bio->bi_opf |= >> write_hint_to_opf(inode_write_hint(ioend->io_inode)); > > Too long line again. > > I think simply adding an inode_hint_to_opf() flag would make the various > opencoded versions of the above call a little easier

Re: [PATCH 4/9] fs: add O_DIRECT support for sending down write life time hints

2017-06-26 Thread Jens Axboe
On 06/26/2017 03:55 AM, Christoph Hellwig wrote: >> @@ -385,6 +385,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio, >> else >> bio->bi_end_io = dio_bio_end_io; >> >> +bio->bi_opf |= write_hint_to_opf(iocb_write_hint(dio->iocb)); >> + >> sdio->bio = bio; >>

Re: [PATCH blktests 2/2] block/011: Perform PCI reset while doing IO

2017-06-26 Thread Johannes Thumshirn
On Fri, Jun 23, 2017 at 09:36:14AM -0600, Jens Axboe wrote: > On 06/23/2017 08:29 AM, Johannes Thumshirn wrote: > > From: Omar Sandoval > > > > This test-case performs I/O with fio while doing PCI disable/enable > > cycles. > > > > In the results we don't care for I/O errors but for hiccups in d

Re: [PATCH 9/9] nvme: add support for streams and directives

2017-06-26 Thread Jens Axboe
On 06/26/2017 03:59 AM, Christoph Hellwig wrote: > Looks mostly good, > > but two nit-picks: > > - can we keep a module option to disable streams, or in fact for >now maybe to explicitly enable it? I expect this to be interesting >at least for the first devices that implement it. Also

Re: [PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

2017-06-26 Thread Jens Axboe
On 06/26/2017 03:51 AM, Christoph Hellwig wrote: > Please document the userspace API (added linux-api and linux-man > to CC for sugestions), especially including the odd effects of the > per-inode settings. Of course, I'll send in a diff for the fcntl(2) man page. > Also I would highly recommend

Re: [PATCH v7 21/22] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-26 Thread Carlos Maiolino
On Fri, Jun 16, 2017 at 03:34:26PM -0400, Jeff Layton wrote: > Just check and advance the data errseq_t in struct file before > before returning from fsync on normal files. Internal filemap_* > callers are left as-is. > Looks good. Reviewed-by: Carlos Maiolino > Signed-off-by: Jeff Layton > -

[PATCH v2 17/51] bvec_iter: introduce BVEC_ITER_ALL_INIT

2017-06-26 Thread Ming Lei
Introduce BVEC_ITER_ALL_INIT for iterating one bio from start to end. Signed-off-by: Ming Lei --- include/linux/bvec.h | 9 + 1 file changed, 9 insertions(+) diff --git a/include/linux/bvec.h b/include/linux/bvec.h index 89b65b82d98f..162ca7caf510 100644 --- a/include/linux/bvec.h +++ b

[PATCH v2 11/51] md: raid1: initialize bvec table via bio_add_page()

2017-06-26 Thread Ming Lei
We will support multipage bvec soon, so initialize bvec table using the standardy way instead of writing the talbe directly. Otherwise it won't work any more once multipage bvec is enabled. Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Signed-off-by: Ming Lei --- drivers/md/raid1.c | 27 +++

[PATCH v2 15/51] btrfs: comment on direct access bvec table

2017-06-26 Thread Ming Lei
Cc: Chris Mason Cc: Josef Bacik Cc: David Sterba Cc: linux-bt...@vger.kernel.org Signed-off-by: Ming Lei --- fs/btrfs/compression.c | 4 fs/btrfs/inode.c | 12 2 files changed, 16 insertions(+) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 2c0b7b5

[PATCH v2 12/51] md: raid10: avoid to access bvec table directly

2017-06-26 Thread Ming Lei
Inside sync_request_write(), .bi_vcnt is written after this bio is reseted, this way won't work any more after multipage bvec is enabled. So reset_bvec_table() is introduced for re-add these pages into bio, then .bi_vcnt needn't to be touched any more. Cc: Shaohua Li Cc: linux-r...@vger.kernel.o

[PATCH v2 13/51] btrfs: avoid access to .bi_vcnt directly

2017-06-26 Thread Ming Lei
BTRFS uses bio->bi_vcnt to figure out page numbers, this way becomes not correct once we start to enable multipage bvec. So use bio_for_each_segment_all() to do that instead. Cc: Chris Mason Cc: Josef Bacik Cc: David Sterba Cc: linux-bt...@vger.kernel.org Signed-off-by: Ming Lei --- fs/btrfs

[PATCH v2 14/51] btrfs: avoid to access bvec table directly for a cloned bio

2017-06-26 Thread Ming Lei
Commit 17347cec15f919901c90(Btrfs: change how we iterate bios in endio) mentioned that for dio the submitted bio may be fast cloned, we can't access the bvec table directly for a cloned bio, so use bio_get_first_bvec() to retrieve the 1st bvec. Cc: Chris Mason Cc: Josef Bacik Cc: David Sterba C

[PATCH v2 19/51] block: comments on bio_for_each_segment[_all]

2017-06-26 Thread Ming Lei
This patch clarifies the fact that even though both bio_for_each_segment() and bio_for_each_segment_all() are named as _segment/_segment_all, they still return one page in each vector, instead of real segment(multipage bvec). With comming multipage bvec, both the two helpers are capable of returni

[PATCH v2 18/51] block: bounce: don't access bio->bi_io_vec in copy_to_high_bio_irq

2017-06-26 Thread Ming Lei
As we need to support multipage bvecs, so don't access bio->bi_io_vec in copy_to_high_bio_irq(), and just use the standard iterator to do that. Signed-off-by: Ming Lei --- block/bounce.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/block/bounce.c b/block/

[PATCH v2 20/51] block: introduce multipage/single page bvec helpers

2017-06-26 Thread Ming Lei
This patch introduces helpers which are suffixed with _mp and _sp for the multipage bvec/segment support. The helpers with _mp suffix are the interfaces for treating one bvec/segment as real multipage one, for example, .bv_len is the total length of the multipage segment. The helpers with _sp suf

[PATCH v2 21/51] block: implement sp version of bvec iterator helpers

2017-06-26 Thread Ming Lei
This patch implements singlepage version of the following 3 helpers: - bvec_iter_offset_sp() - bvec_iter_len_sp() - bvec_iter_page_sp() So that one multipage bvec can be splited to singlepage bvec, and make users of current bvec iterator happy. Signed-off-by: Ming Lei ---

[PATCH v2 22/51] block: introduce bio_for_each_segment_mp()

2017-06-26 Thread Ming Lei
This helper is used to iterate multipage bvec and it is required in bio_clone(). Signed-off-by: Ming Lei --- include/linux/bio.h | 39 ++- include/linux/bvec.h | 37 - 2 files changed, 66 insertions(+), 10 deletions(-) dif

[PATCH v2 23/51] blk-merge: compute bio->bi_seg_front_size efficiently

2017-06-26 Thread Ming Lei
It is enough to check and compute bio->bi_seg_front_size just after the 1st segment is found, but current code checks that for each bvec, which is inefficient. This patch follows the way in __blk_recalc_rq_segments() for computing bio->bi_seg_front_size, and it is more efficient and code becomes

[PATCH v2 24/51] block: blk-merge: try to make front segments in full size

2017-06-26 Thread Ming Lei
When merging one bvec into segment, if the bvec is too big to merge, current policy is to move the whole bvec into another new segment. This patchset changes the policy into trying to maximize size of front segments, that means in above situation, part of bvec is merged into current segment, and t

[PATCH v2 26/51] block: use bio_for_each_segment_mp() to compute segments count

2017-06-26 Thread Ming Lei
Firstly it is more efficient to use bio_for_each_segment_mp() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many segments there are in the bio. Secondaly once bio_for_each_segment_mp() is used, the bvec may need to be splitted because its length can be very long and

[PATCH v2 32/51] btrfs: use bvec_get_last_page to get bio's last page

2017-06-26 Thread Ming Lei
Preparing for supporting multipage bvec. Cc: Chris Mason Cc: Josef Bacik Cc: David Sterba Cc: linux-bt...@vger.kernel.org Signed-off-by: Ming Lei --- fs/btrfs/compression.c | 5 - fs/btrfs/extent_io.c | 8 ++-- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/fs/btrfs

[PATCH v2 33/51] block: deal with dirtying pages for multipage bvec

2017-06-26 Thread Ming Lei
In bio_check_pages_dirty(), bvec->bv_page is used as flag for marking if the page has been dirtied & released, and if no, it will be dirtied in deferred workqueue. With multipage bvec, we can't do that any more, so change the logic into checking all pages in one mp bvec, and only release all these

[PATCH v2 36/51] md: raid1: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Signed-off-by: Ming Lei --- drivers/md/raid1.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 835c42396861..ca4b9ff8d39b 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@

[PATCH v2 37/51] dm-crypt: don't clear bvec->bv_page in crypt_free_buffer_pages()

2017-06-26 Thread Ming Lei
The bio is always freed after running crypt_free_buffer_pages(), so it isn't necessary to clear the bv->bv_page. Cc: Mike Snitzer Cc:dm-de...@redhat.com Signed-off-by: Ming Lei --- drivers/md/dm-crypt.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-cry

[PATCH v2 44/51] gfs2: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: Steven Whitehouse Cc: Bob Peterson Cc: cluster-de...@redhat.com Signed-off-by: Ming Lei --- fs/gfs2/lops.c| 3 ++- fs/gfs2/meta_io.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c index d62939f00d53..294f1926d9be 100644 --- a/fs/g

[PATCH v2 46/51] exofs: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: Boaz Harrosh Signed-off-by: Ming Lei --- fs/exofs/ore.c | 3 ++- fs/exofs/ore_raid.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/exofs/ore.c b/fs/exofs/ore.c index 8bb72807e70d..38a7d8bfdd4c 100644 --- a/fs/exofs/ore.c +++ b/fs/exofs/ore.c @@ -406,8 +406,9

[PATCH v2 41/51] fs/iomap: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- fs/iomap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/iomap.c b/fs/iomap.c index c71a64b97fba..4319284c1fbd 100644 --- a/fs/iomap.c +++ b/fs/iomap.c @@ -696,8 +696,9 @@ static void iomap_dio_bio_end_io(struct bio *bio) } else {

[PATCH v2 45/51] f2fs: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: Jaegeuk Kim Cc: Chao Yu Cc: linux-f2fs-de...@lists.sourceforge.net Signed-off-by: Ming Lei --- fs/f2fs/data.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 622c44a1be78..57d5a2760bf1 100644 --- a/fs/f2fs/data.c +++ b/fs/f2

[PATCH v2 35/51] bcache: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: linux-bca...@vger.kernel.org Signed-off-by: Ming Lei --- drivers/md/bcache/btree.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c index 3da595ae565b..74cbb7387dc5 100644 --- a/drivers/md/bcache/btree.c +++ b/drivers/

[PATCH v2 47/51] fs: crypto: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- fs/crypto/bio.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/crypto/bio.c b/fs/crypto/bio.c index 6181e9526860..d5516ed19166 100644 --- a/fs/crypto/bio.c +++ b/fs/crypto/bio.c @@ -36,8 +36,9 @@ static void completion_pages(struct work_struct

[PATCH v2 49/51] fs/direct-io: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- fs/direct-io.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/direct-io.c b/fs/direct-io.c index c87077d1dc33..a139b3bbad8e 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -489,7 +489,9 @@ static blk_status_t dio_bio_complete(struct dio *

[PATCH v2 50/51] block: enable multipage bvecs

2017-06-26 Thread Ming Lei
This patch pulls the trigger for multipage bvecs. Now any request queue which supports queue cluster will see multipage bvecs. Signed-off-by: Ming Lei --- block/bio.c | 13 + 1 file changed, 13 insertions(+) diff --git a/block/bio.c b/block/bio.c index c460888f14b5..436305cde045 10

[PATCH v2 48/51] fs/btrfs: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: Chris Mason Cc: Josef Bacik Cc: David Sterba Cc: linux-bt...@vger.kernel.org Signed-off-by: Ming Lei --- fs/btrfs/compression.c | 3 ++- fs/btrfs/disk-io.c | 3 ++- fs/btrfs/extent_io.c | 12 fs/btrfs/inode.c | 6 -- fs/btrfs/raid56.c | 6 -- 5 fil

[PATCH v2 51/51] block: bio: pass segments to bio if bio_add_page() is bypassed

2017-06-26 Thread Ming Lei
Under some situations, such as block direct I/O, we can't use bio_add_page() for merging pages into multipage bvec, so a new function is implemented for converting page array into one segment array, then these cases can benefit from multipage bvec too. Signed-off-by: Ming Lei --- block/bio.c | 5

[PATCH v2 42/51] ext4: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: "Theodore Ts'o" Cc: Andreas Dilger Cc: linux-e...@vger.kernel.org Signed-off-by: Ming Lei --- fs/ext4/page-io.c | 3 ++- fs/ext4/readpage.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index 930ca0fc9a0f..0e59404fc530 100644

[PATCH v2 43/51] xfs: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: "Darrick J. Wong" Cc: linux-...@vger.kernel.org Signed-off-by: Ming Lei --- fs/xfs/xfs_aops.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 11ef989b8629..621efe71c70a 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@

[PATCH v2 39/51] fs/mpage: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- fs/mpage.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/mpage.c b/fs/mpage.c index 0da38f401564..bdb4692ae30c 100644 --- a/fs/mpage.c +++ b/fs/mpage.c @@ -46,9 +46,10 @@ static void mpage_end_io(struct bio *bio) { struct bio_vec *b

[PATCH v2 40/51] fs/block: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- fs/block_dev.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index a57c26bcb970..d82e43bd8e82 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -209,6 +209,7 @@ __blkdev_direct_IO_simple(struct kiocb *iocb,

[PATCH v2 38/51] dm-crypt: convert to bio_for_each_segment_all_sp()

2017-06-26 Thread Ming Lei
Cc: Mike Snitzer Cc:dm-de...@redhat.com Signed-off-by: Ming Lei --- drivers/md/dm-crypt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 664ba3504f48..0f2f44a73a32 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-cr

[PATCH v2 34/51] block: convert to singe/multi page version of bio_for_each_segment_all()

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- block/bio.c | 17 +++-- block/blk-zoned.c | 5 +++-- block/bounce.c| 6 -- 3 files changed, 18 insertions(+), 10 deletions(-) diff --git a/block/bio.c b/block/bio.c index 22e5deec7ec7..c460888f14b5 100644 --- a/block/bio.c +++ b/block/bio.c

[PATCH v2 31/51] fs/buffer.c: use bvec iterator to truncate the bio

2017-06-26 Thread Ming Lei
Once multipage bvec is enabled, the last bvec may include more than one page, this patch use bvec_get_last_page() to truncate the bio. Signed-off-by: Ming Lei --- fs/buffer.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 1910f539770b.

[PATCH v2 27/51] block: use bio_for_each_segment_mp() to map sg

2017-06-26 Thread Ming Lei
It is more efficient to use bio_for_each_segment_mp() for mapping sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Signed-off-by: Ming Lei --- block/blk-merge.c | 72 +++ 1 file changed, 52 insertion

[PATCH v2 30/51] block: introduce bvec_get_last_page()

2017-06-26 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last page, so introduce this helper to make them happy. Signed-off-by: Ming Lei --- include/linux/bvec.h | 14 ++ 1 file changed, 14 insertions(+) diff --git a/include/linux/bvec.h b/include/linux/bvec.h index 7addceea9828..6673e3c0b7eb 1006

[PATCH v2 29/51] block: bio: introduce single/multi page version of bio_for_each_segment_all()

2017-06-26 Thread Ming Lei
This patches introduce bio_for_each_segment_all_sp() and bio_for_each_segment_all_mp(). bio_for_each_segment_all_sp() is for replacing bio_for_each_segment_all() in case that the returned bvec has to be single page bvec. bio_for_each_segment_all_mp() is for replacing bio_for_each_segment_all() in

[PATCH v2 28/51] block: introduce bvec_for_each_sp_bvec()

2017-06-26 Thread Ming Lei
This helper can be used to iterate each singlepage bvec from one multipage bvec. Signed-off-by: Ming Lei --- include/linux/bvec.h | 14 ++ 1 file changed, 14 insertions(+) diff --git a/include/linux/bvec.h b/include/linux/bvec.h index 5c51c58fe202..7addceea9828 100644 --- a/include/

[PATCH v2 25/51] block: blk-merge: remove unnecessary check

2017-06-26 Thread Ming Lei
In this case, 'sectors' can't be zero at all, so remove the check and let the bio be splitted. Signed-off-by: Ming Lei --- block/blk-merge.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index bf7a0fa0199f..c6fcc49b9aea 100644 --- a/

[PATCH v2 16/51] block: bounce: avoid direct access to bvec table

2017-06-26 Thread Ming Lei
We will support multipage bvecs in the future, so change to iterator way for getting bv_page of bvec from original bio. Signed-off-by: Ming Lei --- block/bounce.c | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/block/bounce.c b/block/bounce.c index 916ee9a9a216.

[PATCH v2 10/51] dm: limit the max bio size as BIO_MAX_PAGES * PAGE_SIZE

2017-06-26 Thread Ming Lei
For BIO based DM, some targets aren't ready for dealing with bigger incoming bio than 1Mbyte, such as crypt target. Cc: Mike Snitzer Cc:dm-de...@redhat.com Signed-off-by: Ming Lei --- drivers/md/dm.c | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/md/dm.c

[PATCH v2 09/51] block: comment on bio_iov_iter_get_pages()

2017-06-26 Thread Ming Lei
bio_iov_iter_get_pages() used unused bvec spaces for storing page pointer array temporarily, and this patch comments on this usage wrt. multipage bvec support. Signed-off-by: Ming Lei --- block/bio.c | 4 1 file changed, 4 insertions(+) diff --git a/block/bio.c b/block/bio.c index a5db117e

[PATCH v2 08/51] block: comment on bio_alloc_pages()

2017-06-26 Thread Ming Lei
This patch adds comment on usage of bio_alloc_pages(). Signed-off-by: Ming Lei --- block/bio.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index 89a51bd49ab7..a5db117e8dfa 100644 --- a/block/bio.c +++ b/block/bio.c @@ -972,7 +972,9 @@ EXPORT_S

[PATCH v2 07/51] bcache: comment on direct access to bvec table

2017-06-26 Thread Ming Lei
Looks all are safe after multipage bvec is supported. Cc: linux-bca...@vger.kernel.org Signed-off-by: Ming Lei --- drivers/md/bcache/btree.c | 1 + drivers/md/bcache/super.c | 6 ++ drivers/md/bcache/util.c | 7 +++ 3 files changed, 14 insertions(+) diff --git a/drivers/md/bcache/btree

[PATCH v2 06/51] f2fs: f2fs_read_end_io: comment on direct access to bvec table

2017-06-26 Thread Ming Lei
Cc: Jaegeuk Kim Cc: Chao Yu Cc: linux-f2fs-de...@lists.sourceforge.net Signed-off-by: Ming Lei --- fs/f2fs/data.c | 4 1 file changed, 4 insertions(+) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 7697d03e8a98..622c44a1be78 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -56,6 +5

[PATCH v2 04/51] mm: page_io.c: comment on direct access to bvec table

2017-06-26 Thread Ming Lei
Cc: Andrew Morton Cc: linux...@kvack.org Signed-off-by: Ming Lei --- mm/page_io.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/page_io.c b/mm/page_io.c index b6c4ac388209..11c6f4a9a25b 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -43,6 +43,7 @@ static struct bio *get_swap_bio(gfp_

[PATCH v2 05/51] fs/buffer: comment on direct access to bvec table

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- fs/buffer.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/buffer.c b/fs/buffer.c index 4d5d03b42e11..1910f539770b 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -3052,8 +3052,13 @@ static void end_bio_bh_io_sync(struct bio *bio) void guar

[PATCH v2 03/51] kernel/power/swap.c: comment on direct access to bvec table

2017-06-26 Thread Ming Lei
Cc: "Rafael J. Wysocki" Cc: linux...@vger.kernel.org Signed-off-by: Ming Lei --- kernel/power/swap.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/power/swap.c b/kernel/power/swap.c index 57d22571f306..aa52ccc03fcc 100644 --- a/kernel/power/swap.c +++ b/kernel/power/swap.c @@ -238

[PATCH v2 02/51] block: loop: comment on direct access to bvec table

2017-06-26 Thread Ming Lei
Signed-off-by: Ming Lei --- drivers/block/loop.c | 5 + 1 file changed, 5 insertions(+) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 0de11444e317..88063ab17e9a 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -487,6 +487,11 @@ static int lo_rw_aio(struct loop

[PATCH v2 00/51] block: support multipage bvec

2017-06-26 Thread Ming Lei
Hi, This patchset brings multipage bvec into block layer: 1) what is multipage bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multipage bvec introduced? Kent

  1   2   >