[PATCH] DIO rewrite

2013-12-03 Thread Kent Overstreet
This is the dio rewrite based on immutable biovecs that I've been working on off and on for quite some time now; I think it's getting close to ready. It passes xfstests with xfs and CONFIG_XFS_DEBUG=y, though it definitely needs more review and outside testing. It doesn't quite work with btrfs but

[PATCH 11/11] direct-io: Rewrite based on immutable biovecs

2013-12-03 Thread Kent Overstreet
f the getblocks() call - dio_send_bio() is what primarily implements that behaviour. Signed-off-by: Kent Overstreet --- fs/direct-io.c | 1426 +++- 1 file changed, 384 insertions(+), 1042 deletions(-) diff --git a/fs/direct-io.c b/fs/direct

[PATCH 02/11] block: Gut bio_add_page()

2013-12-03 Thread Kent Overstreet
Since generic_make_request() can now handle arbitrary size bios, all we have to do is make sure the bvec array doesn't overflow. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- fs/bio.c | 137 ++- 1 file changed, 57 inser

[PATCH 2/2] block: fixup for generic bio chaining

2013-12-03 Thread Kent Overstreet
btrfs bits got lost in the rebase Signed-off-by: Kent Overstreet Cc: Chris Mason --- fs/btrfs/disk-io.c | 2 +- fs/btrfs/volumes.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 5a10c61..e71039e 100644 --- a/fs/btrfs/disk

[PATCH 1/2] block: Silence spurious compiler warnings

2013-12-03 Thread Kent Overstreet
The uninitialized_var() macro appears to not work on structs... Signed-off-by: Kent Overstreet --- block/blk-merge.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 05c17be..8f8adaa 100644 --- a/block/blk-merge.c +++ b/block

[PATCH] fixups for immutable biovecs

2013-12-03 Thread Kent Overstreet
Jens - for the "silence spurious compiler warnings", if you want to drop the previous version of that patch and replace with this one, it appears uninitialized_var() doesn't work on structs. And I somehow lost the btrfs fixes Chris Mason did for bio chaining and had to redo them so hopefully he can

Re: [PATCH 4/5] blk-lib.c: generic_make_request() handles large bios now

2013-11-27 Thread Kent Overstreet
On Wed, Nov 27, 2013 at 04:54:05AM -0800, Christoph Hellwig wrote: > > int blkdev_issue_discard(struct block_device *bdev, sector_t sector, > > sector_t nr_sects, gfp_t gfp_mask, unsigned long flags) > > { > > - DECLARE_COMPLETION_ONSTACK(wait); > > struct request_queue *q = bde

[PATCH 02/25] block: Convert various code to bio_for_each_segment()

2013-11-26 Thread Kent Overstreet
With immutable biovecs we don't want code accessing bi_io_vec directly - the uses this patch changes weren't incorrect since they all own the bio, but it makes the code harder to audit for no good reason - also, this will help with multipage bvecs later. Signed-off-by: Kent Overstreet

[PATCH 11/25] block: Kill bio_segments()/bi_vcnt usage

2013-11-26 Thread Kent Overstreet
in a couple patches) Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Neil Brown Cc: Nagalakshmi Nandigama Cc: Sreekanth Reddy Cc: "James E.J. Bottomley" --- drivers/block/ps3disk.c | 7 ++- drivers/md/bcache/io.c | 53 + dri

[PATCH 13/25] aoe: Convert to immutable biovecs

2013-11-26 Thread Kent Overstreet
truct members are effectively renamed: buf->resid -> buf->iter.bi_size buf->sector -> buf->iter.bi_sector f->bcnt -> f->iter.bi_size f->lba -> f->iter.bi_sector Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: "Ed L. Cashin"

[PATCH 12/25] block: Convert drivers to immutable biovecs

2013-11-26 Thread Kent Overstreet
Now that we've got a mechanism for immutable biovecs - bi_iter.bi_bvec_done - we need to convert drivers to use primitives that respect it instead of using the bvec array directly. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: NeilBrown Cc: Alasdair Kergon Cc: dm-de...@redha

[PATCH 08/25] block: Immutable bio vecs

2013-11-26 Thread Kent Overstreet
can use this for interesting things, like efficient bio splitting. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Lars Ellenberg Cc: Paul Clements Cc: drbd-u...@lists.linbit.com Cc: nbd-gene...@lists.sourceforge.net --- Documentation/block/biovecs.txt |

[PATCH 10/25] bio-integrity: Convert to bvec_iter

2013-11-26 Thread Kent Overstreet
The bio integrity is also stored in a bvec array, so if we use the bvec iter code we just added, the integrity code won't need to implement its own iteration stuff (bio_integrity_mark_head(), bio_integrity_mark_tail()) Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: "Martin K. Pet

[PATCH 09/25] block: Convert bio_copy_data() to bvec_iter

2013-11-26 Thread Kent Overstreet
Our fancy new bvec iterator makes code like this much easier to write. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- fs/bio.c | 60 +--- 1 file changed, 25 insertions(+), 35 deletions(-) diff --git a/fs/bio.c b/fs/bio.c index 07b4b7a

[PATCH 16/25] block: Refactor bio_clone_bioset() for immutable biovecs

2013-11-26 Thread Kent Overstreet
o for which bi_idx and bi_bvec done are 0 - like they would be if the caller had just allocated a new bio. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- fs/bio.c | 60 +++- 1 file changed, 47 insertions(+), 13 deletions(-) diff --git

[PATCH 15/25] block: Kill bio_iovec_idx(), __bio_iovec()

2013-11-26 Thread Kent Overstreet
we have to iterate over the bvec array with bio_for_each_segment() which checks against the current value of bi_iter.bi_size. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- block/blk-merge.c | 13 +++-- include/linux/bio.h | 26 -- 2 files changed, 19 inserti

[PATCH 14/25] ceph: Convert to immutable biovecs

2013-11-26 Thread Kent Overstreet
Now that we've got a mechanism for immutable biovecs - bi_iter.bi_bvec_done - we need to convert drivers to use primitives that respect it instead of using the bvec array directly. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Sage Weil Cc: ceph-de...@vger.kernel.org --- include/

[PATCH 25/25] block: Kill bio_pair_split()

2013-11-26 Thread Kent Overstreet
Signed-off-by: Kent Overstreet Cc: Jens Axboe --- fs/bio-integrity.c | 45 --- fs/bio.c| 90 - include/linux/bio.h | 30 -- 3 files changed, 165 deletions(-) diff --git a/fs/bio-integrity.c

[PATCH 17/25] block: Add bio_clone_fast()

2013-11-26 Thread Kent Overstreet
at the original bio's biovec. Signed-off-by: Kent Overstreet --- drivers/md/bcache/request.c | 8 ++ fs/bio.c| 60 + include/linux/bio.h | 2 ++ 3 files changed, 64 insertions(+), 6 deletions(-) diff --git a/drivers

[PATCH 21/25] block: Remove bi_idx hacks

2013-11-26 Thread Kent Overstreet
st() was a dangerous thing - various drivers would choke on such things. But with immutable biovecs and our new bio splitting that shares the biovecs, submitting partially completed bios has to work (and should work, now that all the drivers have been completed to the new primitives) Signed-off-by: Kent

[PATCH 22/25] block: Generic bio chaining

2013-11-26 Thread Kent Overstreet
able biovecs is making bio cloning cheaper. But for now, we add a bio_endio_nodec() for these cases. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- drivers/md/bcache/io.c | 2 +- drivers/md/dm-cache-target.c | 6 drivers/md/dm-snap.c | 1 + drivers/md/dm-thin.c | 8

[PATCH 24/25] block: Introduce new bio_split()

2013-11-26 Thread Kent Overstreet
ED doesn't seem to have clearly documented semantics anyways.) Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Martin K. Petersen Cc: Matthew Wilcox Cc: Keith Busch Cc: Vishal Verma Cc: Jiri Kosina Cc: Neil Brown --- drivers/block/nvme-core.c | 106 +++--

[PATCH 19/25] dm: Refactor for new bio cloning/splitting

2013-11-26 Thread Kent Overstreet
ers should never be modifying it. Also kill bio_sector_offset(), dm was the only user and it doesn't make much sense anymore. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Alasdair Kergon Cc: dm-de...@redhat.com Reviewed-by: Mike Snitzer --- drivers/md/dm-bio-record.h | 25 ---

[PATCH 23/25] block: Rename bio_split() -> bio_pair_split()

2013-11-26 Thread Kent Overstreet
This is prep work for introducing a more general bio_split(). Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: NeilBrown Cc: Alasdair Kergon Cc: Lars Ellenberg Cc: Peter Osterlund Cc: Sage Weil --- drivers/block/pktcdvd.c | 2 +- drivers/md/linear.c | 2 +- drivers/md/raid0.c

[PATCH 20/25] block: Don't save/copy bvec array anymore

2013-11-26 Thread Kent Overstreet
Now that drivers have been converted to the bvec_iter primitives, they shouldn't be modifying the biovec anymore and thus saving it is unnecessary - code that was previously making a backup of the bvec array can now just save bio->bi_iter. Signed-off-by: Kent Overstreet Cc: Jens Axboe

[PATCH 18/25] rbd: Refactor bio cloning

2013-11-26 Thread Kent Overstreet
Now that we've got drivers converted to the new immutable bvec primitives, bio splitting becomes much easier - this is how the new bio_split() will work. (Someone more familiar with the ceph code could probably use bio_clone_fast() instead of bio_clone() here). Signed-off-by: Kent Overstree

[PATCH 01/25] block: submit_bio_wait() conversions

2013-11-26 Thread Kent Overstreet
It was being open coded in a few places. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Joern Engel Cc: Prasad Joshi Cc: Neil Brown Cc: Chris Mason Acked-by: NeilBrown --- block/blk-flush.c | 19 +-- drivers/md/md.c| 14 +- fs/btrfs

[PATCH 03/25] bcache: Kill unaligned bvec hack

2013-11-26 Thread Kent Overstreet
Bcache has a hack to avoid cloning the biovec if it's all full pages - but with immutable biovecs coming this won't be necessary anymore. For now, we remove the special case and always clone the bvec array so that the immutable biovec patches are simpler. Signed-off-by: Kent

[PATCH 05/25] dm: Use bvec_iter for dm_bio_record()

2013-11-26 Thread Kent Overstreet
This patch doesn't itself have any functional changes, but immutable biovecs are going to add a bi_bvec_done member to bi_iter, which will need to be saved too here. Signed-off-by: Kent Overstreet Cc: Alasdair Kergon Cc: dm-de...@redhat.com Reviewed-by: Mike Snitzer --- drivers/md/d

[PATCH 06/25] block: Convert bio_iovec() to bvec_iter

2013-11-26 Thread Kent Overstreet
- those uses are all going to be removed, but we'll need the functionality from immutable to remove them - so for now rename the existing bio_iovec() -> __bio_iovec(), and it'll be removed in a couple patches. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: "Ed L. Cashin"

Re: [GIT PULL] Immutable biovecs

2013-11-26 Thread Kent Overstreet
On Mon, Nov 25, 2013 at 10:05:58PM -0800, Christoph Hellwig wrote: > On Mon, Nov 25, 2013 at 01:52:16PM -0800, Kent Overstreet wrote: > > Jens - here's immutable biovecs, rebased and ready for 3.14. Changes since > > the > > last version of the series: > > Can you

[PATCH] block: Silence spurious compiler warnings

2013-11-26 Thread Kent Overstreet
>From 46e7081430f5f483906f496733a23f8e9d898879 Mon Sep 17 00:00:00 2001 From: Kent Overstreet Date: Tue, 26 Nov 2013 16:36:49 -0800 Subject: [PATCH] block: Silence spurious compiler warnings Signed-off-by: Kent Overstreet --- On Tue, Nov 26, 2013 at 12:02:08PM -0700, Jens Axboe wrote: >

Re: [PATCH 2/5] mtip32xx: handle arbitrary size bios

2013-11-25 Thread Kent Overstreet
On Mon, Nov 25, 2013 at 10:10:18PM -0800, Christoph Hellwig wrote: > On Mon, Nov 25, 2013 at 02:30:30PM -0800, Kent Overstreet wrote: > > We get a measurable performance increase by handling this in the driver when > > we're already looping over the biovec, instead of handl

Re: [PATCH 1/5] block: Make generic_make_request handle arbitrary sized bios

2013-11-25 Thread Kent Overstreet
On Mon, Nov 25, 2013 at 10:09:21PM -0800, Christoph Hellwig wrote: > > + q->bio_split = bioset_create(4, 0); > > + if (!q->bio_split) > > + goto fail_id; > > How did we arrive at a mempool size of 4 to make sure we can always make > progress with arbitrarily sized bios? Shouldn't we

Re: GPF in aio_migratepage

2013-11-25 Thread Kent Overstreet
On Tue, Nov 26, 2013 at 01:01:32AM -0500, Dave Jones wrote: > On Mon, Nov 25, 2013 at 10:26:45PM -0500, Dave Jones wrote: > > Hi Kent, > > > > I hit the GPF below on a tree based on > 8e45099e029bb6b369b27d8d4920db8caff5ecce > > which has your commit e34ecee2ae791df674dfb466ce40692ca6218e43 >

[PATCH 5/5] bcache: generic_make_request() handles large bios now

2013-11-25 Thread Kent Overstreet
So we get to delete our hacky workaround. Signed-off-by: Kent Overstreet --- drivers/md/bcache/bcache.h| 18 drivers/md/bcache/io.c| 100 +- drivers/md/bcache/journal.c | 4 +- drivers/md/bcache/request.c | 16 +++ drivers

[PATCH 2/5] mtip32xx: handle arbitrary size bios

2013-11-25 Thread Kent Overstreet
We get a measurable performance increase by handling this in the driver when we're already looping over the biovec, instead of handling it separately in generic_make_request() (or bio_add_page() originally) Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Asai Thambi S P Cc: Sam Bra

[PATCH 3/5] block: Gut bio_add_page()

2013-11-25 Thread Kent Overstreet
Since generic_make_request() can now handle arbitrary size bios, all we have to do is make sure the bvec array doesn't overflow. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- fs/bio.c | 137 ++- 1 file changed, 57 inser

[PATCH 4/5] blk-lib.c: generic_make_request() handles large bios now

2013-11-25 Thread Kent Overstreet
eally shouldn't matter. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- block/blk-lib.c | 175 ++-- 1 file changed, 30 insertions(+), 145 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 2da76c9..368c36a 100644 --- a/block

[PATCH 1/5] block: Make generic_make_request handle arbitrary sized bios

2013-11-25 Thread Kent Overstreet
tforms/iss/simdisk.c) * brd_make_request (ramdisk - drivers/block/brd.c) * loop_make_request * null_queue_bio * bcache's make_request fns Some others are almost certainly safe to remove now, but will be left for future patches. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Neil Brown Cc

[PATCH] Make generic_make_request() handle arbitrary size bios

2013-11-25 Thread Kent Overstreet
This builds off of immutable biovecs; it's a small enough patch series that I'd like to try to get it into 3.14 but it is a pretty significant change in behaviour for the block layer so it should definitely be considered separately from that series. What the series does is pretty simple - like the

[GIT PULL] Immutable biovecs

2013-11-25 Thread Kent Overstreet
ns for you to fetch changes up to 4b1faf931650d4a35b2a570318862821d6a962e3: block: Kill bio_pair_split() (2013-11-23 22:33:57 -0800) ---- Kent Overstreet (25): block: submit_bio_wait() conversions block: Convert various code to

[PATCH] block: submit_bio_wait() conversions

2013-11-23 Thread Kent Overstreet
It was being open coded in a few places. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Joern Engel Cc: Prasad Joshi Cc: Neil Brown Cc: Chris Mason --- block/blk-flush.c | 19 +-- drivers/md/md.c| 12 +--- fs/btrfs/check-integrity.c | 32

Re: [GIT PULL] direct IO support for loop driver

2013-11-21 Thread Kent Overstreet
On Thu, Nov 21, 2013 at 01:58:37AM -0800, Christoph Hellwig wrote: > On Wed, Nov 20, 2013 at 01:19:33PM -0800, Linus Torvalds wrote: > > I really don't see the point of all this crap. All this for the loop > > driver? If so, it had better at least be prettier than it is. > > It's for anyone trying

Re: [GIT PULL] direct IO support for loop driver

2013-11-20 Thread Kent Overstreet
On Wed, Nov 20, 2013 at 01:38:19PM -0800, Linus Torvalds wrote: > On Wed, Nov 20, 2013 at 1:19 PM, Linus Torvalds > wrote: > > > > At that point, I just couldn't take it any more. > > Just to clarify, I think it might be fixable. But it does need fixing, > because I really feel dirty from reading

[GIT PULL] bcache for 3.13

2013-11-10 Thread Kent Overstreet
ok (1): bcache: defensively handle format strings Kent Overstreet (52): bcache: Fix dirty_data accounting bcache: Fix a journalling performance bug bcache: Fix a lockdep splat bcache: Use blkdev_issue_discard() bcache: Add on error panic/unregister setting bcache: Stripe

Re: linux-next: manual merge of the block tree with the tree

2013-11-08 Thread Kent Overstreet
On Fri, Nov 08, 2013 at 12:32:51AM -0800, Christoph Hellwig wrote: > On Fri, Nov 08, 2013 at 12:17:37AM -0800, Kent Overstreet wrote: > > The core issue isn't whether the IO is going to a block based filesystem > > (but thanks for pointing out that that's not necessarily

Re: linux-next: manual merge of the block tree with the tree

2013-11-08 Thread Kent Overstreet
On Fri, Nov 08, 2013 at 12:02:21AM -0800, Christoph Hellwig wrote: > On Thu, Nov 07, 2013 at 11:56:17PM -0800, Kent Overstreet wrote: > > So, I don't think the iov_iter stuff is the right approach for solving > > the loop issue; it's an ugly hack and after immutable biove

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 11:44:45PM -0800, Christoph Hellwig wrote: > On Thu, Nov 07, 2013 at 11:39:59PM -0800, Kent Overstreet wrote: > > On Thu, Nov 07, 2013 at 11:33:24PM -0800, Christoph Hellwig wrote: > > > The changes for direct I/O from kernel space have been in for a lo

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 11:33:24PM -0800, Christoph Hellwig wrote: > The changes for direct I/O from kernel space have been in for a long > time, and they are blocking multiple consumers of the interface from > getting submitted for about a year now. Even if the guts of the > direct-io code will g

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Fri, Nov 08, 2013 at 12:53:07PM +1100, Stephen Rothwell wrote: > Hi all, > > On Thu, 07 Nov 2013 18:04:57 -0600 Dave Kleikamp > wrote: > > > > Can you please drop the aio-direct tree for the time being? > > OK, I was afraid of this, but, yes, I can drop it. I am not quite sure > what affect

Re: [PATCH 1/9] block: Convert various code to bio_for_each_segment()

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 12:26:30PM +0100, Jan Kara wrote: > On Mon 04-11-13 15:36:19, Kent Overstreet wrote: > > With immutable biovecs we don't want code accessing bi_io_vec directly - > > the uses this patch changes weren't incorrect since they all own the > > bi

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 01:20:26PM -0600, Dave Kleikamp wrote: > On 11/02/2013 03:50 PM, Dave Kleikamp wrote: > > On 11/01/2013 03:53 PM, Jens Axboe wrote: > > >> So we've three immediate options: > >> > >> 1) You base it on top of the block tree > >> 2) I carry the loop updates > >> 3) You hand S

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 11:17:22AM -0800, Olof Johansson wrote: > On Sat, Nov 2, 2013 at 1:50 PM, Dave Kleikamp > wrote: > > On 11/01/2013 03:53 PM, Jens Axboe wrote: > >> On 11/01/2013 02:41 PM, Dave Kleikamp wrote: > >>> On 11/01/2013 03:27 PM, Jens Axboe wrote: > On 11/01/2013 02:22 PM, S

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
On Wed, Nov 06, 2013 at 04:25:45PM -0500, Chris Mason wrote: > Quoting Kent Overstreet (2013-11-06 15:57:34) > > On Wed, Nov 06, 2013 at 03:22:36PM -0500, Chris Mason wrote: > > > Quoting Kent Overstreet (2013-11-06 15:02:22) > > [ ... nods, thanks! ... ] > > >

[PATCH] block/s390: Fixes for "Convert bio_for_each_segment() to bvec_iter"

2013-11-06 Thread Kent Overstreet
5d1f127c3e0c57d64ce75ee04a0db2b40a3e21df was incomplete - this just converts s390's drivers. Signed-off-by: Kent Overstreet --- I was poking around a bit, and it looks like there's nothing really preventing these drivers from building on non s390 except that a bunch of headers are in

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
On Wed, Nov 06, 2013 at 03:22:36PM -0500, Chris Mason wrote: > Quoting Kent Overstreet (2013-11-06 15:02:22) > > Yup - that should actually be safe for all the existing bio_clone() users > > actually, I audited all of them - because normally you're not going to > > com

Re: block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
ive context for which reported problem(s) are being > fixed by this patch. > > On Tue, Nov 05 2013 at 10:48pm -0500, > Kent Overstreet wrote: > > > This patch reverts the default behaviour introduced by > > 9fc6286f347d00528adcdcf12396d220f47492ed - bio_clone_biove

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
On Wed, Nov 06, 2013 at 11:11:30AM -0500, Chris Mason wrote: > Quoting Kent Overstreet (2013-11-05 22:48:41) > > This patch reverts the default behaviour introduced by > > 9fc6286f347d00528adcdcf12396d220f47492ed - bio_clone_biovec() no clonger > > shares the source bio&

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-05 Thread Kent Overstreet
On Tue, Nov 05, 2013 at 09:02:19PM -0800, Olof Johansson wrote: > Hi, > > On Tue, Nov 5, 2013 at 7:48 PM, Kent Overstreet wrote: > > > Chris, Olaf, can you two in particular test this? I have tested the bounce > > buffer code (and bcache), but Jens told me today th

[PATCH] block: Revert bio_clone() default behaviour

2013-11-05 Thread Kent Overstreet
we don't want code looking at bi_vcnt at all for bios it does not own. However, this patch _shouldn't_ cause any regressions because of this since we're reverting back to the old bio_clone() behaviour. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Chris Mason Cc: Mi

Re: [dm-devel] [PATCH 4/9] block: Make generic_make_request handle arbitrary sized bios

2013-11-04 Thread Kent Overstreet
On Mon, Nov 04, 2013 at 03:56:52PM -0800, Mike Christie wrote: > On 11/04/2013 03:36 PM, Kent Overstreet wrote: > > @@ -1822,6 +1820,14 @@ void generic_make_request(struct bio *bio) > > */ > > blk_queue_bounce(q, &bio); > > > > +

[PATCH 4/9] block: Make generic_make_request handle arbitrary sized bios

2013-11-04 Thread Kent Overstreet
s are convenient, and more importantly stacked drivers don't have to deal with both their own bio size limitations and the limitations of the (potentially multiple) devices underneath them. In the future this will let us delete merge_bvec_fn and a bunch of other code. Signed-off-by: Kent Overstreet

[PATCH 5/9] block: Gut bio_add_page(), kill bio_add_pc_page()

2013-11-04 Thread Kent Overstreet
Since generic_make_request() can now handle arbitrary size bios, all we have to do is make sure the bvec array doesn't overflow. Signed-off-by: Kent Overstreet --- drivers/scsi/osd/osd_initiator.c | 5 +- drivers/target/target_core_pscsi.c | 5 +- fs/bio.c

[PATCH 3/9] block: Move bouncing to generic_make_request()

2013-11-04 Thread Kent Overstreet
take into account bouncing (as it'll already have been done). Also, __blk_recalc_rq_segments() now doesn't have to take into account potential bouncing - it's already been done. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Jiri Kosina Cc: Asai Thambi S P --- block/blk-cor

[PATCH 6/9] mtip32xx: handle arbitrary size bios

2013-11-04 Thread Kent Overstreet
We get a measurable performance increase by handling this in the driver when we're already looping over the biovec, instead of handling it separately in generic_make_request() (or bio_add_page() originally) Signed-off-by: Kent Overstreet --- drivers/block/mtip32xx/mtip32xx.c

[PATCH 1/9] block: Convert various code to bio_for_each_segment()

2013-11-04 Thread Kent Overstreet
With immutable biovecs we don't want code accessing bi_io_vec directly - the uses this patch changes weren't incorrect since they all own the bio, but it makes the code harder to audit for no good reason - also, this will help with multipage bvecs later. Signed-off-by: Kent Overstreet

[PATCH 8/9] bcache: generic_make_request() handles large bios now

2013-11-04 Thread Kent Overstreet
So we get to delete our hacky workaround. Signed-off-by: Kent Overstreet --- drivers/md/bcache/bcache.h| 18 drivers/md/bcache/io.c| 100 +- drivers/md/bcache/journal.c | 4 +- drivers/md/bcache/request.c | 16 +++ drivers

[PATCH 7/9] blk-lib.c: generic_make_request() handles large bios now

2013-11-04 Thread Kent Overstreet
eally shouldn't matter. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- block/blk-lib.c | 175 ++-- 1 file changed, 30 insertions(+), 145 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 2da76c9..368c36a 100644 --- a/block

[PATCH 9/9] block: Add bio_get_user_pages()

2013-11-04 Thread Kent Overstreet
This replaces some of the code that was in __bio_map_user_iov(), and soon we're going to use this helper in the dio code. Note that this relies on the recent change to make generic_make_request() take arbitrary sized bios - we're not using bio_add_page() here. Signed-off-by: Kent Over

[PATCH 2/9] block: submit_bio_wait() conversions

2013-11-04 Thread Kent Overstreet
It was being open coded in a few places. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Joern Engel Cc: Prasad Joshi --- block/blk-flush.c | 19 +-- fs/logfs/dev_bdev.c | 8 +--- 2 files changed, 2 insertions(+), 25 deletions(-) diff --git a/block/blk-flush.c b

[PATCH] Block layer stuff/DIO rewrite prep for 3.14

2013-11-04 Thread Kent Overstreet
Now that immutable biovecs is in, these are the remaining patches required for my DIO rewrite, along with some related cleanup/refactoring. The key enabler is patch 4 - making generic_make_request() handle arbitary sized bios. This takes what was once bio_add_page()'s responsibility and pushes it

Re: [GIT PULL] bcache changes for 3.13

2013-11-04 Thread Kent Overstreet
On Mon, Nov 04, 2013 at 01:12:17PM -0700, Jens Axboe wrote: > On 11/04/2013 01:07 PM, Kent Overstreet wrote: > > Hey Jens, sorry for being late with this - anyways, it's roughly the same > > set of > > patches you had queued up before plus a few minor fixes, but it&#x

Re: [PATCH for-next] dm: fix missing bi_remaining accounting

2013-11-04 Thread Kent Overstreet
On Mon, Nov 04, 2013 at 10:06:00AM -0500, Mikulas Patocka wrote: > > > On Fri, 1 Nov 2013, Jens Axboe wrote: > > > On 11/01/2013 07:59 AM, Mike Snitzer wrote: > > > Add the missing bi_remaining increment, required by the block layer's > > > new bio-chaining code, to both the verity and old snaps

[GIT PULL] bcache changes for 3.13

2013-11-04 Thread Kent Overstreet
3:20 -0700) Kees Cook (1): bcache: defensively handle format strings Kent Overstreet (53): bcache: Fixed incorrect order of arguments to bio_alloc_bioset() bcache: Convert bch_data_verify() to immutable biovecs bca

[PATCH] block: Document immutable biovecs

2013-10-31 Thread Kent Overstreet
Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: NeilBrown Cc: Christoph Hellwig --- Documentation/block/biovecs.txt | 111 include/linux/blk_types.h | 3 +- 2 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 Documentation

Re: [PATCH] Btrfs: update bi_remaining to relfect our bio endio chaining

2013-10-31 Thread Kent Overstreet
On Thu, Oct 31, 2013 at 03:38:20PM -0400, Chris Mason wrote: > Btrfs is sometimes calling bio_endio twice on the same bio while > we chain things. This makes sure we don't trip over new assertions in > fs/bio.c > > Signed-off-by: Chris Mason Beat me to it :P Tested-by: Ke

Re: [PATCH 17/23] dm: Refactor for new bio cloning/splitting

2013-10-29 Thread Kent Overstreet
On Tue, Oct 29, 2013 at 08:09:08PM -0400, Mike Snitzer wrote: > On Tue, Oct 29 2013 at 4:18pm -0400, > Kent Overstreet wrote: > > > We need to convert the dm code to the new bvec_iter primitives which > > respect bi_bvec_done; they also allow us to drastically simplify d

Re: [PATCH] Immutable biovecs

2013-10-29 Thread Kent Overstreet
On Tue, Oct 29, 2013 at 02:36:47PM -0600, Jens Axboe wrote: > On Tue, Oct 29 2013, Kent Overstreet wrote: > > Jens - this patch series is ready to go, driver maintainers have reviewed > > and > > tested it and the code's been essentially done for probably a year now

[PATCH] Immutable biovecs

2013-10-29 Thread Kent Overstreet
Jens - this patch series is ready to go, driver maintainers have reviewed and tested it and the code's been essentially done for probably a year now - can we get this into 3.13? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.ke

[PATCH 01/23] block: Use rw_copy_check_uvector()

2013-10-29 Thread Kent Overstreet
No need for silly open coding - and struct sg_iovec has exactly the same layout as struct iovec... Signed-off-by: Kent Overstreet Cc: Jens Axboe --- block/scsi_ioctl.c | 39 ++- 1 file changed, 10 insertions(+), 29 deletions(-) diff --git a/block

[PATCH 02/23] block: Consolidate duplicated bio_trim() implementations

2013-10-29 Thread Kent Overstreet
Someone cut and pasted md's md_trim_bio() into xen-blkfront.c. Come on, we should know better than this. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Neil Brown Cc: Konrad Rzeszutek Wilk Cc: Jeremy Fitzhardinge --- drivers/block/xen-blkfront.c

[PATCH 10/23] bio-integrity: Convert to bvec_iter

2013-10-29 Thread Kent Overstreet
The bio integrity is also stored in a bvec array, so if we use the bvec iter code we just added, the integrity code won't need to implement its own iteration stuff (bio_integrity_mark_head(), bio_integrity_mark_tail()) Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: "Martin K. Pet

[PATCH 12/23] block: Convert drivers to immutable biovecs

2013-10-29 Thread Kent Overstreet
Now that we've got a mechanism for immutable biovecs - bi_iter.bi_bvec_done - we need to convert drivers to use primitives that respect it instead of using the bvec array directly. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: NeilBrown Cc: Alasdair Kergon Cc: dm-de...@redha

[PATCH 08/23] block: Immutable bio vecs

2013-10-29 Thread Kent Overstreet
can use this for interesting things, like efficient bio splitting. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Lars Ellenberg Cc: Paul Clements Cc: drbd-u...@lists.linbit.com Cc: nbd-gene...@lists.sourceforge.net --- drivers/block/drbd/drbd_main.c | 4 +-- drivers/block/nbd.c|

[PATCH 13/23] aoe: Convert to immutable biovecs

2013-10-29 Thread Kent Overstreet
truct members are effectively renamed: buf->resid -> buf->iter.bi_size buf->sector -> buf->iter.bi_sector f->bcnt -> f->iter.bi_size f->lba -> f->iter.bi_sector Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: "Ed L. Cashin"

[PATCH 11/23] block: Kill bio_segments()/bi_vcnt usage

2013-10-29 Thread Kent Overstreet
in a couple patches) Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Neil Brown Cc: Nagalakshmi Nandigama Cc: Sreekanth Reddy Cc: "James E.J. Bottomley" --- drivers/block/ps3disk.c | 7 ++- drivers/md/bcache/io.c | 53 + dri

[PATCH 09/23] block: Convert bio_copy_data() to bvec_iter

2013-10-29 Thread Kent Overstreet
Our fancy new bvec iterator makes code like this much easier to write. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- fs/bio.c | 60 +--- 1 file changed, 25 insertions(+), 35 deletions(-) diff --git a/fs/bio.c b/fs/bio.c index b39436a

[PATCH 06/23] block: Convert bio_iovec() to bvec_iter

2013-10-29 Thread Kent Overstreet
- those uses are all going to be removed, but we'll need the functionality from immutable to remove them - so for now rename the existing bio_iovec() -> __bio_iovec(), and it'll be removed in a couple patches. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: "Ed L. Cashin"

[PATCH 14/23] ceph: Convert to immutable biovecs

2013-10-29 Thread Kent Overstreet
Now that we've got a mechanism for immutable biovecs - bi_iter.bi_bvec_done - we need to convert drivers to use primitives that respect it instead of using the bvec array directly. Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Sage Weil Cc: ceph-de...@vger.kernel.org --- include/

[PATCH 15/23] block: Kill bio_iovec_idx(), __bio_iovec()

2013-10-29 Thread Kent Overstreet
we have to iterate over the bvec array with bio_for_each_segment() which checks against the current value of bi_iter.bi_size. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- block/blk-merge.c | 13 +++-- include/linux/bio.h | 26 -- 2 files changed, 19 inserti

[PATCH 22/23] block: Kill bio_pair_split()

2013-10-29 Thread Kent Overstreet
Signed-off-by: Kent Overstreet Cc: Jens Axboe --- fs/bio-integrity.c | 45 --- fs/bio.c| 90 - include/linux/bio.h | 30 -- 3 files changed, 165 deletions(-) diff --git a/fs/bio-integrity.c

[PATCH 20/23] block: Rename bio_split() -> bio_pair_split()

2013-10-29 Thread Kent Overstreet
This is prep work for introducing a more general bio_split(). Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: NeilBrown Cc: Alasdair Kergon Cc: Lars Ellenberg Cc: Peter Osterlund Cc: Sage Weil --- drivers/block/pktcdvd.c | 2 +- drivers/md/linear.c | 2 +- drivers/md/raid0.c

[PATCH 19/23] block: Generic bio chaining

2013-10-29 Thread Kent Overstreet
able biovecs is making bio cloning cheaper. But for now, we add a bio_endio_nodec() for these cases. Signed-off-by: Kent Overstreet Cc: Jens Axboe --- drivers/md/bcache/io.c | 2 +- drivers/md/dm-cache-target.c | 6 fs/bio-integrity.c | 2 +- fs/bio.c

[PATCH 21/23] block: Introduce new bio_split()

2013-10-29 Thread Kent Overstreet
ED doesn't seem to have clearly documented semantics anyways.) Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: Martin K. Petersen Cc: Matthew Wilcox Cc: Keith Busch Cc: Vishal Verma Cc: Jiri Kosina Cc: Neil Brown --- drivers/block/nvme-core.c | 106 +++--

[PATCH 23/23] block: Don't save/copy bvec array anymore, share when cloning

2013-10-29 Thread Kent Overstreet
euse the original bio's bvec array. For code that does need to modify the clone's biovec (the bounce buffer code, mainly), add bio_clone_biovec(). Signed-off-by: Kent Overstreet Cc: Jens Axboe Cc: "Martin K. Petersen" Cc: Alasdair Kergon Cc: dm-de...@redhat.com --- drive

[PATCH 17/23] dm: Refactor for new bio cloning/splitting

2013-10-29 Thread Kent Overstreet
We need to convert the dm code to the new bvec_iter primitives which respect bi_bvec_done; they also allow us to drastically simplify dm's bio splitting code. Also kill bio_sector_offset(), dm was the only user and it doesn't make much sense anymore. Signed-off-by: Kent Overstreet

[PATCH 18/23] block: Remove bi_idx hacks

2013-10-29 Thread Kent Overstreet
st() was a dangerous thing - various drivers would choke on such things. But with immutable biovecs and our new bio splitting that shares the biovecs, submitting partially completed bios has to work (and should work, now that all the drivers have been completed to the new primitives) Signed-off-by: Kent

[PATCH 05/23] dm: Use bvec_iter for dm_bio_record()

2013-10-29 Thread Kent Overstreet
This patch doesn't itself have any functional changes, but immutable biovecs are going to add a bi_bvec_done member to bi_iter, which will need to be saved too here. Signed-off-by: Kent Overstreet Cc: Alasdair Kergon Cc: dm-de...@redhat.com --- drivers/md/dm-bio-record.h | 12 +++---

[PATCH 16/23] rbd: Refactor bio cloning, don't clone biovecs

2013-10-29 Thread Kent Overstreet
Now that we've got drivers converted to the new immutable bvec primitives, bio splitting becomes much easier. In a few patches, bio_clone() will be changed to share the old bio's bvec instead of copying it, and bio_split() will do exactly what's being done here. Signed-off-by: Kent

<    1   2   3   4   5   6   7   8   9   10   >