[PATCH 19/22] block: Rename bio_split() - bio_pair_split()

2013-08-07 Thread Kent Overstreet
This is prep work for introducing a more general bio_split(). Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: NeilBrown ne...@suse.de Cc: Alasdair Kergon a...@redhat.com Cc: Lars Ellenberg lars.ellenb...@linbit.com Cc: Peter Osterlund pete...@telia.com Cc

[PATCH 11/22] block: Kill bio_segments()/bi_vcnt usage

2013-08-07 Thread Kent Overstreet
in a couple patches) Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Neil Brown ne...@suse.de Cc: Nagalakshmi Nandigama nagalakshmi.nandig...@lsi.com Cc: Sreekanth Reddy sreekanth.re...@lsi.com Cc: James E.J. Bottomley jbottom...@parallels.com --- drivers/block/ps3disk.c

Re: [PATCH-v3 1/4] idr: Percpu ida

2013-08-26 Thread Kent Overstreet
On Tue, Aug 20, 2013 at 02:31:57PM -0700, Andrew Morton wrote: On Fri, 16 Aug 2013 23:09:06 + Nicholas A. Bellinger n...@linux-iscsi.org wrote: From: Kent Overstreet k...@daterainc.com Percpu frontend for allocating ids. With percpu allocation (that works), it's impossible

Re: [PATCH-v3 1/4] idr: Percpu ida

2013-08-26 Thread Kent Overstreet
On Wed, Aug 21, 2013 at 06:25:58PM +, Christoph Lameter wrote: On Fri, 16 Aug 2013, Nicholas A. Bellinger wrote: + spinlock_t lock; Remove the spinlock. As Andrew noted, the spinlock is needed because of tag stealing. (You don't think I'd stick a spinlock on a

[GIT PULL] Bcache fix for 3.11

2013-08-26 Thread Kent Overstreet
://evilpiepirate.org/~kent/linux-bcache.git bcache-for-3.11 for you to fetch changes up to 174c1412939888ffedc4da02b375bc579af2a93c: bcache: Fix a dumb CPU spinning bug in writeback (2013-08-23 16:30:22 -0700) Kent Overstreet (1

Bcache fixes for stable

2013-08-26 Thread Kent Overstreet
) Kent Overstreet (1): bcache: FUA fixes Kumar Amit Mehta (1): md: bcache: io.c: fix a potential NULL pointer dereference drivers/md/bcache/btree.c | 23 +-- drivers/md/bcache/io.c | 2 ++ drivers/md/bcache/journal.c | 2 +- drivers/md/bcache

Re: Bcache fixes for stable

2013-08-26 Thread Kent Overstreet
On Mon, Aug 26, 2013 at 03:19:57PM -0700, Greg Kroah-Hartman wrote: On Mon, Aug 26, 2013 at 03:02:49PM -0700, Kent Overstreet wrote: Two fixes that went into 3.11 should've been marked for stable, but I screwed up and forgot to mark them. One of them fixes bugs that appear to being hit

Re: bcache: Fix a writeback performance regression

2013-08-28 Thread Kent Overstreet
22.08.2013 09:32, schrieb Stefan Priebe - Profihost AG: great! Everything seems to work fine now! Except read_dirty always going to negative values after a reboot. Stefan Am 22.08.2013 08:02, schrieb Kent Overstreet: On Thu, Aug 22, 2013 at 07:59:04AM +0200, Stefan Priebe wrote

Re: [PATCH-v4 1/6] idr: Percpu ida

2013-08-28 Thread Kent Overstreet
On Tue, Aug 27, 2013 at 01:43:14PM -0700, Andrew Morton wrote: On Tue, 27 Aug 2013 20:11:41 + Nicholas A. Bellinger n...@linux-iscsi.org wrote: Percpu frontend for allocating ids. With percpu allocation (that works), it's impossible to guarantee it will always be possible to allocate

Re: bcache: Fix a writeback performance regression

2013-08-19 Thread Kent Overstreet
won't happen due to the PD controller based ratelimiting) - boom, deadlock. Here's the fixup patch I just tested and am applying: From 0af68de350e05e43fd093b36dcb0fe8aa838fabf Mon Sep 17 00:00:00 2001 From: Kent Overstreet k...@daterainc.com Date: Mon, 19 Aug 2013 15:26:22 -0700 Subject: [PATCH

Re: [PATCH] idr: Use this_cpu_ptr() for percpu_ida

2013-08-20 Thread Kent Overstreet
On Tue, Aug 20, 2013 at 02:29:56PM -0700, Andrew Morton wrote: On Tue, 20 Aug 2013 14:19:06 -0700 Nicholas A. Bellinger n...@linux-iscsi.org wrote: On Thu, 2013-08-08 at 14:32 +, Christoph Lameter wrote: On Wed, 7 Aug 2013, Kent Overstreet wrote: One thing that was bugging me

Re: [PATCH] bcache: Remove use of down/up_read_non_owner()

2013-08-20 Thread Kent Overstreet
On Tue, Aug 20, 2013 at 11:16:02AM -0400, Steven Rostedt wrote: The down/up_read_non_owner() is a nasty hack in the API of the rwsem operations. It was once removed, but then resurrected for use with bcache. Not only is the API an abomination to the rwsem API, it also prevents bcache from

Re: [PATCH] idr: Use this_cpu_ptr() for percpu_ida

2013-08-20 Thread Kent Overstreet
On Tue, Aug 20, 2013 at 10:07:42PM -0400, Tejun Heo wrote: Hello, Kent. On Tue, Aug 20, 2013 at 07:01:32PM -0700, Kent Overstreet wrote: I think Tejun and I might be at a bit of an impasse with the ida rewrite itself, but I don't think there were any outstanding objections to the percpu

Re: [PATCH] bcache: Remove use of down/up_read_non_owner()

2013-08-20 Thread Kent Overstreet
On Tue, Aug 20, 2013 at 10:41:53PM -0400, Steven Rostedt wrote: I get that there's a problem, but the bcache code REALLY IS USING THE RWSEM AS A LOCK; the answer isn't to open code the lock! Actually, it is using it as a semaphore. The problem with Linux was that it only had spin locks and

Re: [PATCH] idr: Use this_cpu_ptr() for percpu_ida

2013-08-21 Thread Kent Overstreet
On Wed, Aug 21, 2013 at 07:59:41AM -0400, Tejun Heo wrote: Hello, Kent. On Tue, Aug 20, 2013 at 07:31:51PM -0700, Kent Overstreet wrote: All this for a performance improvement of 10x to 50x (or more), for the ida sizes I measured. That's misleading, isn't it? It's comparing

Re: bcache: Fix a writeback performance regression

2013-08-21 Thread Kent Overstreet
On Tue, Aug 20, 2013 at 10:07:45AM +0200, Stefan Priebe - Profihost AG wrote: Am 20.08.2013 10:01, schrieb Stefan Priebe - Profihost AG: Am 20.08.2013 00:27, schrieb Kent Overstreet: On Mon, Aug 19, 2013 at 12:09:24AM +0200, Stefan Priebe wrote: Vanilla 3.10.7 + bcache: Fix a writeback

Re: [PATCH] idr: Use this_cpu_ptr() for percpu_ida

2013-08-21 Thread Kent Overstreet
On Wed, Aug 21, 2013 at 05:16:50PM -0400, Tejun Heo wrote: Hello, Kent. On Wed, Aug 21, 2013 at 02:09:01PM -0700, Kent Overstreet wrote: These micro optimizations mean either less pointer chasing or less branching in the _common_ case; you'd trade common case performance for avoiding

Re: bcache: Fix a writeback performance regression

2013-08-21 Thread Kent Overstreet
On Tue, Aug 20, 2013 at 10:07:45AM +0200, Stefan Priebe - Profihost AG wrote: Am 20.08.2013 10:01, schrieb Stefan Priebe - Profihost AG: Am 20.08.2013 00:27, schrieb Kent Overstreet: On Mon, Aug 19, 2013 at 12:09:24AM +0200, Stefan Priebe wrote: Vanilla 3.10.7 + bcache: Fix a writeback

Re: bcache: Fix a writeback performance regression

2013-08-21 Thread Kent Overstreet
-22 06:28:43 [a0116d13] closure_queue+0x43/0x60 [bcache] 2013-08-22 06:28:43 [a010c49e] request_write+0x19e/0x380 [bcache] Am 22.08.2013 01:47, schrieb Kent Overstreet: On Tue, Aug 20, 2013 at 10:07:45AM +0200, Stefan Priebe - Profihost AG wrote: Am 20.08.2013 10:01

Re: bcache: Fix a writeback performance regression

2013-08-22 Thread Kent Overstreet
On Thu, Aug 22, 2013 at 07:59:04AM +0200, Stefan Priebe wrote: schedule_timeout() is not the same as schedule_timeout_interruptible(). just search and replace? So i can try on my own. The one in read_dirty(), line ~330 -- To unsubscribe from this list: send the line unsubscribe

Re: bcache: Fix a writeback performance regression

2013-08-22 Thread Kent Overstreet
On Thu, Aug 22, 2013 at 09:32:52AM +0200, Stefan Priebe - Profihost AG wrote: great! Everything seems to work fine now! Except read_dirty always going to negative values after a reboot. That one's fixed in 3.11 :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in

Re: [PATCH 04/22] block: Abstract out bvec iterator

2013-08-13 Thread Kent Overstreet
On Tue, Aug 13, 2013 at 10:03:04AM -0400, Ed Cashin wrote: On Aug 9, 2013, Ed Cashin wrote: On Aug 8, 2013, at 9:05 PM, Kent Overstreet wrote: ... It's in the for-jens branch now. Just examining the patches, I like the way it cleans up the aoe code. I had a question about a new

Re: [PATCH] idr: Document ida tree sections

2013-08-13 Thread Kent Overstreet
On Fri, Aug 09, 2013 at 10:57:56AM -0400, Tejun Heo wrote: Hello, On Wed, Aug 07, 2013 at 01:51:17PM -0700, Kent Overstreet wrote: + * So if the max section size is 64k, that's ~4096 sections, with 8 byte + * pointers that's a little over 32k for the pointers to sections

Re: [PATCH] idr: Document ida tree sections

2013-08-13 Thread Kent Overstreet
On Tue, Aug 13, 2013 at 06:19:28PM -0400, Tejun Heo wrote: Hello, On Tue, Aug 13, 2013 at 03:13:08PM -0700, Kent Overstreet wrote: If you're convinced this is a real issue though - how about It is a real issue. Large order allocation is fine for optimization but shouldn't be depended

Re: [PATCH] idr: Document ida tree sections

2013-08-13 Thread Kent Overstreet
On Fri, Aug 09, 2013 at 10:57:56AM -0400, Tejun Heo wrote: Hello, On Wed, Aug 07, 2013 at 01:51:17PM -0700, Kent Overstreet wrote: + * So if the max section size is 64k, that's ~4096 sections, with 8 byte + * pointers that's a little over 32k for the pointers to sections

Re: [PATCH] idr: Document ida tree sections

2013-08-13 Thread Kent Overstreet
On Tue, Aug 13, 2013 at 06:44:28PM -0400, Tejun Heo wrote: Hello, Kent. On Tue, Aug 13, 2013 at 03:27:59PM -0700, Kent Overstreet wrote: It's only naturally a radix tree problem _if_ you require sparseness. Well, it's not necessarily about requiring it but more about surviving

Re: [PATCH] idr: Document ida tree sections

2013-08-13 Thread Kent Overstreet
On Tue, Aug 13, 2013 at 07:22:11PM -0400, Tejun Heo wrote: Hello, On Tue, Aug 13, 2013 at 03:59:27PM -0700, Kent Overstreet wrote: Well, it's not necessarily about requiring it but more about surviving it with some grace when things don't go as expected, which is an important

[GIT PULL] bcache fixes for 3.11

2013-08-14 Thread Kent Overstreet
the label through sysfs Geert Uytterhoeven (1): bcache: Correct printf()-style format length modifier Kent Overstreet (4): bcache: Fix a dumb journal discard bug bcache: Fix for when no journal entries are found bcache: Fix a writeback performance regression bcache: Fix

Re: [PATCH] idr: Document ida tree sections

2013-08-14 Thread Kent Overstreet
On Tue, Aug 13, 2013 at 07:59:47PM -0400, Tejun Heo wrote: Hey, Kent. On Tue, Aug 13, 2013 at 04:51:33PM -0700, Kent Overstreet wrote: Should probably be almost as good, yeah... in theory, but the space efficiency still isn't going to be as good, and it'll probably be more code

Re: [GIT PULL] bcache fixes for 3.11

2013-08-15 Thread Kent Overstreet
On Thu, Aug 15, 2013 at 08:43:22AM +0200, Stefan Priebe - Profihost AG wrote: Am 15.08.2013 00:59, schrieb Kent Overstreet: Jens, here's the latest bcache fixes. Some urgent stuff in here: The following changes since commit 79826c35eb99cd3c0873b8396f45fa26c87fb0b0: bcache

Re: [PATCH 4/4] idr: Percpu ida

2013-06-18 Thread Kent Overstreet
On Tue, Jun 18, 2013 at 02:14:53PM +, Christoph Lameter wrote: On Mon, 17 Jun 2013, Kent Overstreet wrote: +static inline unsigned alloc_local_tag(struct percpu_ida *pool, + struct percpu_ida_cpu *tags) +{ + int tag = -ENOSPC; + + spin_lock

[PATCH v2] lib/idr.c rewrite, percpu ida/tag allocator

2013-06-18 Thread Kent Overstreet
This is the second iteration of patches 1-4 - there's only been a few trivial bugfixes for those. The rest is the idr rewrite - I reimplemented it on top of the new ida implementation and the existing radix tree implementation. Patch series is available in my git repo:

[PATCH 03/10] idr: Rewrite ida

2013-06-18 Thread Kent Overstreet
to leave the implementation as is until if and when this becomes an issue. Signed-off-by: Kent Overstreet koverstr...@google.com Cc: Andrew Morton a...@linux-foundation.org Cc: Tejun Heo t...@kernel.org Cc: Stephen Rothwell s...@canb.auug.org.au Cc: Fengguang Wu fengguang...@intel.com --- include

[PATCH 06/10] idr: Rename idr_get_next() - idr_find_next()

2013-06-18 Thread Kent Overstreet
get() implies taking a ref or sometimes an allocation, which this function definitely does not do - rename it to something more sensible. Signed-off-by: Kent Overstreet koverstr...@google.com Cc: Andrew Morton a...@linux-foundation.org Cc: Tejun Heo t...@kernel.org --- drivers/block/drbd

[PATCH 05/10] idr: Kill old deprecated idr interfaces

2013-06-18 Thread Kent Overstreet
The deprecated idr interfaces don't have any in kernel users, so let's delete them as prep work for the idr rewrite. Signed-off-by: Kent Overstreet koverstr...@google.com Cc: Andrew Morton a...@linux-foundation.org Cc: Tejun Heo t...@kernel.org --- include/linux/idr.h | 63

[PATCH 08/10] idr: Reimplement idr on top of ida/radix trees

2013-06-18 Thread Kent Overstreet
users to the new interface. Signed-off-by: Kent Overstreet koverstr...@google.com Cc: Andrew Morton a...@linux-foundation.org Cc: Tejun Heo t...@kernel.org --- include/linux/idr.h | 157 - init/main.c | 1 - lib/idr.c | 896

[PATCH 04/10] idr: Percpu ida

2013-06-18 Thread Kent Overstreet
not a correctness issue, so I don't think it's worth the extra code. Signed-off-by: Kent Overstreet koverstr...@google.com Cc: Tejun Heo t...@kernel.org Cc: Oleg Nesterov o...@redhat.com Cc: Christoph Lameter c...@linux-foundation.org Cc: Ingo Molnar mi...@redhat.com Cc: Andi Kleen a...@firstfloor.org Cc: Jens

Re: [PATCH] Percpu tag allocator

2013-06-18 Thread Kent Overstreet
On Thu, Jun 13, 2013 at 12:23:39PM -0700, Andrew Morton wrote: On Thu, 13 Jun 2013 12:15:07 -0700 Tejun Heo t...@kernel.org wrote: Oh, I'm sure the current id[r|a] can be improved upon a lot but I'm very skeptical one can reach the level of scalability necessary for, say, pci-e attached

Re: [PATCH 03/10] idr: Rewrite ida

2013-06-19 Thread Kent Overstreet
On Wed, Jun 19, 2013 at 10:40:22AM +0100, Steven Whitehouse wrote: Hi, On Tue, 2013-06-18 at 17:02 -0700, Kent Overstreet wrote: This is a new, from scratch implementation of ida that should be simpler, faster and more space efficient. [...] This does mean that the entire bitmap

[GIT PULL] Bcache fixes for 3.11

2013-07-16 Thread Kent Overstreet
) Dan Carpenter (1): bcache: check for allocation failures Kent Overstreet (7): bcache: Fix a dumb race bcache: Advertise that flushes are supported bcache: Fix a sysfs splat on shutdown bcache: Shutdown fix bcache: Journal replay fix bcache: Fix

[GIT PULL] Bcache changes for 3.12

2013-09-10 Thread Kent Overstreet
) Kent Overstreet (49): bcache: Use blkdev_issue_discard() bcache: Add on error panic/unregister setting bcache: Stripe size isn't necessarily a power of two bcache: Remove unnecessary check in should_split() bcache: Explicitly track btree node's parent

[GIT PULL] bcache changes for 3.13

2013-11-04 Thread Kent Overstreet
) Kees Cook (1): bcache: defensively handle format strings Kent Overstreet (53): bcache: Fixed incorrect order of arguments to bio_alloc_bioset() bcache: Convert bch_data_verify() to immutable biovecs bcache: Fix

Re: [PATCH for-next] dm: fix missing bi_remaining accounting

2013-11-04 Thread Kent Overstreet
On Mon, Nov 04, 2013 at 10:06:00AM -0500, Mikulas Patocka wrote: On Fri, 1 Nov 2013, Jens Axboe wrote: On 11/01/2013 07:59 AM, Mike Snitzer wrote: Add the missing bi_remaining increment, required by the block layer's new bio-chaining code, to both the verity and old snapshot DM

Re: [GIT PULL] bcache changes for 3.13

2013-11-04 Thread Kent Overstreet
On Mon, Nov 04, 2013 at 01:12:17PM -0700, Jens Axboe wrote: On 11/04/2013 01:07 PM, Kent Overstreet wrote: Hey Jens, sorry for being late with this - anyways, it's roughly the same set of patches you had queued up before plus a few minor fixes, but it's been rebased onto your for-3.13

[PATCH] Block layer stuff/DIO rewrite prep for 3.14

2013-11-04 Thread Kent Overstreet
Now that immutable biovecs is in, these are the remaining patches required for my DIO rewrite, along with some related cleanup/refactoring. The key enabler is patch 4 - making generic_make_request() handle arbitary sized bios. This takes what was once bio_add_page()'s responsibility and pushes it

[PATCH 2/9] block: submit_bio_wait() conversions

2013-11-04 Thread Kent Overstreet
It was being open coded in a few places. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Joern Engel jo...@logfs.org Cc: Prasad Joshi prasadjoshi.li...@gmail.com --- block/blk-flush.c | 19 +-- fs/logfs/dev_bdev.c | 8 +--- 2 files

[PATCH 9/9] block: Add bio_get_user_pages()

2013-11-04 Thread Kent Overstreet
This replaces some of the code that was in __bio_map_user_iov(), and soon we're going to use this helper in the dio code. Note that this relies on the recent change to make generic_make_request() take arbitrary sized bios - we're not using bio_add_page() here. Signed-off-by: Kent Overstreet k

[PATCH 8/9] bcache: generic_make_request() handles large bios now

2013-11-04 Thread Kent Overstreet
So we get to delete our hacky workaround. Signed-off-by: Kent Overstreet k...@daterainc.com --- drivers/md/bcache/bcache.h| 18 drivers/md/bcache/io.c| 100 +- drivers/md/bcache/journal.c | 4 +- drivers/md/bcache/request.c

[PATCH 7/9] blk-lib.c: generic_make_request() handles large bios now

2013-11-04 Thread Kent Overstreet
shouldn't matter. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk --- block/blk-lib.c | 175 ++-- 1 file changed, 30 insertions(+), 145 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 2da76c9

[PATCH 6/9] mtip32xx: handle arbitrary size bios

2013-11-04 Thread Kent Overstreet
We get a measurable performance increase by handling this in the driver when we're already looping over the biovec, instead of handling it separately in generic_make_request() (or bio_add_page() originally) Signed-off-by: Kent Overstreet k...@daterainc.com --- drivers/block/mtip32xx/mtip32xx.c

[PATCH 1/9] block: Convert various code to bio_for_each_segment()

2013-11-04 Thread Kent Overstreet
With immutable biovecs we don't want code accessing bi_io_vec directly - the uses this patch changes weren't incorrect since they all own the bio, but it makes the code harder to audit for no good reason - also, this will help with multipage bvecs later. Signed-off-by: Kent Overstreet k

[PATCH 3/9] block: Move bouncing to generic_make_request()

2013-11-04 Thread Kent Overstreet
into account bouncing (as it'll already have been done). Also, __blk_recalc_rq_segments() now doesn't have to take into account potential bouncing - it's already been done. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Jiri Kosina jkos...@suse.cz Cc: Asai Thambi S

[PATCH 4/9] block: Make generic_make_request handle arbitrary sized bios

2013-11-04 Thread Kent Overstreet
are convenient, and more importantly stacked drivers don't have to deal with both their own bio size limitations and the limitations of the (potentially multiple) devices underneath them. In the future this will let us delete merge_bvec_fn and a bunch of other code. Signed-off-by: Kent Overstreet k

[PATCH 5/9] block: Gut bio_add_page(), kill bio_add_pc_page()

2013-11-04 Thread Kent Overstreet
Since generic_make_request() can now handle arbitrary size bios, all we have to do is make sure the bvec array doesn't overflow. Signed-off-by: Kent Overstreet k...@daterainc.com --- drivers/scsi/osd/osd_initiator.c | 5 +- drivers/target/target_core_pscsi.c | 5 +- fs/bio.c

Re: [dm-devel] [PATCH 4/9] block: Make generic_make_request handle arbitrary sized bios

2013-11-04 Thread Kent Overstreet
On Mon, Nov 04, 2013 at 03:56:52PM -0800, Mike Christie wrote: On 11/04/2013 03:36 PM, Kent Overstreet wrote: @@ -1822,6 +1820,14 @@ void generic_make_request(struct bio *bio) */ blk_queue_bounce(q, bio); + if (!blk_queue_largebios(q

[PATCH] block: Revert bio_clone() default behaviour

2013-11-05 Thread Kent Overstreet
at bi_vcnt at all for bios it does not own. However, this patch _shouldn't_ cause any regressions because of this since we're reverting back to the old bio_clone() behaviour. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Chris Mason chris.ma...@fusionio.com Cc

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-05 Thread Kent Overstreet
On Tue, Nov 05, 2013 at 09:02:19PM -0800, Olof Johansson wrote: Hi, On Tue, Nov 5, 2013 at 7:48 PM, Kent Overstreet k...@daterainc.com wrote: Chris, Olaf, can you two in particular test this? I have tested the bounce buffer code (and bcache), but Jens told me today there was an md bug

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
On Wed, Nov 06, 2013 at 11:11:30AM -0500, Chris Mason wrote: Quoting Kent Overstreet (2013-11-05 22:48:41) This patch reverts the default behaviour introduced by 9fc6286f347d00528adcdcf12396d220f47492ed - bio_clone_biovec() no clonger shares the source bio's biovec, cloning the biovec

Re: block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
for which reported problem(s) are being fixed by this patch. On Tue, Nov 05 2013 at 10:48pm -0500, Kent Overstreet k...@daterainc.com wrote: This patch reverts the default behaviour introduced by 9fc6286f347d00528adcdcf12396d220f47492ed - bio_clone_biovec() no clonger shares the source bio's

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
On Wed, Nov 06, 2013 at 03:22:36PM -0500, Chris Mason wrote: Quoting Kent Overstreet (2013-11-06 15:02:22) Yup - that should actually be safe for all the existing bio_clone() users actually, I audited all of them - because normally you're not going to complete the original bio until

[PATCH] block/s390: Fixes for Convert bio_for_each_segment() to bvec_iter

2013-11-06 Thread Kent Overstreet
5d1f127c3e0c57d64ce75ee04a0db2b40a3e21df was incomplete - this just converts s390's drivers. Signed-off-by: Kent Overstreet k...@daterainc.com --- I was poking around a bit, and it looks like there's nothing really preventing these drivers from building on non s390 except that a bunch of headers

Re: [PATCH] block: Revert bio_clone() default behaviour

2013-11-06 Thread Kent Overstreet
On Wed, Nov 06, 2013 at 04:25:45PM -0500, Chris Mason wrote: Quoting Kent Overstreet (2013-11-06 15:57:34) On Wed, Nov 06, 2013 at 03:22:36PM -0500, Chris Mason wrote: Quoting Kent Overstreet (2013-11-06 15:02:22) [ ... nods, thanks! ... ] OTOH - with regards to just the ordering

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 11:17:22AM -0800, Olof Johansson wrote: On Sat, Nov 2, 2013 at 1:50 PM, Dave Kleikamp dave.kleik...@oracle.com wrote: On 11/01/2013 03:53 PM, Jens Axboe wrote: On 11/01/2013 02:41 PM, Dave Kleikamp wrote: On 11/01/2013 03:27 PM, Jens Axboe wrote: On 11/01/2013

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 01:20:26PM -0600, Dave Kleikamp wrote: On 11/02/2013 03:50 PM, Dave Kleikamp wrote: On 11/01/2013 03:53 PM, Jens Axboe wrote: So we've three immediate options: 1) You base it on top of the block tree 2) I carry the loop updates 3) You hand Stephen a merge

Re: [PATCH 1/9] block: Convert various code to bio_for_each_segment()

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 12:26:30PM +0100, Jan Kara wrote: On Mon 04-11-13 15:36:19, Kent Overstreet wrote: With immutable biovecs we don't want code accessing bi_io_vec directly - the uses this patch changes weren't incorrect since they all own the bio, but it makes the code harder to audit

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Fri, Nov 08, 2013 at 12:53:07PM +1100, Stephen Rothwell wrote: Hi all, On Thu, 07 Nov 2013 18:04:57 -0600 Dave Kleikamp dave.kleik...@oracle.com wrote: Can you please drop the aio-direct tree for the time being? OK, I was afraid of this, but, yes, I can drop it. I am not quite

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 11:33:24PM -0800, Christoph Hellwig wrote: The changes for direct I/O from kernel space have been in for a long time, and they are blocking multiple consumers of the interface from getting submitted for about a year now. Even if the guts of the direct-io code will get

Re: linux-next: manual merge of the block tree with the tree

2013-11-07 Thread Kent Overstreet
On Thu, Nov 07, 2013 at 11:44:45PM -0800, Christoph Hellwig wrote: On Thu, Nov 07, 2013 at 11:39:59PM -0800, Kent Overstreet wrote: On Thu, Nov 07, 2013 at 11:33:24PM -0800, Christoph Hellwig wrote: The changes for direct I/O from kernel space have been in for a long time

Re: linux-next: manual merge of the block tree with the tree

2013-11-08 Thread Kent Overstreet
On Fri, Nov 08, 2013 at 12:02:21AM -0800, Christoph Hellwig wrote: On Thu, Nov 07, 2013 at 11:56:17PM -0800, Kent Overstreet wrote: So, I don't think the iov_iter stuff is the right approach for solving the loop issue; it's an ugly hack and after immutable biovecs we're pretty close

Re: linux-next: manual merge of the block tree with the tree

2013-11-08 Thread Kent Overstreet
On Fri, Nov 08, 2013 at 12:32:51AM -0800, Christoph Hellwig wrote: On Fri, Nov 08, 2013 at 12:17:37AM -0800, Kent Overstreet wrote: The core issue isn't whether the IO is going to a block based filesystem (but thanks for pointing out that that's not necessarily true!) but whether we want

Re: [PATCH 16/22] dm: Refactor for new bio cloning/splitting

2013-10-02 Thread Kent Overstreet
On Sat, Sep 28, 2013 at 12:59:09AM -0400, Mike Snitzer wrote: Hey Kent, I haven't been able to pinpoint the issue yet, but using your for-jens branch, if I create a dm-thin volume with this lvm command: lvcreate -L20G -V20G -T vg/pool --name thinlv and try to format /dev/vg/thinlv with

Re: [PATCH 16/22] dm: Refactor for new bio cloning/splitting

2013-10-03 Thread Kent Overstreet
On Wed, Oct 02, 2013 at 11:23:21PM -0400, Mike Snitzer wrote: Great, thanks for finding this, I'll test and review the code further. Cool - let me know if you find anything else (how thoroughly do you think you've tested it so far?) -- To unsubscribe from this list: send the line unsubscribe

[PATCH] block: submit_bio_wait() conversions

2013-11-23 Thread Kent Overstreet
It was being open coded in a few places. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Joern Engel jo...@logfs.org Cc: Prasad Joshi prasadjoshi.li...@gmail.com Cc: Neil Brown ne...@suse.de Cc: Chris Mason chris.ma...@fusionio.com --- block/blk-flush.c

[GIT PULL] Immutable biovecs

2013-11-25 Thread Kent Overstreet
changes up to 4b1faf931650d4a35b2a570318862821d6a962e3: block: Kill bio_pair_split() (2013-11-23 22:33:57 -0800) Kent Overstreet (25): block: submit_bio_wait() conversions block: Convert various code to bio_for_each_segment

[PATCH] Make generic_make_request() handle arbitrary size bios

2013-11-25 Thread Kent Overstreet
This builds off of immutable biovecs; it's a small enough patch series that I'd like to try to get it into 3.14 but it is a pretty significant change in behaviour for the block layer so it should definitely be considered separately from that series. What the series does is pretty simple - like

[PATCH 1/5] block: Make generic_make_request handle arbitrary sized bios

2013-11-25 Thread Kent Overstreet
) * brd_make_request (ramdisk - drivers/block/brd.c) * loop_make_request * null_queue_bio * bcache's make_request fns Some others are almost certainly safe to remove now, but will be left for future patches. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Neil

[PATCH 4/5] blk-lib.c: generic_make_request() handles large bios now

2013-11-25 Thread Kent Overstreet
shouldn't matter. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk --- block/blk-lib.c | 175 ++-- 1 file changed, 30 insertions(+), 145 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 2da76c9

[PATCH 2/5] mtip32xx: handle arbitrary size bios

2013-11-25 Thread Kent Overstreet
We get a measurable performance increase by handling this in the driver when we're already looping over the biovec, instead of handling it separately in generic_make_request() (or bio_add_page() originally) Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Asai

[PATCH 3/5] block: Gut bio_add_page()

2013-11-25 Thread Kent Overstreet
Since generic_make_request() can now handle arbitrary size bios, all we have to do is make sure the bvec array doesn't overflow. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk --- fs/bio.c | 137 ++- 1

[PATCH 5/5] bcache: generic_make_request() handles large bios now

2013-11-25 Thread Kent Overstreet
So we get to delete our hacky workaround. Signed-off-by: Kent Overstreet k...@daterainc.com --- drivers/md/bcache/bcache.h| 18 drivers/md/bcache/io.c| 100 +- drivers/md/bcache/journal.c | 4 +- drivers/md/bcache/request.c

Re: GPF in aio_migratepage

2013-11-25 Thread Kent Overstreet
On Tue, Nov 26, 2013 at 01:01:32AM -0500, Dave Jones wrote: On Mon, Nov 25, 2013 at 10:26:45PM -0500, Dave Jones wrote: Hi Kent, I hit the GPF below on a tree based on 8e45099e029bb6b369b27d8d4920db8caff5ecce which has your commit e34ecee2ae791df674dfb466ce40692ca6218e43 (aio:

Re: [PATCH 1/5] block: Make generic_make_request handle arbitrary sized bios

2013-11-25 Thread Kent Overstreet
On Mon, Nov 25, 2013 at 10:09:21PM -0800, Christoph Hellwig wrote: + q-bio_split = bioset_create(4, 0); + if (!q-bio_split) + goto fail_id; How did we arrive at a mempool size of 4 to make sure we can always make progress with arbitrarily sized bios? Shouldn't we document

Re: [PATCH 2/5] mtip32xx: handle arbitrary size bios

2013-11-25 Thread Kent Overstreet
On Mon, Nov 25, 2013 at 10:10:18PM -0800, Christoph Hellwig wrote: On Mon, Nov 25, 2013 at 02:30:30PM -0800, Kent Overstreet wrote: We get a measurable performance increase by handling this in the driver when we're already looping over the biovec, instead of handling it separately

[PATCH] block: Silence spurious compiler warnings

2013-11-26 Thread Kent Overstreet
From 46e7081430f5f483906f496733a23f8e9d898879 Mon Sep 17 00:00:00 2001 From: Kent Overstreet k...@daterainc.com Date: Tue, 26 Nov 2013 16:36:49 -0800 Subject: [PATCH] block: Silence spurious compiler warnings Signed-off-by: Kent Overstreet k...@daterainc.com --- On Tue, Nov 26, 2013 at 12:02

Re: [GIT PULL] Immutable biovecs

2013-11-26 Thread Kent Overstreet
On Mon, Nov 25, 2013 at 10:05:58PM -0800, Christoph Hellwig wrote: On Mon, Nov 25, 2013 at 01:52:16PM -0800, Kent Overstreet wrote: Jens - here's immutable biovecs, rebased and ready for 3.14. Changes since the last version of the series: Can you do a resend of the patch series to all

[PATCH 05/25] dm: Use bvec_iter for dm_bio_record()

2013-11-26 Thread Kent Overstreet
This patch doesn't itself have any functional changes, but immutable biovecs are going to add a bi_bvec_done member to bi_iter, which will need to be saved too here. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Alasdair Kergon a...@redhat.com Cc: dm-de...@redhat.com Reviewed-by: Mike

[PATCH 06/25] block: Convert bio_iovec() to bvec_iter

2013-11-26 Thread Kent Overstreet
uses are all going to be removed, but we'll need the functionality from immutable to remove them - so for now rename the existing bio_iovec() - __bio_iovec(), and it'll be removed in a couple patches. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Ed L. Cashin

[PATCH 01/25] block: submit_bio_wait() conversions

2013-11-26 Thread Kent Overstreet
It was being open coded in a few places. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Joern Engel jo...@logfs.org Cc: Prasad Joshi prasadjoshi.li...@gmail.com Cc: Neil Brown ne...@suse.de Cc: Chris Mason chris.ma...@fusionio.com Acked-by: NeilBrown ne

[PATCH 03/25] bcache: Kill unaligned bvec hack

2013-11-26 Thread Kent Overstreet
Bcache has a hack to avoid cloning the biovec if it's all full pages - but with immutable biovecs coming this won't be necessary anymore. For now, we remove the special case and always clone the bvec array so that the immutable biovec patches are simpler. Signed-off-by: Kent Overstreet k

[PATCH 18/25] rbd: Refactor bio cloning

2013-11-26 Thread Kent Overstreet
Now that we've got drivers converted to the new immutable bvec primitives, bio splitting becomes much easier - this is how the new bio_split() will work. (Someone more familiar with the ceph code could probably use bio_clone_fast() instead of bio_clone() here). Signed-off-by: Kent Overstreet k

[PATCH 20/25] block: Don't save/copy bvec array anymore

2013-11-26 Thread Kent Overstreet
Now that drivers have been converted to the bvec_iter primitives, they shouldn't be modifying the biovec anymore and thus saving it is unnecessary - code that was previously making a backup of the bvec array can now just save bio-bi_iter. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens

[PATCH 24/25] block: Introduce new bio_split()

2013-11-26 Thread Kent Overstreet
to have clearly documented semantics anyways.) Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Martin K. Petersen martin.peter...@oracle.com Cc: Matthew Wilcox matthew.r.wil...@intel.com Cc: Keith Busch keith.bu...@intel.com Cc: Vishal Verma vishal.l.ve

[PATCH 19/25] dm: Refactor for new bio cloning/splitting

2013-11-26 Thread Kent Overstreet
never be modifying it. Also kill bio_sector_offset(), dm was the only user and it doesn't make much sense anymore. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Alasdair Kergon a...@redhat.com Cc: dm-de...@redhat.com Reviewed-by: Mike Snitzer snit

[PATCH 22/25] block: Generic bio chaining

2013-11-26 Thread Kent Overstreet
is making bio cloning cheaper. But for now, we add a bio_endio_nodec() for these cases. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk --- drivers/md/bcache/io.c | 2 +- drivers/md/dm-cache-target.c | 6 drivers/md/dm-snap.c | 1 + drivers/md/dm

[PATCH 23/25] block: Rename bio_split() - bio_pair_split()

2013-11-26 Thread Kent Overstreet
This is prep work for introducing a more general bio_split(). Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: NeilBrown ne...@suse.de Cc: Alasdair Kergon a...@redhat.com Cc: Lars Ellenberg lars.ellenb...@linbit.com Cc: Peter Osterlund pete...@telia.com Cc

[PATCH 17/25] block: Add bio_clone_fast()

2013-11-26 Thread Kent Overstreet
bio's biovec. Signed-off-by: Kent Overstreet k...@daterainc.com --- drivers/md/bcache/request.c | 8 ++ fs/bio.c| 60 + include/linux/bio.h | 2 ++ 3 files changed, 64 insertions(+), 6 deletions(-) diff --git a/drivers

[PATCH 25/25] block: Kill bio_pair_split()

2013-11-26 Thread Kent Overstreet
Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk --- fs/bio-integrity.c | 45 --- fs/bio.c| 90 - include/linux/bio.h | 30 -- 3 files changed, 165 deletions

[PATCH 21/25] block: Remove bi_idx hacks

2013-11-26 Thread Kent Overstreet
() was a dangerous thing - various drivers would choke on such things. But with immutable biovecs and our new bio splitting that shares the biovecs, submitting partially completed bios has to work (and should work, now that all the drivers have been completed to the new primitives) Signed-off-by: Kent Overstreet k

[PATCH 14/25] ceph: Convert to immutable biovecs

2013-11-26 Thread Kent Overstreet
Now that we've got a mechanism for immutable biovecs - bi_iter.bi_bvec_done - we need to convert drivers to use primitives that respect it instead of using the bvec array directly. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk Cc: Sage Weil s...@inktank.com Cc

[PATCH 15/25] block: Kill bio_iovec_idx(), __bio_iovec()

2013-11-26 Thread Kent Overstreet
to iterate over the bvec array with bio_for_each_segment() which checks against the current value of bi_iter.bi_size. Signed-off-by: Kent Overstreet k...@daterainc.com Cc: Jens Axboe ax...@kernel.dk --- block/blk-merge.c | 13 +++-- include/linux/bio.h | 26 -- 2

<    3   4   5   6   7   8   9   10   11   12   >