[PATCH 24/26] aio: Percpu ioctx refcount

2012-12-03 Thread Kent Overstreet
This just converts the ioctx refcount to the new generic dynamic percpu refcount code. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 27 --- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index bde360d..931606b

[PATCH 23/26] Generic dynamic per cpu refcounting

2012-12-03 Thread Kent Overstreet
enough in practice. Signed-off-by: Kent Overstreet koverstr...@google.com --- include/linux/percpu-refcount.h | 29 +++ lib/Makefile| 2 +- lib/percpu-refcount.c | 164 3 files changed, 194 insertions(+), 1 deletion

[PATCH 11/26] aio: Make aio_put_req() lockless

2012-12-03 Thread Kent Overstreet
. This occurs in two places: kiocb_batch_free(), which is going away in a later patch, and the error path in io_submit_one. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c| 85 + include/linux/aio.h | 4 +-- 2 files changed

[PATCH 22/26] aio: percpu reqs_available

2012-12-03 Thread Kent Overstreet
rounded up to the next page before so this doesn't change the actual semantics. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 99 +++- 1 file changed, 86 insertions(+), 13 deletions(-) diff --git a/fs/aio.c b/fs/aio.c

[PATCH 18/26] aio: Kill batch allocation

2012-12-03 Thread Kent Overstreet
, so that means we can just rip this code out. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c| 117 +++- include/linux/aio.h | 1 - 2 files changed, 15 insertions(+), 103 deletions(-) diff --git a/fs/aio.c b/fs/aio.c

[PATCH 17/26] aio: Change reqs_active to include unreaped completions

2012-12-03 Thread Kent Overstreet
. But - we can improve this quite a bit if we just change reqs_active to mean number of outstanding requests and unreaped completions - that means kiocb allocation doesn't have to look at the ringbuffer, which is a fairly significant win. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs

[PATCH 15/26] aio: Use flush_dcache_page()

2012-12-03 Thread Kent Overstreet
Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 45 + 1 file changed, 17 insertions(+), 28 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index 5eca2a4..e3ca2c0 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -208,33 +208,15 @@ static int

[PATCH 14/26] aio: Make aio_read_evt() more efficient

2012-12-03 Thread Kent Overstreet
the event. v2: Restructure the code so we're not calling prepare_to_wait() until after we've done everything that might block, also got rid of the separate fast path Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 205

[PATCH 13/26] aio: Convert read_events() to hrtimers

2012-12-03 Thread Kent Overstreet
Converting read_events() to prepare_to_wait_exclusive() and hrtimers simplifies it quite a bit. v2: Move finish_wait() call so we're not calling copy_to_user in TASK_INTERRUPTIBLE state Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 77

[PATCH 12/26] aio: Refcounting cleanup

2012-12-03 Thread Kent Overstreet
() calls to call_rcu() (and document them) Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 274 --- 1 file changed, 120 insertions(+), 154 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index 4c9a5bf..7b75590 100644 --- a/fs

[PATCH 02/26] aio: remove dead code from aio.h

2012-12-03 Thread Kent Overstreet
From: Zach Brown z...@redhat.com Signed-off-by: Zach Brown z...@redhat.com Signed-off-by: Kent Overstreet koverstr...@google.com --- include/linux/aio.h | 24 1 file changed, 24 deletions(-) diff --git a/include/linux/aio.h b/include/linux/aio.h index 31ff6db..b46a09f

[PATCH 10/26] aio: do fget() after aio_get_req()

2012-12-03 Thread Kent Overstreet
aio_get_req() will fail if we have the maximum number of requests outstanding, which depending on the application may not be uncommon. So avoid doing an unnecessary fget(). Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 22 +- 1 file changed, 9

[PATCH 09/26] aio: dprintk() - pr_debug()

2012-12-03 Thread Kent Overstreet
Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 57 - 1 file changed, 24 insertions(+), 33 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index 35a2153e..4cceba1 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -8,6 +8,8

[PATCH 07/26] aio: kiocb_cancel()

2012-12-03 Thread Kent Overstreet
Minor refactoring, to get rid of some duplicated code v2: Fix return value for NULL kiocb, so it matches old code; change synchronization to use xchg() instead of a bit in ki_flags, so we can get rid of ki_flags. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 72

[PATCH 06/26] aio: Kill return value of aio_complete()

2012-12-03 Thread Kent Overstreet
Nothing used the return value, and it probably wasn't possible to use it safely for the locked versions (aio_complete(), aio_put_req()). Just kill it. Acked-by: Zach Brown z...@redhat.com Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c| 19

[PATCH 05/26] char: add aio_{read,write} to /dev/{null,zero}

2012-12-03 Thread Kent Overstreet
From: Zach Brown z...@redhat.com These are handy for measuring the cost of the aio infrastructure with operations that do very little and complete immediately. Signed-off-by: Zach Brown z...@redhat.com Signed-off-by: Kent Overstreet koverstr...@google.com --- drivers/char/mem.c | 35

[PATCH 04/26] aio: remove retry-based AIO

2012-12-03 Thread Kent Overstreet
of this is the removal of the locking around the unused run list in the submission path. This has only been compiled. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 348 -- fs/ocfs2/dlmglue.c| 2 +- fs

[PATCH 01/26] mm: remove old aio use_mm() comment

2012-12-03 Thread Kent Overstreet
From: Zach Brown z...@redhat.com use_mm() is used in more places than just aio. There's no need to mention callers when describing the function. Signed-off-by: Zach Brown z...@redhat.com Signed-off-by: Kent Overstreet koverstr...@google.com --- mm/mmu_context.c | 3 --- 1 file changed, 3

Re: Improving AIO cancellation

2013-02-08 Thread Kent Overstreet
On Fri, Feb 08, 2013 at 09:38:11AM -0800, Zach Brown wrote: The draft implementation will look like this. struct bio should have some way to get current status of kiocb that generated bio. So we add a pointer to bool flag. struct bio { bool *cancelled; } in async DIO codepath

Re: [PATCH 23/32] Generic dynamic per cpu refcounting

2013-02-08 Thread Kent Overstreet
On Fri, Feb 08, 2013 at 06:44:08AM -0800, Tejun Heo wrote: (cc'ing Andrew) On Wed, Dec 26, 2012 at 06:00:02PM -0800, Kent Overstreet wrote: This implements a refcount with similar semantics to atomic_get()/atomic_dec_and_test(), that starts out as just an atomic_t but dynamically

Re: [PATCH 23/32] Generic dynamic per cpu refcounting

2013-02-08 Thread Kent Overstreet
On Fri, Feb 08, 2013 at 03:49:02PM +0100, Jens Axboe wrote: On Fri, Feb 08 2013, Tejun Heo wrote: (cc'ing Andrew) On Wed, Dec 26, 2012 at 06:00:02PM -0800, Kent Overstreet wrote: This implements a refcount with similar semantics to atomic_get()/atomic_dec_and_test(), that starts out

[PATCH][WIP] dio rewrite

2013-02-11 Thread Kent Overstreet
I finally started hacking on the dio code, and it's far from done but it's turning out better than I expected so I thought I'd show off what I've got so far. The end result is _vastly_ simpler - direct-io.c is now ~700 lines, vs. ~1300 previously. dio_submit is almost gone, I'm down to 4 things

Re: [PATCH][WIP] dio rewrite

2013-02-11 Thread Kent Overstreet
On Mon, Feb 11, 2013 at 05:24:13PM -0800, Andi Kleen wrote: On Mon, Feb 11, 2013 at 04:53:26PM -0800, Kent Overstreet wrote: I finally started hacking on the dio code, and it's far from done but it's turning out better than I expected so I thought I'd show off what I've got so far

Re: [LSF/MM TOPIC] Re: [dm-devel] Announcement: STEC EnhanceIO SSD caching software for Linux kernel

2013-02-04 Thread Kent Overstreet
On Fri, Jan 18, 2013 at 04:37:59PM -0500, Mike Snitzer wrote: On Fri, Jan 18 2013 at 4:25pm -0500, Darrick J. Wong darrick.w...@oracle.com wrote: Since Joe is putting together a testing tree to compare the three caching things, what do you all think of having a(nother) session about ssd

Re: next-20130117 - kernel BUG with aio

2013-02-05 Thread Kent Overstreet
On Tue, Feb 05, 2013 at 10:53:00AM -0500, valdis.kletni...@vt.edu wrote: On Thu, 31 Jan 2013 16:37:27 -0800, Kent Overstreet said: On Thu, Jan 31, 2013 at 01:59:52PM -0800, Andrew Morton wrote: Did this get fixed? With the patches I sent you, yes - not seeing a new linux-next tree yet

Re: [RFC] [DONOTAPPLY] [PATCH] enhanceio: STEC EnhanceIO SSD caching software for Linux kernel

2013-02-06 Thread Kent Overstreet
On Thu, Feb 07, 2013 at 06:57:40AM +0800, Amit Kale wrote: -Original Message- From: Michel Lespinasse [mailto:wal...@google.com] Sent: Friday, February 01, 2013 4:58 PM To: Darrick J. Wong Cc: Amit Kale; linux-bcache; device-mapper development; Kent Overstreet; Mike Snitzer

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:50:42PM -0800, Tejun Heo wrote: Hello, Kent. On Mon, Jan 28, 2013 at 01:45:06PM -0800, Kent Overstreet wrote: Ahh. Bias value sounds... hacky (i.e. harder to convince myself it's correct) but I see what you're getting at. I don't think it's that hacky. Just

[PATCH] module: Convert to generic percpu refcounts

2013-01-29 Thread Kent Overstreet
I started screwing aronud just to see how hard a conversion would be and what it'd look like. I _think_ this is complete, but there's enough going on I undoubtedly missed something. Completely untested - builds and that's it. I'm sure it's broken. Deletes almost 100 lines of code though. I like

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Kent Overstreet
On Tue, Jan 29, 2013 at 11:29:04AM -0800, Tejun Heo wrote: Hey, Kent. On Tue, Jan 29, 2013 at 08:39:42AM -0800, Kent Overstreet wrote: Oh, if this is going to be widely used I should probably have a different implementation for archs that don't have atomic64_t in hardware. Don't suppose

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Kent Overstreet
On Tue, Jan 29, 2013 at 12:02:18PM -0800, Tejun Heo wrote: Hello, Kent. On Tue, Jan 29, 2013 at 11:51:41AM -0800, Kent Overstreet wrote: What about overflow? Note that we can have systemetic cases where ref is gotten on one cpu and put on another transferring counts in a specific

Re: next-20130117 - kernel BUG with aio

2013-01-31 Thread Kent Overstreet
On Thu, Jan 31, 2013 at 01:59:52PM -0800, Andrew Morton wrote: On Tue, 22 Jan 2013 16:28:18 -0500 valdis.kletni...@vt.edu wrote: On Tue, 22 Jan 2013 21:43:27 +0800, Hillf Danton said: On Mon, Jan 21, 2013 at 9:24 PM, Valdis Kletnieks valdis.kletni...@vt.edu wrote: Am seeing a

Re: [RFC] [DONOTAPPLY] [PATCH] enhanceio: STEC EnhanceIO SSD caching software for Linux kernel

2013-02-01 Thread Kent Overstreet
On Fri, Feb 1, 2013 at 4:57 PM, Michel Lespinasse wal...@google.com wrote: On Fri, Feb 1, 2013 at 4:44 PM, Darrick J. Wong darrick.w...@oracle.com wrote: This is a patch to migrate STEC's enhanceio driver out of their github repository and into the staging tree. From their README:

Re: [RFC] [DONOTAPPLY] [PATCH] enhanceio: STEC EnhanceIO SSD caching software for Linux kernel

2013-02-01 Thread Kent Overstreet
Also, where's the end user documentation? On Fri, Feb 1, 2013 at 5:23 PM, Kent Overstreet koverstr...@google.com wrote: On Fri, Feb 1, 2013 at 4:57 PM, Michel Lespinasse wal...@google.com wrote: On Fri, Feb 1, 2013 at 4:44 PM, Darrick J. Wong darrick.w...@oracle.com wrote: This is a patch

[PATCH 4/4] block: Bio cancellation

2013-02-28 Thread Kent Overstreet
the bio_clone() users would need to be auditied to make sure that it's safe. We can't blindly make e.g. raid5 writes cancellable without the knowledge of the md code. Initial patch by Anatol Pomazau (ana...@google.com). Signed-off-by: Kent Overstreet koverstr...@google.com --- block/blk-core.c

[PATCH 1/4] aio: io_cancel() no longer returns the io_event

2013-02-28 Thread Kent Overstreet
-ki_cancel() returned success. Also tweak the refcounting in kiocb_cancel() to make more sense. Signed-off-by: Kent Overstreet koverstr...@google.com --- drivers/usb/gadget/inode.c | 3 +-- fs/aio.c | 24 ++-- include/linux/aio.h| 2 +- 3 files

[PATCH 3/4] direct-io: Set dio-io_error directly

2013-02-28 Thread Kent Overstreet
The way io errors are returned in the dio code was rather convoluted, and also meant that the specific error code was lost. We need to return the actual error so that for cancellation we can pass up -ECANCELED. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/direct-io.c | 35

[PATCH 2/4] aio: Allow cancellation without a cancel callback

2013-02-28 Thread Kent Overstreet
Prep work for bio cancellation. At least initially, we don't want to implement a callback that has to chase down all the state (multiple bios/requests) associated with the iocb; a simple flag will suffice. Signed-off-by: Kent Overstreet koverstr...@google.com --- drivers/usb/gadget/inode.c | 7

Re: next-20130117 - kernel BUG with aio

2013-01-24 Thread Kent Overstreet
On Thu, Jan 24, 2013 at 12:22:21PM -0500, valdis.kletni...@vt.edu wrote: On Wed, 23 Jan 2013 20:10:03 +0800, Hillf Danton said: Try again? --- --- a/fs/aio.c Tue Jan 22 21:37:54 2013 +++ b/fs/aio.c Wed Jan 23 20:06:14 2013 Now seeing this: [ 2941.495370] [

Re: next-20130117 - kernel BUG with aio

2013-01-24 Thread Kent Overstreet
On Thu, Jan 24, 2013 at 01:27:59PM -0800, Andrew Morton wrote: On Thu, 24 Jan 2013 13:18:50 -0800 Kent Overstreet koverstr...@google.com wrote: So, Andrew - that smoosh struct kiocb patch should just be dropped, even if I fixed that issue clearly the idea is a lot less safe than I

[PATCH 1/3] aio: Fix a null pointer deref in batch_complete_aio

2013-01-24 Thread Kent Overstreet
The batch completion code was trying to be a bit too clever, and skip checking ctx where it couldn't be NULL - but that broke if a kiocb had been cancelled. Move the check to kioctx_ring_unlock(). Reported-by: Valdis Kletnieks valdis.kletni...@vt.edu Signed-off-by: Kent Overstreet koverstr

[PATCH 2/3] aio-kill-ki_retry-fix-fix

2013-01-24 Thread Kent Overstreet
The aio: kill ki-retry patch was assuming that we didn't touch struct kiocb after passing it off to something that would call aio_complete() - which was wrong. So, revert the refcounting changes. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c | 6 -- 1 file changed, 4

[PATCH 3/3] aio-use-cancellation-list-lazily-fix

2013-01-24 Thread Kent Overstreet
synchronizing with aio_complete() which isn't taking any locks. Signed-off-by: Kent Overstreet koverstr...@google.com --- fs/aio.c| 32 ++-- include/linux/aio.h | 11 +++ 2 files changed, 33 insertions(+), 10 deletions(-) diff --git a/fs/aio.c b/fs

Re: next-20130117 - kernel BUG with aio

2013-01-24 Thread Kent Overstreet
On Thu, Jan 24, 2013 at 01:27:59PM -0800, Andrew Morton wrote: Please also take a look at Jan's recent http://www.spinics.net/lists/linux-fsdevel/msg61738.html and have a think about how this plays with your patchset. I can't think of any possible interactions - none of my aio stuff messes

Re: next-20130117 - kernel BUG with aio

2013-01-24 Thread Kent Overstreet
On Thu, Jan 24, 2013 at 02:25:37PM -0800, Zach Brown wrote: No, I didn't see that bug until after I'd fixed the other three, but as far as I can tell everything's fixed with the patches I'm about to mail out - my test VM has been running for the past two days without errors, it's kill

Re: [dm-devel] Announcement: STEC EnhanceIO SSD caching software for Linux kernel

2013-01-24 Thread Kent Overstreet
On Thu, Jan 17, 2013 at 03:39:40AM -0800, Kent Overstreet wrote: Suppose I could fill out the bcache version... On Thu, Jan 17, 2013 at 05:52:00PM +0800, Amit Kale wrote: 11. Error conditions - Handling power failures, intermittent and permanent device failures. Power failures

Re: [dm-devel] Announcement: STEC EnhanceIO SSD caching software for Linux kernel

2013-01-24 Thread Kent Overstreet
On Fri, Jan 18, 2013 at 05:08:37PM +0800, Amit Kale wrote: From: Jason Warr [mailto:ja...@warr.net] On 01/17/2013 11:53 AM, Amit Kale wrote: 9. Performance - Throughput is generally most important. Latency is also one more performance comparison point. Performance under

Re: [PATCH 23/32] Generic dynamic per cpu refcounting

2013-01-24 Thread Kent Overstreet
On Thu, Jan 24, 2013 at 04:51:36PM -0800, Tejun Heo wrote: (cc'ing percpu / rcu crowd) Hello, Kent. On Wed, Dec 26, 2012 at 06:00:02PM -0800, Kent Overstreet wrote: This implements a refcount with similar semantics to atomic_get()/atomic_dec_and_test(), that starts out as just

Re: [PATCH 3/3] aio-use-cancellation-list-lazily-fix

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 03:12:51PM -0800, Andrew Morton wrote: On Fri, 25 Jan 2013 21:30:32 +0800 Hillf Danton dhi...@gmail.com wrote: On Fri, Jan 25, 2013 at 5:43 AM, Kent Overstreet koverstr...@google.com wrote: The cancellation changes were fubar - we can't cancel a kiocb

Re: [PATCH 23/32] Generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Thu, Jan 24, 2013 at 06:09:43PM -0800, Tejun Heo wrote: Hello, again. On Thu, Jan 24, 2013 at 06:03:40PM -0800, Tejun Heo wrote: Yeah, if we're aiming to replace refcnts in file and kobj, dynamic alloc may be justified. Hopefully, the accounting necessary to decide whethre to use

Re: [PATCH 23/32] Generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 04:45:10PM +1030, Rusty Russell wrote: Tejun Heo t...@kernel.org writes: It also implements two stage shutdown, as we need it to tear down the percpu counts. Before dropping the initial refcount, you must call percpu_ref_kill(); this puts the refcount in shutting

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 07:09:41PM +0100, Oleg Nesterov wrote: (add lkml) On 01/24, Kent Overstreet wrote: This has already been on lkml and is in Andrew's tree, Tejun just asked me to send it out again: I'll try to read this code later, just a couple of questions after a quick

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 07:29:24PM +0100, Oleg Nesterov wrote: On 01/25, Oleg Nesterov wrote: +void __percpu_ref_get(struct percpu_ref *ref, bool alloc) +{ + unsigned long pcpu_count; + uint64_t v; + + pcpu_count = ACCESS_ONCE(ref-pcpu_count); + + if

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 08:11:39PM +0100, Oleg Nesterov wrote: On 01/25, Oleg Nesterov wrote: +int percpu_ref_kill(struct percpu_ref *ref) +{ ... + if (status == PCPU_REF_PTR) { + unsigned count = 0, cpu; + + synchronize_rcu(); + +

Re: [PATCH 23/32] Generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 09:59:51AM -0800, Tejun Heo wrote: Hello, Kent. On Mon, Jan 28, 2013 at 09:53:04AM -0800, Kent Overstreet wrote: Like Tejun, I'd prefer to see it always alloc up-front, because it avoids the _noalloc variant (which is backwards: please hand gfp_t, so you don't

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 10:27:37AM -0800, Tejun Heo wrote: Hello, guys. On Mon, Jan 28, 2013 at 10:15:28AM -0800, Kent Overstreet wrote: percpu_ref_kill(); put_and_dsetroy(); And this can race with another holder which drops the last reference, its put_and_dsetroy() can see

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 10:55:52AM -0800, Tejun Heo wrote: Hey, Kent. On Mon, Jan 28, 2013 at 10:49:33AM -0800, Kent Overstreet wrote: Yeah. It'd be really nice if it was doable without synchronize_rcu(), but it'd definitely make get/put heavier. Though, re. close() - considering we

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 12:27:45PM -0800, Tejun Heo wrote: Hey, Kent. On Mon, Jan 28, 2013 at 12:22 PM, Kent Overstreet koverstr...@google.com wrote: Could do that too, but then teardown gets really messy for the user - we need two synchronize_rcu()s: state := dying

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:18:32PM -0800, Tejun Heo wrote: Hello, Kent. On Mon, Jan 28, 2013 at 12:55:40PM -0800, Kent Overstreet wrote: I don't understand why we need two stages. What prevents the killing thread from fetching percpu counters after dying passes one synchronize_sched

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:28:14PM -0800, Tejun Heo wrote: On Mon, Jan 28, 2013 at 01:24:07PM -0800, Kent Overstreet wrote: set dying; synchronize_sched(); collect percpu refs into global atomic_t; put the base ref; After you set state := dying, percpu_ref_put

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:36:22PM -0800, Tejun Heo wrote: On Mon, Jan 28, 2013 at 01:28:14PM -0800, Tejun Heo wrote: But at that point, the operation is already global, so there gotta be a lighter way to synchronize stuff than going through full grace period. ie. You can add a bias value

Re: Bcache v. whatever

2013-01-15 Thread Kent Overstreet
On Tue, Jan 15, 2013 at 11:15:38PM +, James Bottomley wrote: I think md integration would be a bit of a mistake, since md is pretty much only a raid engine at this point in time. There are also a couple of other dm cache targets, so it would be nice to have one rather than three. This

Re: [dm-devel] Announcement: STEC EnhanceIO SSD caching software for Linux kernel

2013-01-17 Thread Kent Overstreet
Suppose I could fill out the bcache version... On Thu, Jan 17, 2013 at 05:52:00PM +0800, Amit Kale wrote: Hi Joe, Kent, [Adding Kent as well since bcache is mentioned below as one of the contenders for being integrated into mainline kernel.] My understanding is that these three caching

Re: [PATCH 1/2] aio: correct calculation of available events

2013-02-13 Thread Kent Overstreet
-by: Kent Overstreet koverstr...@google.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/

Re: [PATCH 2/2] aio: fix kioctx not being freed after cancellation at exit time

2013-02-13 Thread Kent Overstreet
On Wed, Feb 13, 2013 at 12:46:36PM -0500, Benjamin LaHaise wrote: The recent changes overhauling fs/aio.c introduced a bug that results in the kioctx not being freed when outstanding kiocbs are cancelled at exit_aio() time. Specifically, a kiocb that is cancelled has its completion events

Re: [PATCH] aio: only suppress events from cancelled kiocbs if free_ioctx() is in progress

2013-02-13 Thread Kent Overstreet
On Wed, Feb 13, 2013 at 04:52:14PM -0500, Benjamin LaHaise wrote: The io_cancel() syscall allows for cancellation of iocbs in flight to generate a completion event. The current behaviour of batch_complete_aio() is to suppress all completion events. Some types of asynchronous operations

[PATCH][WIP] dio rewrite

2013-02-13 Thread Kent Overstreet
Last posting: http://marc.info/?l=linux-fsdevelm=136063048002755w=2 Got it working and ran some benchmarks. On a high end SSD, doing 4k random reads with fio I got around a 30% increase in throughput. (That was without the aio changes I recently did. With those, throughput was aproximately

[PATCH 02/27] block: Avoid deadlocks with bio allocation by stacking drivers

2013-02-19 Thread Kent Overstreet
-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk Acked-by: Tejun Heo t...@kernel.org Reviewed-by: Muthukumar Ratty mut...@gmail.com --- fs/bio.c| 116 +++- include/linux/bio.h | 9 2 files changed, 123

[PATCH 07/27] md: Convert md_trim_bio() to use bio_advance()

2013-02-19 Thread Kent Overstreet
Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: NeilBrown ne...@suse.de Acked-by: NeilBrown ne...@suse.de --- drivers/md/md.c | 17 - 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index

[PATCH 12/27] block: Remove some unnecessary bi_vcnt usage

2013-02-19 Thread Kent Overstreet
More prep work for immutable bvecs/effecient bio splitting - usage of bi_vcnt has to be auditing, so getting rid of all the unnecessary usage makes that easier. Plus, bio_segments() is really what this code wanted, as it respects the current value of bi_idx. Signed-off-by: Kent Overstreet

[PATCH 14/27] raid10: Use bio_reset()

2013-02-19 Thread Kent Overstreet
More prep work for immutable bio vecs, mainly getting rid of references to bi_idx. bio_reset was being open coded in a few places. The one in sync_request was a bit nontrivial to convert, so could use some extra eyeballs. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax

[PATCH 27/27] bio-integrity: Add explicit field for owner of bip_buf

2013-02-19 Thread Kent Overstreet
This was the only real user of BIO_CLONED, which didn't have very clear semantics. Convert to its own flag so we can get rid of BIO_CLONED. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: Martin K. Petersen martin.peter...@oracle.com --- fs/bio

[PATCH 26/27] block: Add an explicit bio flag for bios that own their bvec

2013-02-19 Thread Kent Overstreet
This is for the new bio splitting code. When we split a bio, if the split occured on a bvec boundry we reuse the bvec for the new bio. But that means bio_free() can't free it, hence the explicit flag. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk Acked

[PATCH 10/27] block: Change bio_split() to respect the current value of bi_idx

2013-02-19 Thread Kent Overstreet
partial completed bios to be submitted, which along with immutable biovecs enables effecient bio splitting. Some of the callers were (double) checking that bios could be split, so update their checks too. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC

[PATCH 11/27] block: Remove bi_idx references

2013-02-19 Thread Kent Overstreet
For immutable bvecs, all bi_idx usage needs to be audited - so here we're removing all the unnecessary uses. Most of these are places where it was being initialized on a bio that was just allocated, a few others are conversions to standard macros. Signed-off-by: Kent Overstreet koverstr

[PATCH 25/27] block: Add bio_alloc_pages()

2013-02-19 Thread Kent Overstreet
More utility code to replace stuff that's getting open coded. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: NeilBrown ne...@suse.de --- drivers/md/raid1.c | 16 +++- fs/bio.c| 28 include/linux/bio.h

[PATCH 24/27] block: Convert some code to bio_for_each_segment_all()

2013-02-19 Thread Kent Overstreet
pass in a struct bio_vec (not a pointer) which is updated with what the current biovec would be (taking into account bi_bvec_done and bi_size). So because of that it's more worthwhile to be consistent about bio_for_each_segment()/bio_for_each_segment_all() usage. Signed-off-by: Kent Overstreet

[PATCH 23/27] block: Add bio_for_each_segment_all()

2013-02-19 Thread Kent Overstreet
implementation. This will also help document the intent of code that's using it - bio_for_each_segment_all() is only legal to use for code that owns the bio. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: Neil Brown ne...@suse.de CC: Boaz Harrosh bharr

[PATCH 13/27] block: Add submit_bio_wait(), remove from md

2013-02-19 Thread Kent Overstreet
Random cleanup - this code was duplicated and it's not really specific to md. Also added the ability to return the actual error code. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: NeilBrown ne...@suse.de Acked-by: Tejun Heo t...@kernel.org --- drivers

[PATCH 22/27] bounce: Refactor __blk_queue_bounce to not use bi_io_vec

2013-02-19 Thread Kent Overstreet
- a later patch renames __bio_for_each_segment() - bio_for_each_segment_all(), and documents that bio_for_each_segment_all() is only for code that owns the bio. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk --- mm/bounce.c | 73

[PATCH 21/27] raid1: use bio_copy_data()

2013-02-19 Thread Kent Overstreet
This doesn't really delete any code _yet_, but once immutable bvecs are done we can just delete the rest of the code in that loop. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: NeilBrown ne...@suse.de --- drivers/md/raid1.c | 5 ++--- 1 file changed, 2

[PATCH 20/27] pktcdvd: Use bio_reset() in disabled code to kill bi_idx usage

2013-02-19 Thread Kent Overstreet
In the short term this'll help with code auditing, and if this code ever gets used now it's converted :) Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jiri Kosina jkos...@suse.cz --- drivers/block/pktcdvd.c | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff

[PATCH 19/27] pktcdvd: use bio_copy_data()

2013-02-19 Thread Kent Overstreet
Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: Jiri Kosina jkos...@suse.cz --- drivers/block/pktcdvd.c | 79 - 1 file changed, 12 insertions(+), 67 deletions(-) diff --git a/drivers/block/pktcdvd.c b

[PATCH 18/27] block: Add bio_copy_data()

2013-02-19 Thread Kent Overstreet
This gets open coded quite a bit and it's tricky to get right, so make a generic version and convert some existing users over to it instead. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk --- fs/bio.c| 70

[PATCH 17/27] raid1: Refactor narrow_write_error() to not use bi_idx

2013-02-19 Thread Kent Overstreet
More bi_idx removal. This code was just open coding bio_clone(). This could probably be further improved by using bio_advance() instead of skipping over null pages, but that'd be a larger rework. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: NeilBrown ne

[PATCH 16/27] raid5: use bio_reset()

2013-02-19 Thread Kent Overstreet
Had to shuffle the code around a bit (where bi_rw and bi_end_io were set), but shouldn't really be anything tricky here Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: NeilBrown ne...@suse.de --- drivers/md/raid5.c | 27 +-- 1 file

[PATCH 15/27] raid1: use bio_reset()

2013-02-19 Thread Kent Overstreet
Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: NeilBrown ne...@suse.de --- drivers/md/raid1.c | 24 ++-- 1 file changed, 6 insertions(+), 18 deletions(-) diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 3dbdb3d..e3f98d2

[PATCH 09/27] block: Use bio_sectors() more consistently

2013-02-19 Thread Kent Overstreet
Bunch of places in the code weren't using it where they could be - this'll reduce the size of the patch that puts bi_sector/bi_size/bi_idx into a struct bvec_iter. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: Ed L. Cashin ecas...@coraid.com CC: Nick

[PATCH 04/27] block: Convert integrity to bvec_alloc_bs()

2013-02-19 Thread Kent Overstreet
-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: Martin K. Petersen martin.peter...@oracle.com --- fs/bio-integrity.c | 132 +++- fs/bio.c| 36 ++ include/linux/bio.h | 8 ++-- 3 files changed, 68

[PATCH 08/27] block: Add bio_end_sector()

2013-02-19 Thread Kent Overstreet
Just a little convenience macro - main reason to add it now is preparing for immutable bio vecs, it'll reduce the size of the patch that puts bi_sector/bi_size/bi_idx into a struct bvec_iter. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: Lars Ellenberg

[PATCH 06/27] block: Refactor blk_update_request()

2013-02-19 Thread Kent Overstreet
partial updates, and dumping before and after on various bio/bvec fields when doing a partial update. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk --- block/blk-core.c | 80 +--- 1 file changed, 12 insertions

[PATCH 05/27] block: Add bio_advance()

2013-02-19 Thread Kent Overstreet
This is prep work for immutable bio vecs; we first want to centralize where bvecs are modified. Next two patches convert some existing code to use this function. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk --- fs/bio.c | 41

[PATCH 01/27] block: Reorder struct bio_set

2013-02-19 Thread Kent Overstreet
This is prep work for the next patch, which embeds a struct bio_list in struct bio_set. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk --- include/linux/bio.h | 66 ++--- 1 file changed, 33 insertions(+), 33

[PATCH 00/27] Block cleanups - prep work for immutable bio vecs/dio rewrite

2013-02-19 Thread Kent Overstreet
Jens - this is the patch series I was working on a couple months ago, and then got sidetracked on. I rebased it onto your for-3.9/core branch. Nothing's changed (besides a few random merge conflicts) since the last time I mailed it out - I think all this stuff is ready. Kent Overstreet (27

[PATCH 03/27] block: Fix a buffer overrun in bio_integrity_split()

2013-02-19 Thread Kent Overstreet
vecs - a later patch is going to make more use of this pointer. Signed-off-by: Kent Overstreet koverstr...@google.com CC: Jens Axboe ax...@kernel.dk CC: Martin K. Petersen martin.peter...@oracle.com --- fs/bio-integrity.c | 5 +++-- include/linux/bio.h | 4 +++- 2 files changed, 6 insertions(+), 3

Re: [RFC] Heads up on a series of AIO patchsets

2007-01-02 Thread Kent Overstreet
Any details? Well, one path I tried I couldn't help but post a blog entry about for my friends. I'm not sure it's the direction I'll take with linux- kernel, but the fundamentals are there: the api should be the syscall interface, and there should be no difference between sync and async

Re: [PATCH] virtio-blk: Fix kconfig option

2012-09-06 Thread Kent Overstreet
On Tue, Sep 04, 2012 at 03:53:53PM +0930, Rusty Russell wrote: Kent Overstreet koverstr...@google.com writes: CONFIG_VIRTIO isn't exposed, everything else is supposed to select it instead. This is a slight mis-understanding. It's supposed to be selected by the particular driver

Re: [PATCH] virtio-blk: Fix kconfig option

2012-09-06 Thread Kent Overstreet
On Thu, Sep 06, 2012 at 11:44:03AM +0300, Michael S. Tsirkin wrote: On Thu, Sep 06, 2012 at 12:41:13AM -0700, Kent Overstreet wrote: On Tue, Sep 04, 2012 at 03:53:53PM +0930, Rusty Russell wrote: Kent Overstreet koverstr...@google.com writes: CONFIG_VIRTIO isn't exposed, everything

Re: [PATCH] virtio-blk: Fix kconfig option

2012-09-06 Thread Kent Overstreet
On Thu, Sep 06, 2012 at 12:49:56PM +0300, Michael S. Tsirkin wrote: On Thu, Sep 06, 2012 at 02:25:12AM -0700, Kent Overstreet wrote: On Thu, Sep 06, 2012 at 11:44:03AM +0300, Michael S. Tsirkin wrote: On Thu, Sep 06, 2012 at 12:41:13AM -0700, Kent Overstreet wrote: On Tue, Sep 04, 2012

Re: [PATCH] virtio-blk: Fix kconfig option

2012-09-06 Thread Kent Overstreet
On Thu, Sep 06, 2012 at 01:18:43PM +0300, Michael S. Tsirkin wrote: On Thu, Sep 06, 2012 at 03:02:48AM -0700, Kent Overstreet wrote: On Thu, Sep 06, 2012 at 12:49:56PM +0300, Michael S. Tsirkin wrote: On Thu, Sep 06, 2012 at 02:25:12AM -0700, Kent Overstreet wrote: On Thu, Sep 06, 2012

[PATCH v9 0/9] Block cleanups

2012-09-06 Thread Kent Overstreet
Since v8: Just a few minor comment/patch descriptions changes Kent Overstreet (9): block: Generalized bio pool freeing block: Ues bi_pool for bio_integrity_alloc() dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Add

<    1   2   3   4   5   6   7   8   9   10   >