Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 6:55 PM, Al Viro wrote: > > What would make unlazy_walk() fail? And if it succeeds, you are not > in RCU mode anymore *without* restarting from scratch... I don't see your point. You don't want to be in RCU mode any more. You want to either

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 6:55 PM, Al Viro wrote: > > What would make unlazy_walk() fail? And if it succeeds, you are not > in RCU mode anymore *without* restarting from scratch... I don't see your point. You don't want to be in RCU mode any more. You want to either succeed or fail with

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 06:45:19PM -0700, Linus Torvalds wrote: > It actually does seem to do that, although in an admittedly rather > questionable way. > > I think it should use path_openat() rather than do_filp_open(), but > passing in LOOKUP_RCU to do_filp_open() actually does work: it just >

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 06:45:19PM -0700, Linus Torvalds wrote: > It actually does seem to do that, although in an admittedly rather > questionable way. > > I think it should use path_openat() rather than do_filp_open(), but > passing in LOOKUP_RCU to do_filp_open() actually does work: it just >

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 6:26 PM, Al Viro wrote: > > Umm... You do realize that LOOKUP_RCU in flags does *NOT* guarantee that > it won't block, right? At the very least one would need to refuse to > fall back on non-RCU mode without a full restart. It actually does seem

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 6:26 PM, Al Viro wrote: > > Umm... You do realize that LOOKUP_RCU in flags does *NOT* guarantee that > it won't block, right? At the very least one would need to refuse to > fall back on non-RCU mode without a full restart. It actually does seem to do that, although in

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 06:20:24PM -0700, Linus Torvalds wrote: > On Mon, Mar 14, 2016 at 10:17 AM, Benjamin LaHaise wrote: > > > > I had some time last week to make an aio openat do what it can in > > submit context. The results are an improvement: when openat is handled > > in

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Mar 19, 2016 at 06:20:24PM -0700, Linus Torvalds wrote: > On Mon, Mar 14, 2016 at 10:17 AM, Benjamin LaHaise wrote: > > > > I had some time last week to make an aio openat do what it can in > > submit context. The results are an improvement: when openat is handled > > in submit context

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Mon, Mar 14, 2016 at 10:17 AM, Benjamin LaHaise wrote: > > I had some time last week to make an aio openat do what it can in > submit context. The results are an improvement: when openat is handled > in submit context it completes in about half the time it takes compared > to

Re: aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Mon, Mar 14, 2016 at 10:17 AM, Benjamin LaHaise wrote: > > I had some time last week to make an aio openat do what it can in > submit context. The results are an improvement: when openat is handled > in submit context it completes in about half the time it takes compared > to the round trip

aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Jan 23, 2016 at 03:39:22PM +1100, Dave Chinner wrote: > On Wed, Jan 20, 2016 at 03:07:26PM -0800, Linus Torvalds wrote: ... > > We could do things like that for the name loopkup for openat() too, where > > we could handle the successful RCU loopkup synchronously, but then if we > > fall

aio openat Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Jan 23, 2016 at 03:39:22PM +1100, Dave Chinner wrote: > On Wed, Jan 20, 2016 at 03:07:26PM -0800, Linus Torvalds wrote: ... > > We could do things like that for the name loopkup for openat() too, where > > we could handle the successful RCU loopkup synchronously, but then if we > > fall

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Fri, Jan 22, 2016 at 11:50:24PM -0500, Benjamin LaHaise wrote: > On Sat, Jan 23, 2016 at 03:24:49PM +1100, Dave Chinner wrote: > > On Wed, Jan 20, 2016 at 04:56:30PM -0500, Benjamin LaHaise wrote: > > > On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > > > > Filesystems *must take

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Fri, Jan 22, 2016 at 11:50:24PM -0500, Benjamin LaHaise wrote: > On Sat, Jan 23, 2016 at 03:24:49PM +1100, Dave Chinner wrote: > > On Wed, Jan 20, 2016 at 04:56:30PM -0500, Benjamin LaHaise wrote: > > > On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > > > > Filesystems *must take

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Jan 23, 2016 at 03:24:49PM +1100, Dave Chinner wrote: > On Wed, Jan 20, 2016 at 04:56:30PM -0500, Benjamin LaHaise wrote: > > On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > > > Filesystems *must take locks* in the IO path. We have to serialise > > > against truncate and

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 03:07:26PM -0800, Linus Torvalds wrote: > On Jan 20, 2016 1:46 PM, "Dave Chinner" wrote: > > > > > > > That said, I also agree that it would be interesting to hear what the > > > > performance impact is for existing performance-sensitive users. Could > > > > we make that

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 04:56:30PM -0500, Benjamin LaHaise wrote: > On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > > Filesystems *must take locks* in the IO path. We have to serialise > > against truncate and other operations at some point in the IO path > > (e.g. block mapping vs

Re: [PATCH 07/13] aio: enabled thread based async fsync

On 2016-01-19 19:59:35 -0800, Linus Torvalds wrote: > Are there other users outside of Solace? It would be good to get comments.. PostgreSQL is a potential user of async fdatasync, fsync, sync_file_range and potentially readahead, write, read. First tests with Dave's async fsync/fsync_range are

Re: [PATCH 07/13] aio: enabled thread based async fsync

On 2016-01-12 12:11:28 +1100, Dave Chinner wrote: > On Mon, Jan 11, 2016 at 05:07:23PM -0500, Benjamin LaHaise wrote: > > Enable a fully asynchronous fsync and fdatasync operations in aio using > > the aio thread queuing mechanism. > > > > Signed-off-by: Benjamin LaHaise > > Signed-off-by:

Re: [PATCH 07/13] aio: enabled thread based async fsync

On 2016-01-19 19:59:35 -0800, Linus Torvalds wrote: > Are there other users outside of Solace? It would be good to get comments.. PostgreSQL is a potential user of async fdatasync, fsync, sync_file_range and potentially readahead, write, read. First tests with Dave's async fsync/fsync_range are

Re: [PATCH 07/13] aio: enabled thread based async fsync

On 2016-01-12 12:11:28 +1100, Dave Chinner wrote: > On Mon, Jan 11, 2016 at 05:07:23PM -0500, Benjamin LaHaise wrote: > > Enable a fully asynchronous fsync and fdatasync operations in aio using > > the aio thread queuing mechanism. > > > > Signed-off-by: Benjamin LaHaise

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 04:56:30PM -0500, Benjamin LaHaise wrote: > On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > > Filesystems *must take locks* in the IO path. We have to serialise > > against truncate and other operations at some point in the IO path > > (e.g. block mapping vs

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 03:07:26PM -0800, Linus Torvalds wrote: > On Jan 20, 2016 1:46 PM, "Dave Chinner" wrote: > > > > > > > That said, I also agree that it would be interesting to hear what the > > > > performance impact is for existing performance-sensitive users. Could >

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Sat, Jan 23, 2016 at 03:24:49PM +1100, Dave Chinner wrote: > On Wed, Jan 20, 2016 at 04:56:30PM -0500, Benjamin LaHaise wrote: > > On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > > > Filesystems *must take locks* in the IO path. We have to serialise > > > against truncate and

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > >> > >> Are there other users outside of Solace? It would be good to get comments.. > > > > I know of quite a few storage/db products that use AIO. The most > > recent high

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > Filesystems *must take locks* in the IO path. We have to serialise > against truncate and other operations at some point in the IO path > (e.g. block mapping vs concurrent allocation and/or removal), and > that can only be done sanely

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 03:44:49PM -0500, Benjamin LaHaise wrote: > On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > > >> > > >> Are there other users outside of Solace? It would be good to get > > >> comments.. > > > > >

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > >> > >> Are there other users outside of Solace? It would be good to get comments.. > > > > I know of quite a few storage/db products that use AIO. The most > > recent high

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: >> >> Are there other users outside of Solace? It would be good to get comments.. > > I know of quite a few storage/db products that use AIO. The most > recent high profile project that have been reporting issues with AIO > on XFS is

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Tue, Jan 19, 2016 at 07:59:35PM -0800, Linus Torvalds wrote: > On Fri, Jan 15, 2016 at 12:21 PM, Benjamin LaHaise wrote: > >> > >> I'll have to think about this some more. > > > > Any further thoughts on this after a few days worth of pondering? > > Sorry about the delay, with the merge

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Tue, Jan 19, 2016 at 07:59:35PM -0800, Linus Torvalds wrote: > On Fri, Jan 15, 2016 at 12:21 PM, Benjamin LaHaise wrote: > >> > >> I'll have to think about this some more. > > > > Any further thoughts on this after a few days worth of pondering? > > Sorry about the delay, with

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: >> >> Are there other users outside of Solace? It would be good to get comments.. > > I know of quite a few storage/db products that use AIO. The most > recent high profile project that have been reporting issues with AIO

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > >> > >> Are there other users outside of Solace? It would be good to get comments.. > > > > I know of quite a few storage/db products that use AIO. The

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Thu, Jan 21, 2016 at 08:45:46AM +1100, Dave Chinner wrote: > Filesystems *must take locks* in the IO path. We have to serialise > against truncate and other operations at some point in the IO path > (e.g. block mapping vs concurrent allocation and/or removal), and > that can only be done sanely

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 03:44:49PM -0500, Benjamin LaHaise wrote: > On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > > >> > > >> Are there other users outside of Solace? It would be good to get > > >>

Re: [PATCH 07/13] aio: enabled thread based async fsync

On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > >> > >> Are there other users outside of Solace? It would be good to get comments.. > > > > I know of quite a few storage/db products that use AIO. The