Re: [RFC, PATCH] Extensible AIO interface

2012-10-04 Thread Kent Overstreet
On Thu, Oct 04, 2012 at 06:58:06AM +0900, Tejun Heo wrote: > Hello, Kent. > > On Tue, Oct 02, 2012 at 08:00:20PM -0700, Kent Overstreet wrote: > > > However, I don't think it's a good idea to try to implement something > > > which is a neutral transport of opaque data between userland and lower >

Re: [RFC, PATCH] Extensible AIO interface

2012-10-04 Thread Kent Overstreet
On Wed, Oct 03, 2012 at 03:15:26PM -0400, Jeff Moyer wrote: > Kent Overstreet writes: > > > On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: > >> Kent Overstreet writes: > >> > >> > So, I and other people keep running into things where we really need to > >> > add an interface to

Re: [RFC, PATCH] Extensible AIO interface

2012-10-04 Thread Kent Overstreet
On Wed, Oct 03, 2012 at 03:15:26PM -0400, Jeff Moyer wrote: Kent Overstreet koverstr...@google.com writes: On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: Kent Overstreet koverstr...@google.com writes: So, I and other people keep running into things where we really need to

Re: [RFC, PATCH] Extensible AIO interface

2012-10-04 Thread Kent Overstreet
On Thu, Oct 04, 2012 at 06:58:06AM +0900, Tejun Heo wrote: Hello, Kent. On Tue, Oct 02, 2012 at 08:00:20PM -0700, Kent Overstreet wrote: However, I don't think it's a good idea to try to implement something which is a neutral transport of opaque data between userland and lower layers.

Re: [RFC, PATCH] Extensible AIO interface

2012-10-03 Thread Dave Chinner
On Tue, Oct 02, 2012 at 07:41:10PM -0700, Kent Overstreet wrote: > On Wed, Oct 03, 2012 at 11:28:25AM +1000, Dave Chinner wrote: > > On Tue, Oct 02, 2012 at 05:20:29PM -0700, Kent Overstreet wrote: > > > On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: > > > > Kent Overstreet writes: >

Re: [RFC, PATCH] Extensible AIO interface

2012-10-03 Thread Tejun Heo
Hello, Kent. On Tue, Oct 02, 2012 at 08:00:20PM -0700, Kent Overstreet wrote: > > However, I don't think it's a good idea to try to implement something > > which is a neutral transport of opaque data between userland and lower > > layers. Things like that sound attractive with unlimited > >

Re: [RFC, PATCH] Extensible AIO interface

2012-10-03 Thread Jeff Moyer
Kent Overstreet writes: > On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: >> Kent Overstreet writes: >> >> > So, I and other people keep running into things where we really need to >> > add an interface to pass some auxiliary... stuff along with a pread() or >> > pwrite(). >> > >>

Re: [RFC, PATCH] Extensible AIO interface

2012-10-03 Thread Jeff Moyer
Kent Overstreet koverstr...@google.com writes: On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: Kent Overstreet koverstr...@google.com writes: So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a

Re: [RFC, PATCH] Extensible AIO interface

2012-10-03 Thread Tejun Heo
Hello, Kent. On Tue, Oct 02, 2012 at 08:00:20PM -0700, Kent Overstreet wrote: However, I don't think it's a good idea to try to implement something which is a neutral transport of opaque data between userland and lower layers. Things like that sound attractive with unlimited

Re: [RFC, PATCH] Extensible AIO interface

2012-10-03 Thread Dave Chinner
On Tue, Oct 02, 2012 at 07:41:10PM -0700, Kent Overstreet wrote: On Wed, Oct 03, 2012 at 11:28:25AM +1000, Dave Chinner wrote: On Tue, Oct 02, 2012 at 05:20:29PM -0700, Kent Overstreet wrote: On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: Kent Overstreet

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Wed, Oct 03, 2012 at 10:41:06AM +0900, Tejun Heo wrote: > Hello, Kent. > > On Tue, Oct 02, 2012 at 02:41:13PM -0700, Kent Overstreet wrote: > > Seems to me it'd be no different from security considerations when > > introducing new ioctls. I.e., messy, ad hoc, easy to get wrong, but > >

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Wed, Oct 03, 2012 at 11:28:25AM +1000, Dave Chinner wrote: > On Tue, Oct 02, 2012 at 05:20:29PM -0700, Kent Overstreet wrote: > > On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: > > > Kent Overstreet writes: > > > > > > > So, I and other people keep running into things where we

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Tejun Heo
Hello, Kent. On Tue, Oct 02, 2012 at 02:41:13PM -0700, Kent Overstreet wrote: > Seems to me it'd be no different from security considerations when > introducing new ioctls. I.e., messy, ad hoc, easy to get wrong, but > sometimes no way around it. > > It really has to be ad hoc if it's

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Dave Chinner
On Tue, Oct 02, 2012 at 05:20:29PM -0700, Kent Overstreet wrote: > On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: > > Kent Overstreet writes: > > > > > So, I and other people keep running into things where we really need to > > > add an interface to pass some auxiliary... stuff

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: > Kent Overstreet writes: > > > So, I and other people keep running into things where we really need to > > add an interface to pass some auxiliary... stuff along with a pread() or > > pwrite(). > > > > A few examples: > > > > * IO

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Martin K. Petersen
> "Kent" == Kent Overstreet writes: >> Hmm, careful here. I think that in DIF/DIX the checksums are >> per-sector, not per IO, right? That'd mean that the PAGE_SIZE attr >> limit in this patch would be magically creating different max IO size >> limits on different architectures. That

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Tue, Oct 02, 2012 at 10:43:23AM -0700, Zach Brown wrote: > > The generic code wouldn't know about any user pointers inside > > attributes, so it'd have to be downstream consumers. Hopefully there > > won't be many attributes with user pointers in them (I don't expect > > there to be), so we

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Andi Kleen
Kent Overstreet writes: > So, I and other people keep running into things where we really need to > add an interface to pass some auxiliary... stuff along with a pread() or > pwrite(). How would you enumerate this? How does the application know what the underlying stack supports/need? How is

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Zach Brown
> The generic code wouldn't know about any user pointers inside > attributes, so it'd have to be downstream consumers. Hopefully there > won't be many attributes with user pointers in them (I don't expect > there to be), so we won't have too much of this messyness. I really don't like this. We

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Jeff Moyer
Kent Overstreet writes: > So, I and other people keep running into things where we really need to > add an interface to pass some auxiliary... stuff along with a pread() or > pwrite(). > > A few examples: > > * IO scheduler hints. Some userspace program wants to, per IO, specify > either

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Jeff Moyer
Kent Overstreet koverstr...@google.com writes: So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a pread() or pwrite(). A few examples: * IO scheduler hints. Some userspace program wants to, per IO, specify

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Zach Brown
The generic code wouldn't know about any user pointers inside attributes, so it'd have to be downstream consumers. Hopefully there won't be many attributes with user pointers in them (I don't expect there to be), so we won't have too much of this messyness. I really don't like this. We

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Andi Kleen
Kent Overstreet koverstr...@google.com writes: So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a pread() or pwrite(). How would you enumerate this? How does the application know what the underlying stack

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Tue, Oct 02, 2012 at 10:43:23AM -0700, Zach Brown wrote: The generic code wouldn't know about any user pointers inside attributes, so it'd have to be downstream consumers. Hopefully there won't be many attributes with user pointers in them (I don't expect there to be), so we won't have

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Martin K. Petersen
Kent == Kent Overstreet koverstr...@google.com writes: Hmm, careful here. I think that in DIF/DIX the checksums are per-sector, not per IO, right? That'd mean that the PAGE_SIZE attr limit in this patch would be magically creating different max IO size limits on different architectures.

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: Kent Overstreet koverstr...@google.com writes: So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a pread() or pwrite(). A few examples: *

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Dave Chinner
On Tue, Oct 02, 2012 at 05:20:29PM -0700, Kent Overstreet wrote: On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: Kent Overstreet koverstr...@google.com writes: So, I and other people keep running into things where we really need to add an interface to pass some auxiliary...

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Tejun Heo
Hello, Kent. On Tue, Oct 02, 2012 at 02:41:13PM -0700, Kent Overstreet wrote: Seems to me it'd be no different from security considerations when introducing new ioctls. I.e., messy, ad hoc, easy to get wrong, but sometimes no way around it. It really has to be ad hoc if it's extensible,

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Wed, Oct 03, 2012 at 11:28:25AM +1000, Dave Chinner wrote: On Tue, Oct 02, 2012 at 05:20:29PM -0700, Kent Overstreet wrote: On Tue, Oct 02, 2012 at 01:41:17PM -0400, Jeff Moyer wrote: Kent Overstreet koverstr...@google.com writes: So, I and other people keep running into things

Re: [RFC, PATCH] Extensible AIO interface

2012-10-02 Thread Kent Overstreet
On Wed, Oct 03, 2012 at 10:41:06AM +0900, Tejun Heo wrote: Hello, Kent. On Tue, Oct 02, 2012 at 02:41:13PM -0700, Kent Overstreet wrote: Seems to me it'd be no different from security considerations when introducing new ioctls. I.e., messy, ad hoc, easy to get wrong, but sometimes no way

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
On Mon, Oct 01, 2012 at 04:44:39PM -0700, Zach Brown wrote: > And what about duplicate instances of a given attribute id? Use the > first? The last? Error? Depends on the id? I thought of a better idea, instead of explicitly checking for disallowed dups: We want to return -ENOTHANDLED for

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
On Mon, Oct 01, 2012 at 04:44:39PM -0700, Zach Brown wrote: > > Not just per sector, Per hardware sector. For passing around checksums > > userspace would have to find out the hardware sector size and checksum > > type/size via a different interface, and then the attribute would > > contain a

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Zach Brown
> Not just per sector, Per hardware sector. For passing around checksums > userspace would have to find out the hardware sector size and checksum > type/size via a different interface, and then the attribute would > contain a pointer to a buffer that can hold the appropriate number of > checksums.

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
On Mon, Oct 01, 2012 at 04:12:22PM -0700, Zach Brown wrote: > On Mon, Oct 01, 2012 at 03:23:41PM -0700, Kent Overstreet wrote: > > So, I and other people keep running into things where we really need to > > add an interface to pass some auxiliary... stuff along with a pread() or > > pwrite(). > >

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Zach Brown
On Mon, Oct 01, 2012 at 03:23:41PM -0700, Kent Overstreet wrote: > So, I and other people keep running into things where we really need to > add an interface to pass some auxiliary... stuff along with a pread() or > pwrite(). Sure. Martin (cc:ed) will sympathize. > A few examples: > > * IO

[RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a pread() or pwrite(). A few examples: * IO scheduler hints. Some userspace program wants to, per IO, specify either priorities or a cgroup - by specifying a cgroup

[RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a pread() or pwrite(). A few examples: * IO scheduler hints. Some userspace program wants to, per IO, specify either priorities or a cgroup - by specifying a cgroup

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Zach Brown
On Mon, Oct 01, 2012 at 03:23:41PM -0700, Kent Overstreet wrote: So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a pread() or pwrite(). Sure. Martin (cc:ed) will sympathize. A few examples: * IO scheduler

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
On Mon, Oct 01, 2012 at 04:12:22PM -0700, Zach Brown wrote: On Mon, Oct 01, 2012 at 03:23:41PM -0700, Kent Overstreet wrote: So, I and other people keep running into things where we really need to add an interface to pass some auxiliary... stuff along with a pread() or pwrite(). Sure.

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Zach Brown
Not just per sector, Per hardware sector. For passing around checksums userspace would have to find out the hardware sector size and checksum type/size via a different interface, and then the attribute would contain a pointer to a buffer that can hold the appropriate number of checksums. All

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
On Mon, Oct 01, 2012 at 04:44:39PM -0700, Zach Brown wrote: Not just per sector, Per hardware sector. For passing around checksums userspace would have to find out the hardware sector size and checksum type/size via a different interface, and then the attribute would contain a pointer to a

Re: [RFC, PATCH] Extensible AIO interface

2012-10-01 Thread Kent Overstreet
On Mon, Oct 01, 2012 at 04:44:39PM -0700, Zach Brown wrote: And what about duplicate instances of a given attribute id? Use the first? The last? Error? Depends on the id? I thought of a better idea, instead of explicitly checking for disallowed dups: We want to return -ENOTHANDLED for not