Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-07 Thread Russell King
On Sun, Jan 07, 2007 at 10:09:13AM -0600, James Bottomley wrote: > On Wed, 2007-01-03 at 15:09 +, Russell King wrote: > > On Wed, Jan 03, 2007 at 09:00:58AM -0600, James Bottomley wrote: > > > However, I was wondering if there might be a different way around this. > > > We can't really walk

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-07 Thread James Bottomley
On Wed, 2007-01-03 at 15:09 +, Russell King wrote: > On Wed, Jan 03, 2007 at 09:00:58AM -0600, James Bottomley wrote: > > However, I was wondering if there might be a different way around this. > > We can't really walk all the user mappings because of the locks, but > > could we store the user

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-07 Thread James Bottomley
On Wed, 2007-01-03 at 15:09 +, Russell King wrote: On Wed, Jan 03, 2007 at 09:00:58AM -0600, James Bottomley wrote: However, I was wondering if there might be a different way around this. We can't really walk all the user mappings because of the locks, but could we store the user flush

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-07 Thread Russell King
On Sun, Jan 07, 2007 at 10:09:13AM -0600, James Bottomley wrote: On Wed, 2007-01-03 at 15:09 +, Russell King wrote: On Wed, Jan 03, 2007 at 09:00:58AM -0600, James Bottomley wrote: However, I was wondering if there might be a different way around this. We can't really walk all the

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-03 Thread Russell King
On Wed, Jan 03, 2007 at 09:00:58AM -0600, James Bottomley wrote: > However, I was wondering if there might be a different way around this. > We can't really walk all the user mappings because of the locks, but > could we store the user flush hints in the page (or a related > structure)? On parisc

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-03 Thread James Bottomley
On Wed, 2007-01-03 at 14:16 +, Russell King wrote: > On Tue, Jan 02, 2007 at 04:20:58PM -0800, David Miller wrote: > > From: James Bottomley <[EMAIL PROTECTED]> > > Date: Tue, 02 Jan 2007 17:34:18 -0600 > > > > > Erm ... for a device driver, if we're preparing to do I/O on the page > > >

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-03 Thread Russell King
On Tue, Jan 02, 2007 at 04:20:58PM -0800, David Miller wrote: > From: James Bottomley <[EMAIL PROTECTED]> > Date: Tue, 02 Jan 2007 17:34:18 -0600 > > > Erm ... for a device driver, if we're preparing to do I/O on the page > > something must have made the user caches coherent ... that can't be > >

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-03 Thread Russell King
On Tue, Jan 02, 2007 at 04:20:58PM -0800, David Miller wrote: From: James Bottomley [EMAIL PROTECTED] Date: Tue, 02 Jan 2007 17:34:18 -0600 Erm ... for a device driver, if we're preparing to do I/O on the page something must have made the user caches coherent ... that can't be kmap,

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-03 Thread James Bottomley
On Wed, 2007-01-03 at 14:16 +, Russell King wrote: On Tue, Jan 02, 2007 at 04:20:58PM -0800, David Miller wrote: From: James Bottomley [EMAIL PROTECTED] Date: Tue, 02 Jan 2007 17:34:18 -0600 Erm ... for a device driver, if we're preparing to do I/O on the page something must have

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-03 Thread Russell King
On Wed, Jan 03, 2007 at 09:00:58AM -0600, James Bottomley wrote: However, I was wondering if there might be a different way around this. We can't really walk all the user mappings because of the locks, but could we store the user flush hints in the page (or a related structure)? On parisc we

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread David Miller
From: James Bottomley <[EMAIL PROTECTED]> Date: Tue, 02 Jan 2007 17:34:18 -0600 > Erm ... for a device driver, if we're preparing to do I/O on the page > something must have made the user caches coherent ... that can't be > kmap, because the driver might elect to DMA on the page ... unless >

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread James Bottomley
On Tue, 2007-01-02 at 15:19 -0800, David Miller wrote: > From: James Bottomley <[EMAIL PROTECTED]> > Date: Tue, 02 Jan 2007 16:53:23 -0600 > > > OK, so lets get down to brass tacks and look at the API characteristics. > > > > Some of the issues are: > > > > 1. Should kmap() actually flush

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread David Miller
From: James Bottomley <[EMAIL PROTECTED]> Date: Tue, 02 Jan 2007 16:53:23 -0600 > OK, so lets get down to brass tacks and look at the API characteristics. > > Some of the issues are: > > 1. Should kmap() actually flush all the user spaces? > 2. Do we need additional hints in to

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread James Bottomley
On Mon, 2007-01-01 at 23:45 +, Russell King wrote: > > However the cache flushing in kmap/kunmap idea might be cleaner and > > better. > > It has the significant advantage that, unlike the flush* calls, they > can't really be forgotten by folk programming on cache alias-free > hardware.

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread Dan Williams
On 1/1/07, Russell King <[EMAIL PROTECTED]> wrote: On Mon, Jan 01, 2007 at 11:15:04PM +0100, Miklos Szeredi wrote: > > > > I'm willing to do that - and I guess this means we can probably do this > > > > instead of walking the list of VMAs for the shared mapping, thereby > > > > hitting both

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread Dan Williams
On 1/1/07, Russell King [EMAIL PROTECTED] wrote: On Mon, Jan 01, 2007 at 11:15:04PM +0100, Miklos Szeredi wrote: I'm willing to do that - and I guess this means we can probably do this instead of walking the list of VMAs for the shared mapping, thereby hitting both anonymous and

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread James Bottomley
On Mon, 2007-01-01 at 23:45 +, Russell King wrote: However the cache flushing in kmap/kunmap idea might be cleaner and better. It has the significant advantage that, unlike the flush* calls, they can't really be forgotten by folk programming on cache alias-free hardware. That's a

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread David Miller
From: James Bottomley [EMAIL PROTECTED] Date: Tue, 02 Jan 2007 16:53:23 -0600 OK, so lets get down to brass tacks and look at the API characteristics. Some of the issues are: 1. Should kmap() actually flush all the user spaces? 2. Do we need additional hints in to kmap/kunmap?

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread James Bottomley
On Tue, 2007-01-02 at 15:19 -0800, David Miller wrote: From: James Bottomley [EMAIL PROTECTED] Date: Tue, 02 Jan 2007 16:53:23 -0600 OK, so lets get down to brass tacks and look at the API characteristics. Some of the issues are: 1. Should kmap() actually flush all the user

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-02 Thread David Miller
From: James Bottomley [EMAIL PROTECTED] Date: Tue, 02 Jan 2007 17:34:18 -0600 Erm ... for a device driver, if we're preparing to do I/O on the page something must have made the user caches coherent ... that can't be kmap, because the driver might elect to DMA on the page ... unless another

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Russell King
On Mon, Jan 01, 2007 at 11:15:04PM +0100, Miklos Szeredi wrote: > > > > I'm willing to do that - and I guess this means we can probably do this > > > > instead of walking the list of VMAs for the shared mapping, thereby > > > > hitting both anonymous and shared mappings with the same code? > > >

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Mon, 2007-01-01 at 15:04 -0800, David Miller wrote: > I thought this was accepted and Ralf is using it on MIPS? It partially is ... we're using it on parisc as well, but only as a supplement to the current linux flushing APIs. There's still no guarantee in the standard linux API that

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Russell King
On Mon, Jan 01, 2007 at 03:01:52PM -0800, David Miller wrote: > From: James Bottomley <[EMAIL PROTECTED]> > Date: Mon, 01 Jan 2007 10:34:12 -0600 > > > Erm, well the whole reason for the flush_anon_pages() was that you told > > me not to do it in flush_dcache_page() ... > > > > Although this is

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread David Miller
From: James Bottomley <[EMAIL PROTECTED]> Date: Mon, 01 Jan 2007 10:44:36 -0600 > Actually, this was proposed here: > > http://marc.theaimsgroup.com/?t=11540975413 > > When I updated the interface to work for the combined VIPT/PIPT cache on > the latest pariscs. However, there were no

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread David Miller
From: James Bottomley <[EMAIL PROTECTED]> Date: Mon, 01 Jan 2007 10:34:12 -0600 > Erm, well the whole reason for the flush_anon_pages() was that you told > me not to do it in flush_dcache_page() ... > > Although this is perhaps part of the confusion over what > flush_dcache_page() is actually

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Miklos Szeredi
> > > I'm willing to do that - and I guess this means we can probably do this > > > instead of walking the list of VMAs for the shared mapping, thereby > > > hitting both anonymous and shared mappings with the same code? > > > > But for the get_user_pages() case there's no point, is there? The

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Sun, 2006-12-31 at 13:12 -0800, David Miller wrote: > From: Linus Torvalds <[EMAIL PROTECTED]> > Date: Sun, 31 Dec 2006 12:58:45 -0800 (PST) > > > So there really is two different cases here: > > > > - flush the cache as seen by A PARTICULAR virtual mapping. > > > >This is ptrace, but

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Russell King
On Mon, Jan 01, 2007 at 09:35:17AM -0500, James Bottomley wrote: > On Sat, 2006-12-30 at 10:26 -0800, Linus Torvalds wrote: > > > > On Sat, 30 Dec 2006, Russell King wrote: > > > > > > And here's the flush_anon_page() part. > > This looks fine to me (if you need my ack). > > > > Add

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Thu, 2006-12-21 at 16:57 +, Russell King wrote: > I'm not entirely convinced that it can be replaced. What if the page > is in the page cache and is shared with other processes? That quite > clearly falls under flush_dcache_page()'s remit. Actually, it should work. flush_dcache_page()

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Sat, 2006-12-30 at 10:26 -0800, Linus Torvalds wrote: > > On Sat, 30 Dec 2006, Russell King wrote: > > > > And here's the flush_anon_page() part. This looks fine to me (if you need my ack). > > Add flush_anon_page() for ARM, to avoid data corruption issues when using > > fuse or other

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Sat, 2006-12-30 at 10:26 -0800, Linus Torvalds wrote: On Sat, 30 Dec 2006, Russell King wrote: And here's the flush_anon_page() part. This looks fine to me (if you need my ack). Add flush_anon_page() for ARM, to avoid data corruption issues when using fuse or other subsystems

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Thu, 2006-12-21 at 16:57 +, Russell King wrote: I'm not entirely convinced that it can be replaced. What if the page is in the page cache and is shared with other processes? That quite clearly falls under flush_dcache_page()'s remit. Actually, it should work. flush_dcache_page() is

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Russell King
On Mon, Jan 01, 2007 at 09:35:17AM -0500, James Bottomley wrote: On Sat, 2006-12-30 at 10:26 -0800, Linus Torvalds wrote: On Sat, 30 Dec 2006, Russell King wrote: And here's the flush_anon_page() part. This looks fine to me (if you need my ack). Add flush_anon_page() for ARM,

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Sun, 2006-12-31 at 13:12 -0800, David Miller wrote: From: Linus Torvalds [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 12:58:45 -0800 (PST) So there really is two different cases here: - flush the cache as seen by A PARTICULAR virtual mapping. This is ptrace, but it's other

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Miklos Szeredi
I'm willing to do that - and I guess this means we can probably do this instead of walking the list of VMAs for the shared mapping, thereby hitting both anonymous and shared mappings with the same code? But for the get_user_pages() case there's no point, is there? The VMA and the

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread David Miller
From: James Bottomley [EMAIL PROTECTED] Date: Mon, 01 Jan 2007 10:34:12 -0600 Erm, well the whole reason for the flush_anon_pages() was that you told me not to do it in flush_dcache_page() ... Although this is perhaps part of the confusion over what flush_dcache_page() is actually supposed

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread David Miller
From: James Bottomley [EMAIL PROTECTED] Date: Mon, 01 Jan 2007 10:44:36 -0600 Actually, this was proposed here: http://marc.theaimsgroup.com/?t=11540975413 When I updated the interface to work for the combined VIPT/PIPT cache on the latest pariscs. However, there were no takers for

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Russell King
On Mon, Jan 01, 2007 at 03:01:52PM -0800, David Miller wrote: From: James Bottomley [EMAIL PROTECTED] Date: Mon, 01 Jan 2007 10:34:12 -0600 Erm, well the whole reason for the flush_anon_pages() was that you told me not to do it in flush_dcache_page() ... Although this is perhaps part

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread James Bottomley
On Mon, 2007-01-01 at 15:04 -0800, David Miller wrote: I thought this was accepted and Ralf is using it on MIPS? It partially is ... we're using it on parisc as well, but only as a supplement to the current linux flushing APIs. There's still no guarantee in the standard linux API that kmap();

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2007-01-01 Thread Russell King
On Mon, Jan 01, 2007 at 11:15:04PM +0100, Miklos Szeredi wrote: I'm willing to do that - and I guess this means we can probably do this instead of walking the list of VMAs for the shared mapping, thereby hitting both anonymous and shared mappings with the same code? But for the

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Linus Torvalds <[EMAIL PROTECTED]> Date: Sun, 31 Dec 2006 12:58:45 -0800 (PST) > So there really is two different cases here: > > - flush the cache as seen by A PARTICULAR virtual mapping. > >This is ptrace, but it's other things like "unmap page from this VM" >too. > > -

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Linus Torvalds
On Sun, 31 Dec 2006, David Miller wrote: > > Even in the ptrace() case, you do want to flush all the other VMA's > that might be out there with an aliased cached copy in the cpu cache. I don't think that's necessarily true. If the same page is cached differently (and virtually) in multiple

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Miklos Szeredi <[EMAIL PROTECTED]> Date: Sun, 31 Dec 2006 13:24:53 +0100 > > I'm willing to do that - and I guess this means we can probably do this > > instead of walking the list of VMAs for the shared mapping, thereby > > hitting both anonymous and shared mappings with the same code? >

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sun, Dec 31, 2006 at 01:24:53PM +0100, Miklos Szeredi wrote: > > I'm willing to do that - and I guess this means we can probably do this > > instead of walking the list of VMAs for the shared mapping, thereby > > hitting both anonymous and shared mappings with the same code? > > But for the

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Miklos Szeredi
> I'm willing to do that - and I guess this means we can probably do this > instead of walking the list of VMAs for the shared mapping, thereby > hitting both anonymous and shared mappings with the same code? But for the get_user_pages() case there's no point, is there? The VMA and the virtual

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Russell King <[EMAIL PROTECTED]> Date: Sun, 31 Dec 2006 10:00:07 + > I'm willing to do that - and I guess this means we can probably do this > instead of walking the list of VMAs for the shared mapping, thereby > hitting both anonymous and shared mappings with the same code? That's

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sun, Dec 31, 2006 at 01:47:56AM -0800, David Miller wrote: > From: Arjan van de Ven <[EMAIL PROTECTED]> > Date: Sun, 31 Dec 2006 10:27:22 +0100 > > > However, it's not only FUSE which is suffering - direct-IO also doesn't > > > work. > > > > for direct-IO the kernel won't touch the data *at

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sun, Dec 31, 2006 at 10:27:22AM +0100, Arjan van de Ven wrote: > > > > > However, it's not only FUSE which is suffering - direct-IO also doesn't > > work. > > for direct-IO the kernel won't touch the data *at all*... (that's the > point ;) Wrong. One word: PIO. We _still_ to this day

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Arjan van de Ven <[EMAIL PROTECTED]> Date: Sun, 31 Dec 2006 10:27:22 +0100 > > > > > However, it's not only FUSE which is suffering - direct-IO also doesn't > > work. > > for direct-IO the kernel won't touch the data *at all*... (that's the > point ;) > > is it still an issue then?

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Russell King <[EMAIL PROTECTED]> Date: Sun, 31 Dec 2006 09:23:18 + > We do this on ARM - if page_mapping() is NULL, we flush the kernel > alias unconditionally. However, we have no view where the user > mapping of that page is, which is where the problem is. Cache lines > remain

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Miklos Szeredi <[EMAIL PROTECTED]> Date: Sun, 31 Dec 2006 10:10:35 +0100 > > Therefore, FUSE probably could have been fixed by judicious use > > of copy_{to,from}_user_page() calls instead of adding this new > > ad-hoc flush_anon_page() thing. > > Probably, but I don't think either

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Arjan van de Ven
> > However, it's not only FUSE which is suffering - direct-IO also doesn't > work. for direct-IO the kernel won't touch the data *at all*... (that's the point ;) is it still an issue then? -- if you want to mail me at work (you don't), use arjan (at) linux.intel.com Test the interaction

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sat, Dec 30, 2006 at 09:23:38PM -0800, David Miller wrote: > From: Russell King <[EMAIL PROTECTED]> > Date: Sat, 30 Dec 2006 22:46:04 + > > > iirc, flush_anon_page() was introduced to fix non-working fuse on parisc, > > which occurs because fuse wants to use get_user_pages() to read data

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Miklos Szeredi
> Therefore, FUSE probably could have been fixed by judicious use > of copy_{to,from}_user_page() calls instead of adding this new > ad-hoc flush_anon_page() thing. Probably, but I don't think either interface is perfect. copy_*_user_page() will double flush the user mapping (get_user_pages()

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Miklos Szeredi
Therefore, FUSE probably could have been fixed by judicious use of copy_{to,from}_user_page() calls instead of adding this new ad-hoc flush_anon_page() thing. Probably, but I don't think either interface is perfect. copy_*_user_page() will double flush the user mapping (get_user_pages()

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sat, Dec 30, 2006 at 09:23:38PM -0800, David Miller wrote: From: Russell King [EMAIL PROTECTED] Date: Sat, 30 Dec 2006 22:46:04 + iirc, flush_anon_page() was introduced to fix non-working fuse on parisc, which occurs because fuse wants to use get_user_pages() to read data from the

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Arjan van de Ven
However, it's not only FUSE which is suffering - direct-IO also doesn't work. for direct-IO the kernel won't touch the data *at all*... (that's the point ;) is it still an issue then? -- if you want to mail me at work (you don't), use arjan (at) linux.intel.com Test the interaction

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Miklos Szeredi [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 10:10:35 +0100 Therefore, FUSE probably could have been fixed by judicious use of copy_{to,from}_user_page() calls instead of adding this new ad-hoc flush_anon_page() thing. Probably, but I don't think either interface is

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Russell King [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 09:23:18 + We do this on ARM - if page_mapping() is NULL, we flush the kernel alias unconditionally. However, we have no view where the user mapping of that page is, which is where the problem is. Cache lines remain allocated

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Arjan van de Ven [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 10:27:22 +0100 However, it's not only FUSE which is suffering - direct-IO also doesn't work. for direct-IO the kernel won't touch the data *at all*... (that's the point ;) is it still an issue then? It can be an

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sun, Dec 31, 2006 at 10:27:22AM +0100, Arjan van de Ven wrote: However, it's not only FUSE which is suffering - direct-IO also doesn't work. for direct-IO the kernel won't touch the data *at all*... (that's the point ;) Wrong. One word: PIO. We _still_ to this day have no

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sun, Dec 31, 2006 at 01:47:56AM -0800, David Miller wrote: From: Arjan van de Ven [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 10:27:22 +0100 However, it's not only FUSE which is suffering - direct-IO also doesn't work. for direct-IO the kernel won't touch the data *at all*... (that's

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Russell King [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 10:00:07 + I'm willing to do that - and I guess this means we can probably do this instead of walking the list of VMAs for the shared mapping, thereby hitting both anonymous and shared mappings with the same code? That's pretty

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Miklos Szeredi
I'm willing to do that - and I guess this means we can probably do this instead of walking the list of VMAs for the shared mapping, thereby hitting both anonymous and shared mappings with the same code? But for the get_user_pages() case there's no point, is there? The VMA and the virtual

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Russell King
On Sun, Dec 31, 2006 at 01:24:53PM +0100, Miklos Szeredi wrote: I'm willing to do that - and I guess this means we can probably do this instead of walking the list of VMAs for the shared mapping, thereby hitting both anonymous and shared mappings with the same code? But for the

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Miklos Szeredi [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 13:24:53 +0100 I'm willing to do that - and I guess this means we can probably do this instead of walking the list of VMAs for the shared mapping, thereby hitting both anonymous and shared mappings with the same code? But for

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread Linus Torvalds
On Sun, 31 Dec 2006, David Miller wrote: Even in the ptrace() case, you do want to flush all the other VMA's that might be out there with an aliased cached copy in the cpu cache. I don't think that's necessarily true. If the same page is cached differently (and virtually) in multiple

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-31 Thread David Miller
From: Linus Torvalds [EMAIL PROTECTED] Date: Sun, 31 Dec 2006 12:58:45 -0800 (PST) So there really is two different cases here: - flush the cache as seen by A PARTICULAR virtual mapping. This is ptrace, but it's other things like unmap page from this VM too. - flush the cache

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread David Miller
From: Russell King <[EMAIL PROTECTED]> Date: Sat, 30 Dec 2006 22:46:04 + > iirc, flush_anon_page() was introduced to fix non-working fuse on parisc, > which occurs because fuse wants to use get_user_pages() to read data from > the current processes memory space. > > get_user_pages() contains

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Russell King
On Sat, Dec 30, 2006 at 10:26:20AM -0800, Linus Torvalds wrote: > > > On Sat, 30 Dec 2006, Russell King wrote: > > > > And here's the flush_anon_page() part. > > > > Add flush_anon_page() for ARM, to avoid data corruption issues when using > > fuse or other subsystems using get_user_pages(). >

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Linus Torvalds
On Sat, 30 Dec 2006, Russell King wrote: > > And here's the flush_anon_page() part. > > Add flush_anon_page() for ARM, to avoid data corruption issues when using > fuse or other subsystems using get_user_pages(). Btw, since this doesn't actually change any code for anybody but ARM, just adds

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Linus Torvalds
On Sat, 30 Dec 2006, Russell King wrote: > > Given that no one has any outstanding issues with the following patch, I'm > going to ask akpm to put this into -mm, and shortly after (a week or so) > I'll submit it and the ARM flush_anon_page() patch to Linus for -rc to fix > ARM data corruption

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Russell King
On Sat, Dec 30, 2006 at 04:39:55PM +, Russell King wrote: > Given that no one has any outstanding issues with the following patch, I'm > going to ask akpm to put this into -mm, and shortly after (a week or so) > I'll submit it and the ARM flush_anon_page() patch to Linus for -rc to fix > ARM

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Russell King
Given that no one has any outstanding issues with the following patch, I'm going to ask akpm to put this into -mm, and shortly after (a week or so) I'll submit it and the ARM flush_anon_page() patch to Linus for -rc to fix ARM data corruption issues. If anyone _does_ have a problem, holler ASAP.

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Russell King
Given that no one has any outstanding issues with the following patch, I'm going to ask akpm to put this into -mm, and shortly after (a week or so) I'll submit it and the ARM flush_anon_page() patch to Linus for -rc to fix ARM data corruption issues. If anyone _does_ have a problem, holler ASAP.

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Russell King
On Sat, Dec 30, 2006 at 04:39:55PM +, Russell King wrote: Given that no one has any outstanding issues with the following patch, I'm going to ask akpm to put this into -mm, and shortly after (a week or so) I'll submit it and the ARM flush_anon_page() patch to Linus for -rc to fix ARM data

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Linus Torvalds
On Sat, 30 Dec 2006, Russell King wrote: Given that no one has any outstanding issues with the following patch, I'm going to ask akpm to put this into -mm, and shortly after (a week or so) I'll submit it and the ARM flush_anon_page() patch to Linus for -rc to fix ARM data corruption issues.

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Linus Torvalds
On Sat, 30 Dec 2006, Russell King wrote: And here's the flush_anon_page() part. Add flush_anon_page() for ARM, to avoid data corruption issues when using fuse or other subsystems using get_user_pages(). Btw, since this doesn't actually change any code for anybody but ARM, just adds a

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread Russell King
On Sat, Dec 30, 2006 at 10:26:20AM -0800, Linus Torvalds wrote: On Sat, 30 Dec 2006, Russell King wrote: And here's the flush_anon_page() part. Add flush_anon_page() for ARM, to avoid data corruption issues when using fuse or other subsystems using get_user_pages(). Btw, since

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-30 Thread David Miller
From: Russell King [EMAIL PROTECTED] Date: Sat, 30 Dec 2006 22:46:04 + iirc, flush_anon_page() was introduced to fix non-working fuse on parisc, which occurs because fuse wants to use get_user_pages() to read data from the current processes memory space. get_user_pages() contains a call

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-22 Thread Randolph Chung
Is the documentation wrong? Yes. As I've already explained there is no guarantee that get_user_pages() is only called to obtain pages for the current process, and flush_anon_pages() is called irrespective of which user process is being 'got'. ok, it's easy enough to fix, I'm just trying to

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-22 Thread Russell King
On Fri, Dec 22, 2006 at 07:51:34AM +0800, Randolph Chung wrote: > >I understand now. I'm not sure how the PARISC implementation can be > >correct in this light. > > According to cachetlb.txt: > > void flush_anon_page(struct page *page, unsigned long vmaddr) > When the kernel needs to

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-22 Thread Russell King
On Fri, Dec 22, 2006 at 07:51:34AM +0800, Randolph Chung wrote: I understand now. I'm not sure how the PARISC implementation can be correct in this light. According to cachetlb.txt: void flush_anon_page(struct page *page, unsigned long vmaddr) When the kernel needs to access

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-22 Thread Randolph Chung
Is the documentation wrong? Yes. As I've already explained there is no guarantee that get_user_pages() is only called to obtain pages for the current process, and flush_anon_pages() is called irrespective of which user process is being 'got'. ok, it's easy enough to fix, I'm just trying to

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Randolph Chung
I understand now. I'm not sure how the PARISC implementation can be correct in this light. According to cachetlb.txt: void flush_anon_page(struct page *page, unsigned long vmaddr) When the kernel needs to access the contents of an anonymous page, it calls this function

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Miklos Szeredi
> > > >The root of the problem is that copy_to_user() may cause page faults > >on the userspace buffer, and the page fault might (in case of a > >maliciously crafted filesystem) recurse into the filesystem itself. > > Would it be worthwhile to mlock the page? I know that needs root > privs or

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Jan Engelhardt
On Dec 21 2006 18:51, Miklos Szeredi wrote: > >The root of the problem is that copy_to_user() may cause page faults >on the userspace buffer, and the page fault might (in case of a >maliciously crafted filesystem) recurse into the filesystem itself. Would it be worthwhile to mlock the page? I

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Miklos Szeredi
> > > > > > Yes, note the flush_dcache_page() call in fuse_copy_finish(). That > > > > > > could be replaced by the flush_kernel_dcache_page() (added by James > > > > > > Bottomley together with flush_anon_page()) when all relevant > > > > > > architectures have defined it. > > > > > > > > > > I

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Russell King
On Thu, Dec 21, 2006 at 07:30:11PM +0100, Miklos Szeredi wrote: > > > > > Yes, note the flush_dcache_page() call in fuse_copy_finish(). That > > > > > could be replaced by the flush_kernel_dcache_page() (added by James > > > > > Bottomley together with flush_anon_page()) when all relevant > > > >

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Miklos Szeredi
> > > > Yes, note the flush_dcache_page() call in fuse_copy_finish(). That > > > > could be replaced by the flush_kernel_dcache_page() (added by James > > > > Bottomley together with flush_anon_page()) when all relevant > > > > architectures have defined it. > > > > > > I should say that

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Russell King
On Thu, Dec 21, 2006 at 06:55:47PM +0100, Miklos Szeredi wrote: > > > Yes, note the flush_dcache_page() call in fuse_copy_finish(). That > > > could be replaced by the flush_kernel_dcache_page() (added by James > > > Bottomley together with flush_anon_page()) when all relevant > > > architectures

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Miklos Szeredi
> > Yes, note the flush_dcache_page() call in fuse_copy_finish(). That > > could be replaced by the flush_kernel_dcache_page() (added by James > > Bottomley together with flush_anon_page()) when all relevant > > architectures have defined it. > > I should say that flush_anon_page() in its

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Miklos Szeredi
> On Thu, Dec 21, 2006 at 04:53:56PM +0100, Miklos Szeredi wrote: > > I'll first answer the last paragraph. > > > > > I suggest that in order for fuse to work reliably on ARM, it is modified > > > to behave more like a reasonable device driver, and use the functions > > > defined in asm/uaccess.h

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Russell King
On Thu, Dec 21, 2006 at 05:29:42PM +0100, Arjan van de Ven wrote: > > > So, given all this additional complexity _and_ that it would only be > > safe on non-preempt UP, the question becomes: is using get_user_pages() > > to access the current processes memory space legal? Given the above, > > I

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Russell King
On Thu, Dec 21, 2006 at 04:53:56PM +0100, Miklos Szeredi wrote: > Yes, note the flush_dcache_page() call in fuse_copy_finish(). That > could be replaced by the flush_kernel_dcache_page() (added by James > Bottomley together with flush_anon_page()) when all relevant > architectures have defined

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Russell King
On Thu, Dec 21, 2006 at 04:53:56PM +0100, Miklos Szeredi wrote: > I'll first answer the last paragraph. > > > I suggest that in order for fuse to work reliably on ARM, it is modified > > to behave more like a reasonable device driver, and use the functions > > defined in asm/uaccess.h when it

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Arjan van de Ven
> So, given all this additional complexity _and_ that it would only be > safe on non-preempt UP, the question becomes: is using get_user_pages() > to access the current processes memory space legal? Given the above, > I would say not. I'd say that copy_from_user is the right api for this, not

Re: fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Miklos Szeredi
I'll first answer the last paragraph. > I suggest that in order for fuse to work reliably on ARM, it is modified > to behave more like a reasonable device driver, and use the functions > defined in asm/uaccess.h when it wants to access the current processes > VM space. Fuse needs to use

fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Russell King
I've recently been asked to look at why fuse doesn't work on ARM. In doing so and thinking about what fuse is doing, I've come to the conclusion that the way fuse accesses the _current_ processes memory space is less than ideal. The problem is as follows: - current process calls writev() -

fuse, get_user_pages, flush_anon_page, aliasing caches and all that again

2006-12-21 Thread Russell King
I've recently been asked to look at why fuse doesn't work on ARM. In doing so and thinking about what fuse is doing, I've come to the conclusion that the way fuse accesses the _current_ processes memory space is less than ideal. The problem is as follows: - current process calls writev() -

  1   2   >