Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Jason Gunthorpe
On Thu, Mar 21, 2013 at 07:42:37PM +0200, Michael S. Tsirkin wrote: > It doesn't actually, and our app would sometimes write to these pages. > It simply does not care which version does the remote get in this case > since we track writes and resend later. Heh, somehow I thought you might say that

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Jason Gunthorpe
On Thu, Mar 21, 2013 at 08:16:33PM +0200, Michael S. Tsirkin wrote: > This is the one I find redundant. Since the write will be done by > the adaptor under direct control by the application, why does it > make sense to declare this beforehand? If you don't want to allow > local write access to me

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Jason Gunthorpe
On Thu, Mar 21, 2013 at 09:15:41PM +0200, Michael S. Tsirkin wrote: > On Thu, Mar 21, 2013 at 12:41:35PM -0600, Jason Gunthorpe wrote: > > On Thu, Mar 21, 2013 at 08:16:33PM +0200, Michael S. Tsirkin wrote: > > > > > This is the one I find redundant. Since the write will be done by > > > the adapt

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Jason Gunthorpe
On Thu, Mar 21, 2013 at 11:39:47AM +0200, Michael S. Tsirkin wrote: > On Thu, Mar 21, 2013 at 02:13:38AM -0700, Roland Dreier wrote: > > On Thu, Mar 21, 2013 at 1:51 AM, Michael S. Tsirkin wrote: > > >> In that case, no, I don't see any reason for LOCAL_WRITE, since the > > >> only RDMA operations

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Jason Gunthorpe
On Thu, Mar 21, 2013 at 07:15:25PM +0200, Michael S. Tsirkin wrote: > No because application does this: > init page > > ... > > after a lot of time > > .. > > register > send > unregister > > so it can not be read only. mprotect(READONLY) register send unregister mprotect(WRITABLE) ? With

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 12:41:35PM -0600, Jason Gunthorpe wrote: > On Thu, Mar 21, 2013 at 08:16:33PM +0200, Michael S. Tsirkin wrote: > > > This is the one I find redundant. Since the write will be done by > > the adaptor under direct control by the application, why does it > > make sense to decl

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 11:11:15AM -0600, Jason Gunthorpe wrote: > On Thu, Mar 21, 2013 at 11:39:47AM +0200, Michael S. Tsirkin wrote: > > On Thu, Mar 21, 2013 at 02:13:38AM -0700, Roland Dreier wrote: > > > On Thu, Mar 21, 2013 at 1:51 AM, Michael S. Tsirkin > > > wrote: > > > >> In that case, n

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 11:57:32AM -0600, Jason Gunthorpe wrote: > On Thu, Mar 21, 2013 at 07:42:37PM +0200, Michael S. Tsirkin wrote: > > > It doesn't actually, and our app would sometimes write to these pages. > > It simply does not care which version does the remote get in this case > > since w

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 11:21:50AM -0600, Jason Gunthorpe wrote: > On Thu, Mar 21, 2013 at 07:15:25PM +0200, Michael S. Tsirkin wrote: > > > No because application does this: > > init page > > > > ... > > > > after a lot of time > > > > .. > > > > register > > send > > unregister > > > > so i

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 11:11:15AM -0600, Jason Gunthorpe wrote: > On Thu, Mar 21, 2013 at 11:39:47AM +0200, Michael S. Tsirkin wrote: > > On Thu, Mar 21, 2013 at 02:13:38AM -0700, Roland Dreier wrote: > > > On Thu, Mar 21, 2013 at 1:51 AM, Michael S. Tsirkin > > > wrote: > > > >> In that case, n

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Roland Dreier
On Thu, Mar 21, 2013 at 1:51 AM, Michael S. Tsirkin wrote: >> In that case, no, I don't see any reason for LOCAL_WRITE, since the >> only RDMA operations that will access this memory are remote reads. > > What is the meaning of LOCAL_WRITE then? There are no local > RDMA writes as far as I can see

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Roland Dreier
On Wed, Mar 20, 2013 at 11:18 PM, Michael S. Tsirkin wrote: > core/umem.c seems to get the arguments to get_user_pages > in the reverse order: it sets writeable flag and > breaks COW for MAP_SHARED if and only if hardware needs to > write the page. > > This breaks memory overcommit for users such

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Roland Dreier
>> I think this change will break the case where userspace tries to >> register an MR with read-only permission, but intends locally through >> the CPU to write to the memory. > Shouldn't it set LOCAL_WRITE then? We're talking about the permissions for the register MR operation, right? (That's w

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 08:23:48AM -0400, Michael R. Hines wrote: > Yes, I'd be happy to try the patch. > > Got meetings all day.. but will dive in soon. The patch is unlikely to be the final version. In particular you need to change !umem->writable to umem->writable. > On 03/21/2013 02:18 A

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael R. Hines
Yes, I'd be happy to try the patch. Got meetings all day.. but will dive in soon. On 03/21/2013 02:18 AM, Michael S. Tsirkin wrote: core/umem.c seems to get the arguments to get_user_pages in the reverse order: it sets writeable flag and breaks COW for MAP_SHARED if and only if hardware nee

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 11:32:30AM +0200, Michael S. Tsirkin wrote: > On Wed, Mar 20, 2013 at 11:55:54PM -0700, Roland Dreier wrote: > > On Wed, Mar 20, 2013 at 11:18 PM, Michael S. Tsirkin > > wrote: > > > core/umem.c seems to get the arguments to get_user_pages > > > in the reverse order: it se

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 02:13:38AM -0700, Roland Dreier wrote: > On Thu, Mar 21, 2013 at 1:51 AM, Michael S. Tsirkin wrote: > >> In that case, no, I don't see any reason for LOCAL_WRITE, since the > >> only RDMA operations that will access this memory are remote reads. > > > > What is the meaning

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Wed, Mar 20, 2013 at 11:55:54PM -0700, Roland Dreier wrote: > On Wed, Mar 20, 2013 at 11:18 PM, Michael S. Tsirkin wrote: > > core/umem.c seems to get the arguments to get_user_pages > > in the reverse order: it sets writeable flag and > > breaks COW for MAP_SHARED if and only if hardware needs

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Thu, Mar 21, 2013 at 12:15:33AM -0700, Roland Dreier wrote: > >> I think this change will break the case where userspace tries to > >> register an MR with read-only permission, but intends locally through > >> the CPU to write to the memory. > > > Shouldn't it set LOCAL_WRITE then? > > We're t

Re: [Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-21 Thread Michael S. Tsirkin
On Wed, Mar 20, 2013 at 11:55:54PM -0700, Roland Dreier wrote: > On Wed, Mar 20, 2013 at 11:18 PM, Michael S. Tsirkin wrote: > > core/umem.c seems to get the arguments to get_user_pages > > in the reverse order: it sets writeable flag and > > breaks COW for MAP_SHARED if and only if hardware needs

[Qemu-devel] [PATCH] rdma: don't make pages writeable if not requiested

2013-03-20 Thread Michael S. Tsirkin
core/umem.c seems to get the arguments to get_user_pages in the reverse order: it sets writeable flag and breaks COW for MAP_SHARED if and only if hardware needs to write the page. This breaks memory overcommit for users such as KVM: each time we try to register a page to send it to remote, this b