Re: [ofa-general] mmap() and ibv_reg_mr() and RDMA
On Tue, 22 May 2007 15:19:15 -0700 Roland Dreier <[EMAIL PROTECTED]> wrote: > > this area is mapped via the character device and with > > the help of remap_pfn_range() into userspace... this works fine i > > can access it from userspace and write/read from it: > > I think that's the problem. remap_pfn_range() sets VM_PFNMAP on the > vma used to map the pfns. When ibv_reg_mr() calls into the kernel to > do the actual mapping, it ends up doing get_user_pages() which fails > in vm_normal_page() for such a vma. > > I don't immediately see a good way to handle this. > many thanks for your fast answer. i will try to access the memory via get_user_pages() too instead of mmap'ing it... just the other way around - should be no problem. regards, joerg ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] mmap() and ibv_reg_mr() and RDMA
> this area is mapped via the character device and with > the help of remap_pfn_range() into userspace... this works fine i can > access it from userspace and write/read from it: I think that's the problem. remap_pfn_range() sets VM_PFNMAP on the vma used to map the pfns. When ibv_reg_mr() calls into the kernel to do the actual mapping, it ends up doing get_user_pages() which fails in vm_normal_page() for such a vma. I don't immediately see a good way to handle this. - R. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] mmap() and ibv_reg_mr() and RDMA
On Tue, 22 May 2007 13:46:04 -0700
Roland Dreier <[EMAIL PROTECTED]> wrote:
> > I want to do RDMA-write on mmap'ed memory.
> > but it fails to register the memory region.
> >
> > is there something special, to use ibv_reg_mr() on memory which I
> > got from mmap()?
> > it works fine with plain allocated memory (with memalign()).
> > memory is page-aligned in both cases.
>
> How exactly are you mmap()ing the memory? memalign(), malloc() etc
> are implemented with mmap() internally, so obviously memory
> registration of some mmap()ed memory is fine.
i created a character device in the kernel and registered memory with
kzalloc():
if ((kmalloc_ptr = kzalloc((NPAGES + 2) * PAGE_SIZE, GFP_KERNEL |
__GFP_DMA)) == NULL) { return -ENOMEM; }
rounded to page bondary:
kmalloc_area = (struct serverinfo *)unsigned long)kmalloc_ptr) +
PAGE_SIZE - 1) & PAGE_MASK);
this area is mapped via the character device and with
the help of remap_pfn_range() into userspace... this works fine i can
access it from userspace and write/read from it:
#define MMAP_AREA_LEN (NPAGES*getpagesize())
...
mmap_area = (struct serverinfo*)mmap(0, MMAP_AREA_LEN,
PROT_READ|PROT_WRITE, MAP_SHARED| MAP_LOCKED, fd, MMAP_AREA_LEN);
but when i try to register the mmap_area with ibv_reg_mr() it
fails.
regards,
joerg
___
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] mmap() and ibv_reg_mr() and RDMA
> I want to do RDMA-write on mmap'ed memory. > but it fails to register the memory region. > > is there something special, to use ibv_reg_mr() on memory which I got > from mmap()? > it works fine with plain allocated memory (with memalign()). > memory is page-aligned in both cases. How exactly are you mmap()ing the memory? memalign(), malloc() etc are implemented with mmap() internally, so obviously memory registration of some mmap()ed memory is fine. - R. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
