On 7/29/10 3:41 PM, Jason Gunthorpe wrote:
On Thu, Jul 29, 2010 at 03:29:37PM -0500, Tom Tucker wrote:
Also, I'd like to see a strong defence of this new user space API
particularly:
1) Why can't this be done with the existing ibv_reg_mr, like huge
pages are.
The ibv_reg_mr API assumes that the memory being registered was
allocated in user mode and is part of the current->mm VMA. It uses
get_user_pages which will scoff and jeer at kernel memory.
I'm confused? What is the vaddr input then? How does userspace get
that value? Isn't it created by mmap or the like?
Yes.
Ie for the PCI-E example you gave I assume the flow is that userspace
mmaps devices/pci0000:00/0000:00:XX.X/resourceX to get the IO memory
and then passes that through to ibv_reg_mr?
Not exactly. It would mmap the device that manages the adapter hosting
the memory.
IMHO, ibv_reg_mr API should accept any valid vaddr available to the
process and if it bombs for certain kinds of vaddrs then it is just a
bug..
Perhaps.
2) How is it possible for userspace to know when it should use
ibv_reg_mr vs ibv_reg_io_mr?
By virtue of the device that it is mmap'ing. If I mmap my_vmstat_driver,
I know that the memory I am mapping is a kernel buffer.
Yah, but what if the next version of your vmstat driver changes the
kind of memory it returns?
It's a general service for a class of memory, not an enabler for a
particular application's peculiarities.
On first glance, this seems like a hugely bad API to me :)
Well hopefully now that it's purpose is revealed you will change your
view and we can collaboratively make it better :-)
I don't object to the idea, just to the notion that user space is
supposed to somehow know that one vaddr is different from another
vaddr and call the right API - seems impossible to use correctly to
me.
What would you have to do to implement this using scheme using
ibv_reg_mr as the entry point?
The kernel service on the other side of ibv_reg_mr verb could divine the
necessary information by searching all vma owned by current and looking
at vma_flags to decide what type it was.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
