On 11/26/2018 07:44 PM, Jason Gunthorpe wrote:
On Mon, Nov 26, 2018 at 04:42:49PM +0100, Jörn Schumacher wrote:
On 11/19/2018 08:42 PM, Hefty, Sean wrote:
The only alternative I can think of is to try a normal registration
call, and if that fails, try again using the physical flag. Would
this work, or does the normal registration call succeed, but produce
an unusable MR?
This would not work because of a subtlety of the physical memory
registration. The reason is that actually NULL is passed as address in
the call. Check the github link to my patch in the other E-Mail, there
is a line that replaces the address with NULL.
If a user passes an illegal virtual address the call should fail. But
if the libfabric call falls back to the physical address registration,
this would then actually succeed as the address is replaced with NULL.
I looked back at the patches and related documentation. IMO, the verbs
physical memory registration interface is just weird. There is no association
between the actual pages and the region AFAICT.
Indeed this is a rather strange extension.
I came across a potential solution to adjust our driver to produce memory
that is compatible with the RDMA stack in the kernel. Supposedly there is an
alternative to remap_pfn_range. In that case we would not need the physical
memory registration in libfabric anymore and the overall solution would be
cleaner (not dependent on the verbs provider)
Several other people have been interested in this, I think many would
appreciate it if you share your solution to the linux-rdma mailing
list.
Yes, I was thinking of writing this up for the list, will do that in the
next couple of days after running some tests.
Jörn
_______________________________________________
ofiwg mailing list
[email protected]
https://lists.openfabrics.org/mailman/listinfo/ofiwg
