On 11/19/2018 08:42 PM, Hefty, Sean wrote:
The only alternative I can think of is to try a normal registration
call, and if that fails, try again using the physical flag. Would
this work, or does the normal registration call succeed, but produce
an unusable MR?
This would not work because of a subtlety of the physical memory
registration. The reason is that actually NULL is passed as address in
the call. Check the github link to my patch in the other E-Mail, there
is a line that replaces the address with NULL.
If a user passes an illegal virtual address the call should fail. But
if the libfabric call falls back to the physical address registration,
this would then actually succeed as the address is replaced with NULL.
I looked back at the patches and related documentation. IMO, the verbs
physical memory registration interface is just weird. There is no association
between the actual pages and the region AFAICT.
Indeed this is a rather strange extension.
I came across a potential solution to adjust our driver to produce
memory that is compatible with the RDMA stack in the kernel. Supposedly
there is an alternative to remap_pfn_range. In that case we would not
need the physical memory registration in libfabric anymore and the
overall solution would be cleaner (not dependent on the verbs provider)
I will validate this and report back here. Thanks so far for the help.
Cheers,
Jörn
_______________________________________________
ofiwg mailing list
[email protected]
https://lists.openfabrics.org/mailman/listinfo/ofiwg
