On Thu, Nov 15, 2018 at 04:37:21PM +0000, Hefty, Sean wrote: > > However, the verbs extensions in the header verbs_exp.h provide > > additional functionality. Among other things this includes the > > capability to register a physical address as MR instead of a virtual > > one. For Mellanox drivers this is described in [1]. The call is > > ibv_exp_reg_mr with the IBV_EXP_ACCESS_PHYSICAL_ADDR flag. > > > > Using the physical address for the reg_mr call we can then transfer > > data directly from our custom PCIe card to the network adapter, > > without any copies. Pretty nice! > > > > To use the ibv_exp_reg_mr call I had to slightly patch libfabric, see > > [2]. Libfabric already uses the ibv_exp_reg_mr call if the verbs_exp.h > > is available, so just the flag needs to be added. > > > > As for my feature request, can this be added as official feature to > > libfabric? I am not sure how many other providers could actually > > support physical address memory registration, but it is a nice feature > > when working with custom hardware and drivers. My patch is of course a > > bit "crude" and this would need to be properly implemented. > > > > Thanks. > > > > Looking at the changes, I see a couple of options. We can add this > as an extension for verbs only. This could make use of the provider > specific flag bits passed into the mr_reg call. Or, we can define > this as a generic mr_reg flag.
This is not something I expect we will ever be supporting in upstream kernel verbs - it simply too insecure. I don't think libfabric should think about something we in the kernel would not allow *any* libfabric provider to do. Jason _______________________________________________ ofiwg mailing list [email protected] https://lists.openfabrics.org/mailman/listinfo/ofiwg
