On 6/8/2015 11:49 PM, Hefty, Sean wrote:
Sean,
IMO, we need to introduce vendor specific header files and interfaces.
It is unmaintainable to drive an API from the bottom up and expose the 'bare
metal'
implementation of a bunch of disjoint pieces of hardware. (Yeah, because we
need yet another way of registering memory...
just reading the process for a yet another set of hoops that an app must jump
through in order to register memory makes my head hurt.)
Everything about this continual approach needs to end.
I think that the related thread on this patchset makes it clear that
there is a fundamental limitation in the RDMA stack where it allows
to register page lists and not generic SG lists. I strongly disagree
that this is an exposure of some bare metal implementation.
Kernel 4.1 introduced the new pmem driver for byte addressable storage
(https://lwn.net/Articles/640115/). It won't be long before we see HA
models where secondary persistent memory devices will sit across an
RDMA fabric.
(http://www.snia.org/sites/default/files/DougVoigt_RDMA_Requirements_for_HA.pdf)
We cannot afford to work around the stack block alignment limitation
and meet the latency requirements of such fast devices. This means no
bounce-buffering what-so-ever and we need efficient remote access to
multiple byte ranges.
Moreover, this feature also opens fast registration to user-space (i.e.
"user-space FRWR"). user-space cannot use FRWR as it is not exposed to
the memory physical mapping. Indirect registration allows users to
fast register with ibv_sges. HPC applications need efficient access
to remote scatters.
Do we want to live with this limitation forever?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
