Thanks Jeff and Alex for your answers and comments. mlockall(), especially with the MCL_FUTURE argument is indeed interesting.
Thanks Jeff for your clarification of what memory registration really means (e.g. locking and telling the network stack the virtual to physical mapping). Also concerning the ummunotify kernel module, I would like to point out that while the link sent to github bug report suggests it is problematic, the top level Open MPI README file still recommends it. Should the README file need to be updated ? Regards, Martin Audet