On Jun 20, 2016, at 4:27 PM, Alex A. Granovsky <g...@classic.chem.msu.su> wrote:
> Would be the use of  mlockall helpful for this approach?

That's an interesting idea; I didn't know about the existence of 

It has a few drawbacks, of course (e.g., processes can't shrink), but they 
might be acceptable tradeoffs for typical HPC/MPI application scenarios.

Keep in mind that locking memory is only one half of the registration process: 
the other half is notifying the network stack so that they can record/track 
virtual memory-->physical memory mapping.  Meaning: Open MPI will still need 
its registration cache infrastructure -- but it *might* be able to be slightly 
simpler because the eviction mechanisms will never be invoked.

Also keep in mind that the memory mechanisms -- regardless of whether it's 
mlockall(MCL_FUTURE) or the newly-revamped "patcher" system in the upcoming 
Open MPI v2.0.0 -- are not in the critical performance path (e.g., that stuff 
doesn't happen during a call to MPI_SEND).  The part that *is* in the critical 
performance path is the registration cache -- i.e., the part where Open MPI 
asks "is this buffer/memory registered [with the network stack]?"  That part is 
designed to be fast and, at least at the moment, will still need to be there.

If there's some kind of equivalent to mlockall(MCL_FUTURE) that *also* 
transparently registers all new memory with the relevant underlying network 
stack(s) and contexts, that would be neat.  

Jeff Squyres
For corporate legal information go to: 

Reply via email to