Chris, If the sys call are there we should clearly take advantage of them. The patch looks good, I vote for it!
george. On Jan 12, 2012, at 04:34 , Christopher Yeoh wrote: > Hi Brad, > > WHAT: Adds Cross Memory Attach support to the sm btl > > WHY: For faster intranode communication > > WHERE: ompi/mca/btl/sm/ > > WHEN: Open MPI trunk > > TIMEOUT: 13/2/2012 > > Cross Memory Attach (CMA) is a pair of new syscalls (process_vm_readv > and process_vm_writev) which allow for fast intranode > communication. It has added to the Linux 3.2 kernel. There is a man page > for the new system calls here: > > http://ozlabs.org/~cyeoh/cma/process_vm_readv.txt > > Attached is a patch for the OMPI trunk tree which augments the sm btl > to use these calls. > > - CMA is quite similar in many respects to KNEM and what I've done is to > pretty much copy what KNEM does in many cases. > > - Both KNEM and CMA can be compiled in at the same time, though if > both are enabled at runtime, only KNEM runs. To enable CMA use > --mca btl_sm_use_cma 1 > > - To enable CMA at compile time, add --with-cma=yes to the configure > command line. Support for the syscalls is in the git glibc archive, > but its not yet out in the distros so as an interim > workaround I have added some arch/os specific wrappers which are used > if the syscalls are not found at configure time. The syscalls numbers > won't change as 3.2 is out. > > - I'm far from sure that the way I have used CMA in OMPI is the best > way to do it, so any feedback is very welcome. > > Regards, > > Chris > -- > cy...@au.ibm.com > > > <cma-openmpi-v1.patch>_______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
