I don't know the reasoning for requiring --with-cma to enable CMA but I am looking at auto-detecting CMA instead of requiring Open MPI to be configured with --with-cma. This will likely go into the 1.9 release series and not 1.8.
-Nathan On Thu, Feb 19, 2015 at 09:31:43PM -0500, Eric Chamberland wrote: > Maybe it is a stupid question, but... why it is not tested and enabled by > default at configure time since it is part of the kernel? > > Eric > > > On 02/19/2015 03:53 PM, Nathan Hjelm wrote: > >Great! I will add an MCA variable to force CMA and also enable it if 1) > >no yama and 2) no PR_SET_PTRACER. > > > >You might also look at using xpmem. You can find a version that supports > >3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > >userspace library that can be used by vader as a single-copy mechanism. > > > >In benchmarks it performs better than CMA but it may or may not perform > >better with a real application. > > > >See: > > > >http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > > >-Nathan > > > >On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > >>On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > >>>On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > >>>>On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > >>>>>If you have yama installed you can try: > >>>>Nope, I do not have it installed... is it absolutely necessary? (and would > >>>>it change something when it fails when I am root?) > >>>> > >>>>Other question: In addition to "--with-cma" configure flag, do we have to > >>>>pass any options to "mpicc" when compiling/linking an mpi application to > >>>>use > >>>>cma? > >>>No. CMA should work out of the box. You appear to have a setup I haven't > >>>yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > >>>prctl. Its quite possible there are no restriction on ptrace in this > >>>setup. Can you try changing the following line at > >>>opal/mca/btl/vader/btl_vader_component.c:370 from: > >>> > >>>bool cma_happy = false; > >>> > >>>to > >>> > >>>bool cma_happy = true; > >>> > >>ok! (as of the officiel release, this is line 386.) > >> > >>>and let me know if that works. If it does I will update vader to allow > >>>CMA in this configuration. > >>Yep! It now works perfectly. Testing with > >>https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my > >>own computer (dual Xeon), I have this: > >> > >>Without CMA: > >> > >>***Message size: 1000000 *** best / avg / worst (MB/sec) > >> task pair: 0 - 1: 8363.52 / 7946.77 / 5391.14 > >> > >>with CMA: > >> task pair: 0 - 1: 9137.92 / 8955.98 / 7489.83 > >> > >>Great! > >> > >>Now I have to bench my real application... ;-) > >> > >>Thanks! > >> > >>Eric > >> > >>_______________________________________________ > >>users mailing list > >>us...@open-mpi.org > >>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > >>Link to this post: > >>http://www.open-mpi.org/community/lists/users/2015/02/26355.php > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26362.php
pgpKWremq_SLC.pgp
Description: PGP signature
