I don't think that will fix this issue. In this case yama is not installed and it appears PR_SET_PTRACER is not available. This forces vader to assume that CMA can not be used when that isn't always the case. I think it might be safe to assume that CMA is unrestricted here.
-Nathan On Thu, Feb 19, 2015 at 04:35:00PM -0500, Aurélien Bouteiller wrote: > Nathan, > > I think I already pushed a patch for this particular issue last month. I do > not know if it has been back ported to release yet. > > See > here:https://github.com/open-mpi/ompi/commit/ee3b0903164898750137d3b71a8f067e16521102 > > Aurelien > > -- > ~~~ Aurélien Bouteiller, Ph.D. ~~~ > ~ Research Scientist @ ICL ~ > The University of Tennessee, Innovative Computing Laboratory > 1122 Volunteer Blvd, suite 309, Knoxville, TN 37996 > tel: +1 (865) 974-9375 fax: +1 (865) 974-8296 > https://icl.cs.utk.edu/~bouteill/ > > > > > > Le 19 févr. 2015 à 15:53, Nathan Hjelm <hje...@lanl.gov> a écrit : > > > > > > Great! I will add an MCA variable to force CMA and also enable it if 1) > > no yama and 2) no PR_SET_PTRACER. > > > > You might also look at using xpmem. You can find a version that supports > > 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > > userspace library that can be used by vader as a single-copy mechanism. > > > > In benchmarks it performs better than CMA but it may or may not perform > > better with a real application. > > > > See: > > > > http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > > > -Nathan > > > > On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > >> On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > >>> On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > >>>> > >>>> On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > >>>>> > >>>>> If you have yama installed you can try: > >>>> > >>>> Nope, I do not have it installed... is it absolutely necessary? (and > >>>> would > >>>> it change something when it fails when I am root?) > >>>> > >>>> Other question: In addition to "--with-cma" configure flag, do we have to > >>>> pass any options to "mpicc" when compiling/linking an mpi application to > >>>> use > >>>> cma? > >>> > >>> No. CMA should work out of the box. You appear to have a setup I haven't > >>> yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > >>> prctl. Its quite possible there are no restriction on ptrace in this > >>> setup. Can you try changing the following line at > >>> opal/mca/btl/vader/btl_vader_component.c:370 from: > >>> > >>> bool cma_happy = false; > >>> > >>> to > >>> > >>> bool cma_happy = true; > >>> > >> > >> ok! (as of the officiel release, this is line 386.) > >> > >>> and let me know if that works. If it does I will update vader to allow > >>> CMA in this configuration. > >> > >> Yep! It now works perfectly. Testing with > >> https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my > >> own computer (dual Xeon), I have this: > >> > >> Without CMA: > >> > >> ***Message size: 1000000 *** best / avg / worst (MB/sec) > >> task pair: 0 - 1: 8363.52 / 7946.77 / 5391.14 > >> > >> with CMA: > >> task pair: 0 - 1: 9137.92 / 8955.98 / 7489.83 > >> > >> Great! > >> > >> Now I have to bench my real application... ;-) > >> > >> Thanks! > >> > >> Eric > >> > >> _______________________________________________ > >> users mailing list > >> us...@open-mpi.org > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > >> Link to this post: > >> http://www.open-mpi.org/community/lists/users/2015/02/26355.php > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2015/02/26356.php > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26358.php
pgp7NUNlOKKzV.pgp
Description: PGP signature
