Aurélien, I should also point out your fix has already been applied to the 1.8 branch and will be included in 1.8.5.
-Nathan On Thu, Feb 19, 2015 at 02:57:38PM -0700, Nathan Hjelm wrote: > > Hmm, wait. Yes. Your change went in after 1.8.4 and has the same > effect. If yama ins't installed it is safe to assume that the ptrace > scope is effectively 0. So, your patch does fix the issue. > > -Nathan > > On Thu, Feb 19, 2015 at 02:53:47PM -0700, Nathan Hjelm wrote: > > > > I don't think that will fix this issue. In this case yama is not > > installed and it appears PR_SET_PTRACER is not available. This forces > > vader to assume that CMA can not be used when that isn't always the > > case. I think it might be safe to assume that CMA is unrestricted here. > > > > -Nathan > > > > On Thu, Feb 19, 2015 at 04:35:00PM -0500, Aurélien Bouteiller wrote: > > > Nathan, > > > > > > I think I already pushed a patch for this particular issue last month. I > > > do not know if it has been back ported to release yet. > > > > > > See > > > here:https://github.com/open-mpi/ompi/commit/ee3b0903164898750137d3b71a8f067e16521102 > > > > > > Aurelien > > > > > > -- > > > ~~~ Aurélien Bouteiller, Ph.D. ~~~ > > > ~ Research Scientist @ ICL ~ > > > The University of Tennessee, Innovative Computing Laboratory > > > 1122 Volunteer Blvd, suite 309, Knoxville, TN 37996 > > > tel: +1 (865) 974-9375 fax: +1 (865) 974-8296 > > > https://icl.cs.utk.edu/~bouteill/ > > > > > > > > > > > > > > > > Le 19 févr. 2015 à 15:53, Nathan Hjelm <hje...@lanl.gov> a écrit : > > > > > > > > > > > > Great! I will add an MCA variable to force CMA and also enable it if 1) > > > > no yama and 2) no PR_SET_PTRACER. > > > > > > > > You might also look at using xpmem. You can find a version that supports > > > > 3.x @ https://github.com/hjelmn/xpmem . It is a kernel module + > > > > userspace library that can be used by vader as a single-copy mechanism. > > > > > > > > In benchmarks it performs better than CMA but it may or may not perform > > > > better with a real application. > > > > > > > > See: > > > > > > > > http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy > > > > > > > > -Nathan > > > > > > > > On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote: > > > >> On 02/19/2015 02:58 PM, Nathan Hjelm wrote: > > > >>> On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote: > > > >>>> > > > >>>> On 02/19/2015 11:56 AM, Nathan Hjelm wrote: > > > >>>>> > > > >>>>> If you have yama installed you can try: > > > >>>> > > > >>>> Nope, I do not have it installed... is it absolutely necessary? (and > > > >>>> would > > > >>>> it change something when it fails when I am root?) > > > >>>> > > > >>>> Other question: In addition to "--with-cma" configure flag, do we > > > >>>> have to > > > >>>> pass any options to "mpicc" when compiling/linking an mpi > > > >>>> application to use > > > >>>> cma? > > > >>> > > > >>> No. CMA should work out of the box. You appear to have a setup I > > > >>> haven't > > > >>> yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER > > > >>> prctl. Its quite possible there are no restriction on ptrace in this > > > >>> setup. Can you try changing the following line at > > > >>> opal/mca/btl/vader/btl_vader_component.c:370 from: > > > >>> > > > >>> bool cma_happy = false; > > > >>> > > > >>> to > > > >>> > > > >>> bool cma_happy = true; > > > >>> > > > >> > > > >> ok! (as of the officiel release, this is line 386.) > > > >> > > > >>> and let me know if that works. If it does I will update vader to allow > > > >>> CMA in this configuration. > > > >> > > > >> Yep! It now works perfectly. Testing with > > > >> https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on > > > >> my > > > >> own computer (dual Xeon), I have this: > > > >> > > > >> Without CMA: > > > >> > > > >> ***Message size: 1000000 *** best / avg / worst (MB/sec) > > > >> task pair: 0 - 1: 8363.52 / 7946.77 / 5391.14 > > > >> > > > >> with CMA: > > > >> task pair: 0 - 1: 9137.92 / 8955.98 / 7489.83 > > > >> > > > >> Great! > > > >> > > > >> Now I have to bench my real application... ;-) > > > >> > > > >> Thanks! > > > >> > > > >> Eric > > > >> > > > >> _______________________________________________ > > > >> users mailing list > > > >> us...@open-mpi.org > > > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > > >> Link to this post: > > > >> http://www.open-mpi.org/community/lists/users/2015/02/26355.php > > > > _______________________________________________ > > > > users mailing list > > > > us...@open-mpi.org > > > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > Link to this post: > > > > http://www.open-mpi.org/community/lists/users/2015/02/26356.php > > > > > > _______________________________________________ > > > users mailing list > > > us...@open-mpi.org > > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > > Link to this post: > > > http://www.open-mpi.org/community/lists/users/2015/02/26358.php > > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2015/02/26359.php > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26360.php
pgpKlnnBA6kvo.pgp
Description: PGP signature
