Maybe it is a stupid question, but... why it is not tested and enabled
by default at configure time since it is part of the kernel?
Eric
On 02/19/2015 03:53 PM, Nathan Hjelm wrote:
Great! I will add an MCA variable to force CMA and also enable it if 1)
no yama and 2) no PR_SET_PTRACER.
You might also look at using xpmem. You can find a version that supports
3.x @ https://github.com/hjelmn/xpmem . It is a kernel module +
userspace library that can be used by vader as a single-copy mechanism.
In benchmarks it performs better than CMA but it may or may not perform
better with a real application.
See:
http://blogs.cisco.com/performance/the-vader-shared-memory-transport-in-open-mpi-now-featuring-3-flavors-of-zero-copy
-Nathan
On Thu, Feb 19, 2015 at 03:32:43PM -0500, Eric Chamberland wrote:
On 02/19/2015 02:58 PM, Nathan Hjelm wrote:
On Thu, Feb 19, 2015 at 12:16:49PM -0500, Eric Chamberland wrote:
On 02/19/2015 11:56 AM, Nathan Hjelm wrote:
If you have yama installed you can try:
Nope, I do not have it installed... is it absolutely necessary? (and would
it change something when it fails when I am root?)
Other question: In addition to "--with-cma" configure flag, do we have to
pass any options to "mpicc" when compiling/linking an mpi application to use
cma?
No. CMA should work out of the box. You appear to have a setup I haven't
yet tested. It doesn't have yama nor does it have the PR_SET_PTRACER
prctl. Its quite possible there are no restriction on ptrace in this
setup. Can you try changing the following line at
opal/mca/btl/vader/btl_vader_component.c:370 from:
bool cma_happy = false;
to
bool cma_happy = true;
ok! (as of the officiel release, this is line 386.)
and let me know if that works. If it does I will update vader to allow
CMA in this configuration.
Yep! It now works perfectly. Testing with
https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_bandwidth.c, on my
own computer (dual Xeon), I have this:
Without CMA:
***Message size: 1000000 *** best / avg / worst (MB/sec)
task pair: 0 - 1: 8363.52 / 7946.77 / 5391.14
with CMA:
task pair: 0 - 1: 9137.92 / 8955.98 / 7489.83
Great!
Now I have to bench my real application... ;-)
Thanks!
Eric
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/02/26355.php
