We compiled OpenMPI 2.1.1 using the EasyBuild configuration for CentOS as below and tested on Mellanox QDR hardware.
./configure --prefix=/homes/daveturner/libs/openmpi-2.1.1c
--enable-shared
--enable-mpi-thread-multiple
--with-verbs
--enable-mpirun-prefix-by-default
--with-mpi-cxx
--enable-mpi-cxx
--with-hwloc=$EBROOTHWLOC
--disable-dlopen
The red curve in the attached NetPIPE graph shows the poor performance above
8 kB for the uni-directional tests with bi-directional and aggregate
tests also showing similar problems. When I compile using the same
configuration but with the --disable-dlopen parameter removed then the
performance is very good as the green curve in the graph shows.
We see the same problems with OpenMPI 2.0.2.
Replacing --disable-dlopen with --disable-mca-dso showed good performance.
Replacing --disable-dlopen with --enable-static showed good performance.
So it's only --disable-dlopen that leads to poor performance.
http://netpipe.cs.ksu.edu
Dave Turner
--
Work: davetur...@ksu.edu (785) 532-7791
2219 Engineering Hall, Manhattan KS 66506
Home: drdavetur...@gmail.com
cell: (785) 770-5929
MPI_on_QDR_dlopen_paramter.pdf
Description: Adobe PDF document
_______________________________________________ devel mailing list devel@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/devel
