Dear Open MPI developers, there is already a thread about 'sm BTL performace of the openmpi-2.0.0' https://www.open-mpi.org/community/lists/devel/2016/07/19288.php and we also see 30% bandwidth loss, on communication *via InfiniBand*.
And we also have a clue: the IB buffers seem not to be aligned in 2.0.0 - in contrast to previous series (from at least 1.8.x).
That means,- if we use a simple wrapper wrapping 'malloc' to the 32-bit-aligned-variant, we get the full bandwidth using the same compiled binary; and - there is nothing to grep in 'ompi_info -all | grep memalign' in 2.0.0 while in 1.10.3 there are 'btl_openib_memalign' and 'btl_openib_memalign_threshold' parameters.
=> seem the whole 'IB buffer alignment' part vanished in /2.0.0 ?Could we get the aligned IB buffers in 2.x series back, please? It's about 30% of performance....
Best PaulP.S. btl_openib_get_alignment and btl_openib_put_alignment are by default '0' - setting they high did not change the behaviour...
-- Dipl.-Inform. Paul Kapinos - High Performance Computing, RWTH Aachen University, IT Center Seffenter Weg 23, D 52074 Aachen (Germany) Tel: +49 241/80-24915
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ devel mailing list devel@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
