Ben,
You may try to disable registration cache, it may relieve pressure on memory
resources.
--mca mpi_leave_pinned 0
You may find a bit more details here:
http://www.open-mpi.org/faq/?category=openfabrics#large-message-leave-pinned
Using the option you may observe drop in BW performance.
Hi,
I would suggest use MXM (part of mofed, can be downloaded as standalone rpm
from http://mellanox.com/products/mxm for ofed)
It uses UD (constant memory footprint) and should provide good performance.
The next MXM v2.0 will support RC and DC (reliable UD) as well.
Once mxm is installed from
I'm part of a team that maintains a global climate model running under
mpi. Recently we have been trying it out with different mpi stacks
at high resolution/processor counts.
At one point in the code there is a large number of mpi_isends/mpi_recv
(tens to hundreds of thousands) when data
