Hello,
Thank for the answer.
I am testing with OpenMPI-1.4.3: my computation is queuing. But I did
not read anything obvious related to my issue. Have you read something
which could solve it?
I am going to submit my computation with --mca mpi_leave_pinned 0, but
do you have any idea how it affect the performance? Compared to using
Ethernet?
Many thanks for your support.
On 06/23/2011 03:01 PM, Josh Hursey wrote:
I wonder if this is related to memory pinning. Can you try turning off
the leave pinned, and see if the problem persists (this may affect
performance, but should avoid the crash):
mpirun ... --mca mpi_leave_pinned 0 ...
Also it looks like Smoky has a slightly newer version of the 1.4
branch that you should try to switch to if you can. The following
command will show you all of the available installs on that machine:
shell$ module avail ompi
For a list of supported compilers for that version try the 'show' option:
shell$ module show ompi/1.4.3
-------------------------------------------------------------------
/sw/smoky/modulefiles-centos/ompi/1.4.3:
module-whatis This module configures your environment to make Open
MPI 1.4.3 available.
Supported Compilers:
pathscale/3.2.99
pathscale/3.2
pgi/10.9
pgi/10.4
intel/11.1.072
gcc/4.4.4
gcc/4.4.3
-------------------------------------------------------------------
Let me know if that helps.
Josh
On Wed, Jun 22, 2011 at 4:16 AM, Mathieu Gontier
<mathieu.gont...@gmail.com> wrote:
Dear all,
First of all, all my apologies because I post this message to both the bug
and user mailing list. But for the moment, I do not know if it is a bug!
I am running a CFD structured flow solver at ORNL, and I have an access to a
small cluster (Smoky) using OpenMPI-1.4.2 with Infiniband by default.
Recently we increased the size of our models, and since that time we have
run into many infiniband related problems. The most serious problem is a
hard crash with the following error message:
[smoky45][[60998,1],32][/sw/sources/ompi/1.4.2/ompi/mca/btl/openib/connect/btl_openib_connect_oob.c:464:qp_create_one]
error creating qp errno says Cannot allocate memory
If we force the solver to use ethernet (mpirun -mca btl ^openib) the
computations works correctly, although very slowly (a single iteration take
ages). Do you have any idea what could be causing these problems?
If it is due to a bug or a limitation into OpenMPI, do you think the version
1.4.3, the coming 1.4.4 or any 1.5 version could solve the problem? I read
the release notes, but I did not read any obvious patch which could fix my
problem. The system administrator is ready to compile a new package for us,
but I do not want to ask to install to many of them.
Thanks.
--
Mathieu Gontier
skype: mathieu_gontier
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
/
Mathieu Gontier
skype: mathieu_gontier /
