On Apr 6, 2008, at 5:58 PM, Teng Lin wrote:
In order to run a 32-bit program on a 64-bit cluster, one has to build 32-bit OpenMPI. Following some instructions on this mailing list, I successfully built OpenMPI 1.2.4 on 64-bit OS.
It's not clear from your text -- did you build Open MPI as 32 or 64 bit?
However, I run into openib problem when I try to run hello_c program. I also built 64-bit OpenMPI from same source. The interesting fact is 64-bit OpenMPI works just fine. Below is the output from orterun, ############################################################################ iceland:/home/tlin/test_pbs>/home/tin/openmpi-1.2.4/bin/orterun -np 2 --hostfile mach.lst /home/tlin/test_pbs/hello_c.32 -------------------------------------------------------------------------- The OpenIB BTL failed to initialize while trying to create an internal queue. This typically indicates a failed OpenFabrics installation or faulty hardware. The failure occured here: Host: cl1n004 OMPI source: btl_openib.c:828 Function: ibv_create_cq() Error: Invalid argument (errno=22) Device: mthca0 You may need to consult with your system administrator to get this problem fixed. --------------------------------------------------------------------------
[snip]
###################################################################### I saw this error on before on other cluster. Following the instruction on (http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages) does fix the problem. However, I doubt that is the reason why 32 bit OpenMPI does not work on this cluster. Output from limit looks fine to me. And if that is the case, 64 bit OpenMPI will not work. Any ideas?
I'm unclear exactly what you're saying is happening -- can you clarify? - 32 bit Open MPI does not build at all - 64 bit Open MPI builds but yields the error message shown above Is that right? Can you send all the information listed here: http://www.open-mpi.org/community/help/ -- Jeff Squyres Cisco Systems
