OK, I think I got things sorted out. Thanx for your inputs.
Ted
--- On Fri, 2/6/09, Jeff Squyres <jsquy...@cisco.com> wrote:
From: Jeff Squyres <jsquy...@cisco.com>
Subject: Re: [OMPI users] Global Communicator
To: "Open MPI Users" <us...@open-mpi.org>
Cc: ted...@wag.caltech.edu
List-Post: users@lists.open-mpi.org
Date: Friday, February 6, 2009, 2:46 PM
On Feb 6, 2009, at 11:59 AM, McCalla, Mac wrote:
> Does the "default mpirun command" implementation match the build
environment for quest_ompi.x ?
> ie., what mpi implementation (mpich, LAM/MPI, OPENMPI, or other) was
quest_ompi.x compiled/linked with? and does that match the result of
"which mpirun"? You might try running a job using your PBS system
that simply executes the which mpirun command and see what you get.
Mac is dead-on in his analysis.
You must compile your MPI application with exactly the same MPI implementation
that you run it -- there is currently no binary compatibility between MPI
implementations. So if you compile your MPI application with Open MPI, then you
must run it with Open MPI's mpirun/mpiexec (which, as the man page notes,
are exact synonyms of each other in Open MPI -- this is not necessarily true in
other MPI implementations).
If you compile your application with MPICH, then you must use MPICH's
mpirun/mpiexec -- and you'll need to ask them for help if things don't
work (i.e., it's an entirely different project and we're not qualified
to give you help with their software).
Finally, note that Open MPI "understands" PBS/Torque. If your system
administrator build Open MPI with PBS/Torque support, you don't need to list
the hostfile on Open MPI's mpirun/mpiexec command line -- Open MPI will
"just figure it out" and know a) what hosts to use and b) how many
processes to launch. Specifically, you can use this:
mpiexec ${CODE} >/ul/tedhyu/fuelcell/HOH/test/HH.out
(or, as Mac suggests, you might want to provide a full path name to mpiexec to
ensure that you're getting the "Right" mpiexec)
You can check your Open MPI installation to see if it has PBS/Torque support
with the following:
ompi_info | grep tm
If you see anything in the output like this (although the exact version numbers
may be different):
MCA ras: tm (MCA v2.0, API v2.0, Component v1.4)
then your Open MPI has this native PBS/Torque support built-in and you can
avoid specifying a hostfile and -np arguments.
--Jeff Squyres
Cisco Systems
