Note that (beginning with 1.3) you can also use "platform files" to
save configure and default mca params so that you build consistently.
Check the examples in contrib/platform. Most of us developers use
these religiously, as do our host organizations, for precisely this
reason.
I believe there should be something on the FAQ about platform files -
if not, I'll try to add it in the next few days.
If you want to contribute platform files to support some community
with similar configurations, please send them to me - we shouldn't
need a contributors agreement for them as there is no code involved.
Ralph
On Feb 20, 2009, at 8:23 AM, Gus Correa wrote:
Hi Gerry
I usually put configure commands (and environment variables)
on little shell scripts, which I edit to fit the combination
of hardware/compiler(s), and keep them in the build directory.
Otherwise I would forget the details next time I need to build.
If Myrinet and GigE are on separate clusters,
you'll have to install OpenMPI on each one, sorry.
However, if Myrinet and GigE are available on the same cluster,
you can build a single OpenMPI,
and choose the "byte transport layer (BTL)"
to be Myrinet or GigE (or IB, for that matter),
and even the NICs/networks to use,
on your job submission script.
Check the OpenMPI FAQ:
http://www.open-mpi.org/faq/?category=myrinet#myri-btl-gm
http://www.open-mpi.org/faq/?category=tcp#tcp-btl
http://www.open-mpi.org/faq/?category=openfabrics#ib-btl
http://www.open-mpi.org/faq/?category=tcp#tcp-multi-network
http://www.open-mpi.org/faq/?category=tcp#tcp-selection
Gus Correa
PS - BTW - Our old non-Rocks cluster has Myrinet-2000 (GM).
After I get the new cluster up and running and in production,
I am thinking of revamping the old cluster, and install Rocks on it.
I would love to learn from your experience with your
Rocks+Myrinet cluster, if you have the time to post a short
"bullet list" of "do's and don'ts".
(The Rocks list may be more appropriate than the OpenMPI for this.)
Last I checked Myrinet had a roll only for Rocks 5.0,
not 5.1, right?
Did you install it with on top of Rocks 5.0 or 5.1?
(For instance, my recollection of old postings on the list,
is that the Torque 5.0 roll worked with Rocks 5.1,
but it is always a risky business to mix different releases.)
Many thanks,
Gus Correa
---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------
Gerry Creager wrote:
Gus,
I'll give that a try real quick (or as quickly as the compiles can
run.
I'd not thought of this solution. I've been context-switching too
much lately. I've gotta look at this for a gigabit cluster as well.
Thanks!
Gus Correa wrote:
Hi Gerry
You may need to compile a hybrid OpenMPI
using gcc for C, PGI f90 for Fortran on the OpenMPI configure
script.
This should give you the required mpicc and mpif90 to do the job.
I guess this is what Elvedin meant on his message.
I have these hybrids for OpenMPI and MPICH2 here
(not Myrinet but GigE), and they work
fine with a WRF relative (CAM3, atmospheric climate).
Two cents from
Gus Correa
---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------
Gerry Creager wrote:
Elvedin,
Yeah, I thought about that after finding a reference to this in
the archives, so I redirected the path to MPI toward the gnu-
compiled version. It died in THIS manner:
make[3]: Entering directory `/home/gerry/WRFv3/WRFV3/external/
RSL_LITE'
mpicc -cc=gcc -DFSEEKO64_OK -w -O3 -DDM_PARALLEL -c c_code.c
pgcc-Error-Unknown switch: -cc=gcc
make[3]: [c_code.o] Error 1 (ignored)
Methinks the wrf configuration script and make file will need
some tweeks.
Interesting thing: I have another system (alas, with mpich) where
it compiles just fine. I'm trying to sort this out, as on 2
systems, with openMPI, it does odd dances before dying.
I'm still trying things. I've gotta get this up both for MY
research and to support other users.
Thanks, Gerry
Elvedin Trnjanin wrote:
WRF almost requires that you use gcc for the C/C++ part and the
PGI Fortran compilers, if you choose that option. I'd suggest
compiling OpenMPI in the same way as that has resolved our
various issues. Have you tried that with the same result?
Gerry Creager wrote:
Howdy,
I'm new to this list. I've done a little review but likely
missed something specific to what I'm asking. I'll keep
looking but need to resolve this soon.
I'm running a Rocks cluster (centos 5), with PGI 7.2-3
compilers, Myricom MX2 hardware and drivers, and OpenMPI1.3
I installed the Myricom roll which has OpenMPI compiled with
gcc. I recently compiled the openmpi code w/ PGI.
I've the MPICH_F90 pointing to the right place, and we're
looking for the right includes and libs by means of
LD_LIBRARY_PATH, etc.
When I tried to run, I got the following error:
make[3]: Entering directory `/home/gerry/WRFv3/WRFV3/external/
RSL_LITE'
mpicc -DFSEEKO64_OK -w -O3 -DDM_PARALLEL -c c_code.c
PGC/x86-64 Linux 7.2-3: compilation completed with warnings
mpicc -DFSEEKO64_OK -w -O3 -DDM_PARALLEL -c buf_for_proc.c
PGC-S-0036-Syntax error: Recovery attempted by inserting
identifier .Z0000 before '(' (/share/apps/openmpi-1.3-pgi/
include/mpi.h: 889)
PGC-S-0082-Function returning array not allowed (/share/apps/
openmpi-1.3-pgi/include/mpi.h: 889)
PGC-S-0043-Redefinition of symbol, MPI_Comm (/share/apps/
openmpi-1.3-pgi/include/mpi.h: 903)
PGC/x86-64 Linux 7.2-3: compilation completed with severe errors
make[3]: [buf_for_proc.o] Error 2 (ignored)
Note that I had modified the makefile to use PGI in place of
gcc, and thus, the PGI-compiled openMPI.
Thanks, Gerry
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
