Some supplementary information to the wiki at
https://svn.open-mpi.org/trac/ompi/wiki/CrayXT3.
I. Accessing the Open MPI source:
* Subversion is installed on redstorm in /projects/unsupported/bin
* Reddish has subversion in the default path (you don't have to load a
module)
* The proxy information in the wiki is accurate, and works on both
redstorm and reddish
II. Building Open MPI on reddish:
* 'module load PrgEnv-pgi-xc' to cross compile for redstorm
* Reddish and redstorm do not have recent enough version of autotools,
so you must build your own (currently available in
/project/openmpi/rbbrigh/tools)
* Suggested configuration: 'configure CC=qk-gcc CXX=qk-pgCC
F77=qk-pgf77 FC=qk-pgf90 --disable-mpi-profile --with-platform=redstorm
--host=x86_64-cray-linux-gnu --build=x86_64-unknown-linux-gnu
--disable-mpi-f90 --disable-mem-debug --disable-mem-profile
--disable-debug build_alias=x86_64-unknown-linux-gnu
host_alias=x86_64-cray-linux-gnu'
III. Building with Open MPI:
* No executables will be installed in $PREFIX/bin, so must compile
without mpicc, e.g. 'qk-gcc -I$PREFIX/include *.c -L$PREFIX/lib -lmpi
-lopen-rte -lopen-pal'
* When linking with libopen-pal, the following warning is normal: 'In
function `checkpoint_response': warning: mkfifo is not implemented and
will always fail'
IV. Running on Redstorm:
* scp your executable from reddish to redstorm
* Use 'qsub' to submit job and 'yod' to launch job (if you do an
interactive session, you can bypass PBS)
* qsub expects project/task information - you can either provide this
with -A option or set it in $XT_ACCOUNT environmental variable
* Sample job script for qsub with 8 nodes/processes and 10 minute
runtime:
#!/bin/sh
#PBS -lsize=8
#PBS -lwalltime=10:00
cd $PBS_O_WORKDIR
yod a.out
* Use 'xtshowmesh' and 'qstat' to query job status and cluster
configuration
