Hello fellow FiPyers,
There are several bits and pieces mentioning FiPy and Trilinos here, in the user's manual, and around the Internet, but I haven't found a detailed guide to install FiPy with Trilinos from source. I recently gave it a try on Debian -- and it wasn't too bad. The whole process, which involved a couple false starts, took me a day and a half. It is my hope in publishing this bare-bones guide that fellow travelers can get it running on Linux in just a few hours. If you'd like to follow this guide, the standard disclaimer applies: you are downloading source code from the Internet and executing it on your machine. Proceed with the utmost caution. My machine runs Debian GNU/Linux 7.8 "wheezy" on a 4-core Intel processor. I am not a sudoer, so a virtual environment was used: conda, from the Anaconda Python distribution. The various source codes were extracted and compiled within the ~/Downloads directory, then installed into the conda environment. Many of the Python programs are available directly through Anaconda, and should probably be installed through that package manager rather than from source. I chose to compile everything to (a) see if I could and (b) ensure compiler and library consistency. The attached build notes indicate the order in which packages were installed, provide the home- or download-page URL for the software project, and specify the commands I used to compile and install the code. Installation of Anaconda is excluded, since I did not have to do it; commands to download, extract, and cd into the source code are excluded since the latest versions change almost daily. After installation, I executed Daniel Wheeler's test script (https://gist.github.com/wd15/8717979) to confirm everything worked. This resulted in a couple of interesting things. First, FiPy as-is compares versions as strings and believes gmsh 2.10.0 is less than 2.5.0. I revised my local copy to compare versions as '.'-delimited tuples, which allowed the code to execute in parallel; I'm working on a patch. Second, the runtimes increased through np=4, then dropped at np=5 and beyond to values similar to Dr. Wheeler's (http://wd15.github.io/2014/01/30/fipy-trilinos-anaconda/). The machine has only four cores and is not hyperthreaded, so this was a surprise. It could be that timing differences for heavier workloads will follow expected scaling laws, but I am baffled by the result. Results table (run type, MPI rank, grid size, time; modified script at https://gist.github.com/anonymous/43afb6c567ebfd6d3b2f): $ echo -n "serial"; python mpitest.py; echo -n "trilin"; python mpitest.py --trilinos; for n in {1..6}; do echo -n "np = ${n}"; mpirun -np $n python mpitest.py --trilinos; done serial 0 27900 0.134404 trilin 0 27900 0.275607 np = 1 0 27900 0.664498 np = 2 0 14973 2.783230 1 14973 2.797370 np = 3 0 10230 3.167559 1 10480 3.174004 2 10538 3.196746 np = 4 0 7958 5.139595 1 7971 5.139695 2 8106 5.139710 3 7955 5.139925 np = 5 0 6542 0.499731 1 6538 0.500179 2 6534 0.500313 3 6655 0.498517 4 6689 0.500652 np = 6 0 5463 0.447847 1 5584 0.448344 2 5594 0.445663 3 5512 0.449028 4 5570 0.446170 5 5571 0.449486 Thoughts on this anomalous behavior would be greatly appreciated! Good luck, Trevor
Installing FiPy in a clean Anaconda environment to support Trilinos Questions/comments to [email protected] (Trevor Keller) $ conda create -n fipyompi python=2.7 $ source activate fipyompi $ export ANACONDA=/abs/path/to/conda/envs/fipyompi # append this to your .bashrc Notes: make: -j4 issued due to 4-way parallel CPU cmake: assumes mkdir build; cd build in every instance configure: issue ./configure --help for options. Make sure DEBUG is disabled. Build order: cmake http://www.cmake.org/download/ ./bootstrap --prefix=$ANACONDA --parallel=4; gmake -j4 && gmake install export CMAKE_ROOT=$ANACONDA/share/cmake-3.3 Flex http://sourceforge.net/projects/flex/files/ ./configure --prefix=$ANACONDA; make -j4 && make install Bison http://www.gnu.org/software/bison/ ./configure --prefix=$ANACONDA; make -j4 && make install Pth http://www.gnu.org/software/pth/ ./configure --prefix=$ANACONDA; make && make install # make -j4 failed doxygen http://www.stack.nl/~dimitri/doxygen/download.html#srcbin cmake -DCMAKE_INSTALL_PREFIX=$ANACONDA -DCMAKE_BUILD_TYPE=Release .. make -j4 && make install openmpi http://www.open-mpi.org/software/ompi/v1.8/ ./configure --prefix=$ANACONDA; make -j4 && make install bzip2 http://www.bzip.org/downloads.html make -f Makefile-libbz2_so # build shared library make PREFIX=$ANACONDA install libboost http://www.boost.org/users/download/ ./bootstrap.sh --prefix=$ANACONDA # Add "using mpi ;" to project-config.jam (I put it under "using gcc ;") ZLIB_LIBPATH=$ANACONDA/lib BZIP2_INCLUDE=$ANACONDA/include ./b2 --prefix=$ANACONDA --with-mpi= variant=release link=shared install glm http://glm.g-truc.net/0.9.4/ cmake -DCMAKE_INSTALL_PREFIX=$ANACONDA -DCMAKE_BUILD_TYPE=Release .. make -j4 && make install cython http://cython.org/#download python setup.py install HDF5 https://www.hdfgroup.org/HDF5/release/cmakebuild.html # Edit HDF518config.cmake to enable shared libraries (search BUILD_SHARED_LIBS:BOOL=OFF) ./build-hdf518-unix.sh ./HDF5-1.8.15-patch1-Linux.sh # accept default path cd HDF5-1.8.15-patch1-Linux; for d in */; do cp -r $d $ANACONDA/; done OpenBLAS https://github.com/xianyi/OpenBLAS/wiki/Installation-Guide make -j4 PREFIX=$ANACONDA FC=gfortran && make PREFIX=$ANACONDA install numpy http://sourceforge.net/projects/numpy/files/ python setup.py install scipy http://sourceforge.net/projects/scipy/files/ python setup.py install pysparse http://pysparse.sourceforge.net/ python setup.py install pyamg https://github.com/pyamg/pyamg python setup.py install mpi4py https://bitbucket.org/mpi4py python setup.py install netcdf https://www.unidata.ucar.edu/downloads/netcdf/index.jsp # Use the stable C distribution. HDF5 support requires you build in parallel. CC=mpicc ./configure --prefix=$ANACONDA --with-zlib=$ANACONDA; make check install && make install cp build/include/netcdf_par.h $ANACONDA/include/ # for some reason, make install neglects this file. Trilinos needs it. swig http://www.swig.org/download.html ./configure --prefix=$ANACONDA; make -j4 && make install fltk http://www.fltk.org/software.php cmake -DCMAKE_INSTALL_PREFIX=$ANACONDA -DCMAKE_BUILD_TYPE=Release ..; make -j4 && make install mmg3d http://www.math.u-bordeaux1.fr/~dobrzyns/logiciels/download.php cmake -DCMAKE_INSTALL_PREFIX=$ANACONDA -DCMAKE_BUILD_TYPE=Release ..; make -j4 && make install tetgen http://wias-berlin.de/software/tetgen/#Download cmake -DCMAKE_INSTALL_PREFIX=$ANACONDA -DCMAKE_BUILD_TYPE=Release ..; make -j4 && make install gmsh http://geuz.org/gmsh/ for d in */; do cp -r $d $ANACONDA/; done # binary distribution PETSc http://www.mcs.anl.gov/petsc/download/ ./configure --prefix=$ANACONDA --with-mpi=1 --with-debugging=0 COPTFLAGS='-O3 -march=native -mtune=native' CXXOPTFLAGS='-O3 -march=native -mtune=native' FOPTFLAGS='-O3 -march=native -mtune=native' make PETSC_DIR=. PETSC_ARCH=arch-linux2-c-opt all make PETSC_DIR=. PETSC_ARCH=arch-linux2-c-opt install petsc4py https://bitbucket.org/petsc/petsc4py PETSC_DIR=$ANACONDA PETSC_ARCH=arch-linux2-c-opt python setup.py install trilinos https://trilinos.org/download/ CMAKE_PREFIX_PATH=$ANACONDA cmake -DCMAKE_BUILD_TYPE:STRING=RELEASE -DTrilinos_ENABLE_PyTrilinos:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=ON -DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=ON -DTPL_ENABLE_MPI:BOOL=ON -DMPI_BASE_DIR:PATH=$ANACONDA -DTrilinos_ENABLE_TESTS:BOOL=ON -DDART_TESTING_TIMEOUT:STRING=600 -DCMAKE_INSTALL_PREFIX:PATH=$ANACONDA -DPyTrilinos_INSTALL_PREFIX:PATH=$ANACONDA .. make -j4 && make install LSMLIB http://ktchu.serendipityresearch.org/software/lsmlib/index.html ./configure --prefix=$ANACONDA; make -j4 && make install scikit-fmm https://github.com/scikit-fmm/scikit-fmm python setup.py install fipy https://github.com/usnistgov/fipyompi python setup.py install
_______________________________________________ fipy mailing list [email protected] http://www.ctcms.nist.gov/fipy [ NIST internal ONLY: https://email.nist.gov/mailman/listinfo/fipy ]
