On Monday, November 09, 2015 06:10:51 PM Andrew Downs wrote: > > LDFLAGS=--static > -I/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/include/ > -I/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/include/intel64/l > p64/ > BLAS_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel > 64/libmkl_sequential.a -Wl,--end-group > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_ > blacs_intelmpi_lp64.a -lpthread -lm > LAPACK_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/int > el64/libmkl_intel_lp64.a > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_ > core.a > SCALAPACK_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/ > intel64/libmkl_scalapack_lp64.a -Wl,--start-group > FFT_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel6 > 4/libmkl_cdft_core.a >
Hello, this looks funny to me, because you have -Wl,--start-group in one variable and -Wl,--end-group in another, and it does no look like the two got concatenated in a meaningful way during compilation. If you got the line from the intel mkl link lien advisor helper, then you do not have to cut the line and put it here and there, it is better to put it in BLAS_LIBS and leave LAPACK_LIBS and SCALAPACK_LIBS empty. Also, your FFTW line does not seem to be correct, if you want to use MKL FFT you have to use the proper interface. If your system system admin has already compiled it, you can probably grab it from the /whatever/mkl/lib/intel64, it is called libfftw3xf_intel.a. If it is not already compiled, you can copy the directory /whatever/mkl/interfaces/fftw3xf/ to your home, move to it and compile the library with make intel64 Furthermore, there is no point in putting all the -I/stuff/ in LDFLAGS: the include directories go to IFLAGS (they are passed to the compiler, no to the linker). Finally, you are using the parallel MKL with IntelMPI, i.e. the version of MKL that interfaces with the the Intel implementation of MPI libraries. Are you sure your MPI wrapper uses the same implementation? (Answer: no, it would be called mpiifort). From some googling around, it looks like Cray defaults to MPICH2. So, what you have to do is ditch the wrapper and link MPI manually. In order to do that you go to https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor and enter all your informations: MKL 11.3 Linux None Intel Fortran Intel 64 Static LP64 Sequential [x] Scalapack MPICH2 And you copy the line: ${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group ${MKLROOT}/ lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a $ {MKLROOT}/lib/intel64/libmkl_sequential.a -Wl,--end-group ${MKLROOT}/lib/ intel64/libmkl_blacs_intelmpi_lp64.a -lpthread -lm Take care to READ THE NOTES! i.e. be sure that MKLROOT variable is define, and eventually run source /whatever/mkl/bin/mklvars.sh intel64 Then after configuring you clean up BLAS_LIBS and LAPACK_LIBS and put the Link Advisor line in SCALAPACK_LIBS. The problem is that even after fixing everything up to here it may still not work, because the ftn wrapper links MPI after MKL, while MKL scalapack requires MPI. ANYWAY, WHAT'S ABOVE BEFORE GOING FURTHER, as it usually works. If it does not work, because you get undefined references to mpi stupp: type "ftn -show" to get the MPI linking line Than run configure as normal, just to get a canvas make.sys, but then open with a text editor and: 1. Put all the -I from the Link Advisor and from "ftn -show" in the IFLAGS 2. clean up BLAS_LIBS and LAPACK_LIBS 3. put the line from Link Advisor in SCALAPACK_LIBS 4. add all the libraries from "ftn -show" at the end of LIBS. 4b. to be extra safe you can add these libraries between --start-group and -- end-group in SCALAPACK_LIBS, but this is usually not necessary. 5. change MPIF90 from ftn to ifort 6. put the fftw3 library you have compiled in FFT_LIBS (ditch anything that could be already there), eventually change -D__FFTW with -D__FFTW3 in DFLAGS. In my case this becomes /home/whatever/fftw3xf/libfftw3xf_intel.a 7. "make pw.x" and hope you did not miss anything HTH -- Dr. Lorenzo Paulatto IdR @ IMPMC -- CNRS & Université Paris 6 +33 (0)1 44 275 084 / skype: paulatz http://www.impmc.upmc.fr/~paulatto/ 23-24/4é16 Boîte courrier 115, 4 place Jussieu 75252 Paris Cédex 05 _______________________________________________ Pw_forum mailing list [email protected] http://pwscf.org/mailman/listinfo/pw_forum
