Hi Andrew, This has happened before to me also, and it seems to happen because the login nodes for the XC40 have a slightly different environment than the compute nodes. The configure script is a little smarter on the MOM nodes it seems. I usually do the compilation in an interactive job on the debug (salloc -p debugq —nodes=1). Running a script on the MOM nodes is fine but make sure you use "aprun -n 1 -N 1 make" for the actual make step lest you upset your sysadmin. :o)
In the GNU environment, if you leave LAPACK_LIBS, SCALAPACK_LIBS, BLAS_LIBS, all that stuff (FFTs as well) blank and make sure they’re external, you will be fine. The Cray LibSci library is part of the “ftn” default linking environment and has all the correct symbols. I agree, the intel environment is a bit harder if you want MKL. You could just use the intel environment with the Cray libsci if you want the extra optimisations from ifort. Both MKL and Cray use an FFTW implementation anyway for Quantum Espresso anyway don’t they? No real benefit to MKL unless you’re after 10%-level optimisations which seems like a waste of time unless you’re actually one of the HPC technicians who run the machine. Best regards, Kane > On 11 Nov 2015, at 21:48, Andrew Downs <[email protected]> wrote: > > Kane, > > Thank you, your suggestions worked, although I think I had to go a step > further and get into make.sys and comment out the line where it found the > intel BLAS libraries and point it to the internal BLAS. > Something about the intel MKL trips up compilation for us, my other > researcher isn't able to get it to work either and he's got a good 20 years > of experience on me. > > -Andrew > > Sent: Monday, November 09, 2015 at 10:23 PM > From: "Kane O'Donnell" <[email protected]> > To: "PWSCF Forum" <[email protected]> > Subject: Re: [Pw_forum] compile attempt with external BLAS and intel MKL > (espresso-5.2.1) > > Hi Andrew, > > Are you trying to just build espresso 5.2.1, or are you trying to build it > using some custom linear algebra libraries? On a Cray you’re encouraged > strongly *not* to stuff around with custom BLAS/FFT/scalapack/etc. The > programming environments are set up so you don’t need to do that, and for > example on our XC40 you can build SVN espresso just with: > > module swap PrgEnv-cray PrgEnv-gnu > ./configure —enable-parallel —with-scalapack=yes FC=ftn CC=cc > make all > > You can do the same for PrgEnv-intel (it’s exactly the same configure line) > but it’s still wise to use the MKL link line advisor to change make.sys as > necessary - you can copy and paste directly from the link line advisor as > Cray set $MKLROOT properly when you load PrgEnv-intel. > > Our cluster guys made a wiki entry about QE on the XC40 using the intel suite > here: > > https://portal.ivec.org/docs/Supercomputers/Magnus/Quantum_Espresso > <https://portal.ivec.org/docs/Supercomputers/Magnus/Quantum_Espresso> > > It’s not quite the same as the link line advisor but it works (except > XSpectra, you need the gnu environment to get that to work on our XC40 for > some reason). > > Kane > > > On 10 Nov 2015, at 01:10, Andrew Downs <[email protected] > <x-msg://34/[email protected]>> wrote: > > Hello, > > I've been able to jump a few hurdles that have come up in my compile process, > but this one has me stumped. > I'm compiling on a Cray XC-40, and another researcher here was able to build > 5.1.1 with out BLAS and MKL libraries. > He and I assumed 5.2.1 should also work under the same circumstances, but > this may turn out not to be the case. > > my configure line is as follows: > > ./configure ARCH=x86_64 MPIF90=ftn FFLAGS="-O3" CC=icc CFLAGS=-O3 > --prefix=/usr/cta/unsupported/qe/espresso-5.2.1 > > relevant part of make.sys: > > LDFLAGS=--static > -I/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/include/ > -I/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/include/intel64/lp64/ > BLAS_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_sequential.a > -Wl,--end-group > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a > -lpthread -lm > LAPACK_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_intel_lp64.a > > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_core.a > SCALAPACK_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a > -Wl,--start-group > FFT_LIBS=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_cdft_core.a > > > make all output: > ftn --static > -I/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/include/ > -I/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/include/intel64/lp64/ > -o pw.x \ > pwscf.o libpw.a ../../Modules/libqemod.a ../../flib/ptools.a > ../../flib/flib.a ../../clib/clib.a ../../iotk/src/libiotk.a > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a > -Wl,--start-group > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_intel_lp64.a > > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_core.a > > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_cdft_core.a > > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_sequential.a > -Wl,--end-group > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a > -lpthread -lm > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_core.a(mkl_memory.o): > In function `mkl_serv_set_memory_limit': > ../../../../serv/kernel/mkl_memory.c:(.text+0x521): warning: Using 'dlopen' > in statically linked applications requires at runtime the shared libraries > from the glibc version used for linking > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a(dgesd2d_.o): > In function `dgesd2d_': > ../../../../scalapack/BLACS/SRC/MPI/dgesd2d_.c:(.text+0x92): undefined > reference to `mkl_serv_get_mpi_wrappers' > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a(dgesd2d_.o): > In function `dsendrecv_': > ../../../../scalapack/BLACS/SRC/MPI/dgesd2d_.c:(.text+0x142): undefined > reference to `mkl_serv_get_mpi_wrappers' > ../../../../scalapack/BLACS/SRC/MPI/dgesd2d_.c:(.text+0x162): undefined > reference to `mkl_serv_get_mpi_wrappers' > ../../../../scalapack/BLACS/SRC/MPI/dgesd2d_.c:(.text+0x1aa): undefined > reference to `mkl_serv_get_mpi_wrappers' > ../../../../scalapack/BLACS/SRC/MPI/dgesd2d_.c:(.text+0x1da): undefined > reference to `mkl_serv_get_mpi_wrappers' > /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a(dgesd2d_.o):../../../../scalapack/BLACS/SRC/MPI/dgesd2d_.c:(.text+0x1ea): > more undefined references to `mkl_serv_get_mpi_wrappers' follow > make[2]: *** [pw.x] Error 1 > make[2]: Leaving directory `/p/home/adowns/espresso-5.2.1/PW/src' > make[1]: *** [pw] Error 1 > make[1]: Leaving directory `/p/home/adowns/espresso-5.2.1/PW' > make: *** [pw] Error 1 > > _______________________________________________ > Pw_forum mailing list > [email protected] <x-msg://34/[email protected]> > http://pwscf.org/mailman/listinfo/pw_forum > <http://pwscf.org/mailman/listinfo/pw_forum>_______________________________________________ > Pw_forum mailing list [email protected] > http://pwscf.org/mailman/listinfo/pw_forum > <http://pwscf.org/mailman/listinfo/pw_forum>_______________________________________________ > Pw_forum mailing list > [email protected] > http://pwscf.org/mailman/listinfo/pw_forum
_______________________________________________ Pw_forum mailing list [email protected] http://pwscf.org/mailman/listinfo/pw_forum
