RE: [SIESTA-L] compiling Siesta in parallel with intelmpi
hi, did you do a make clean ..you may have leftovers *.mod *.o files from a previous compilation... s.auluck Thanks, I was able to compile without openmp now. The siesta-ldau bug I found is still there though (see next mail/thread). Giovanni De : siesta-l-requ...@uam.es [siesta-l-requ...@uam.es] de la part de Herbert Fruchtl [herbert.fruc...@st-andrews.ac.uk] Envoyé : mardi 13 mai 2014 14:55 À : siesta-l@uam.es Objet : Re: [SIESTA-L] compiling Siesta in parallel with intelmpi You need the sequential mkl. Here's my BLAS line: BLAS_LIBS= -L/usr/local/intel/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core Herbert On 13/05/14 13:42, Borghi Giovanni wrote: Ok below an arch.make which works, and lets siesta run in parallel, apparently -openmp is required, otherwise I get errors of the type (full output included below the arch.make) /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_fini_8' kmpc appears to be library coming with openmp. *** SIESTA_ARCH=x86_64-unknown-linux-gnu--Intel FPP= FPP_OUTPUT= FC=mpiifort RANLIB=ranlib SYS=nag SP_KIND=4 DP_KIND=8 KINDS=$(SP_KIND) $(DP_KIND) FFLAGS=-g -check all -traceback -openmp FPPFLAGS= -DMPI -DFC_HAVE_FLUSH -DFC_HAVE_ABORT LDFLAGS=-static-intel -openmp ARFLAGS_EXTRA= FCFLAGS_fixed_f= FCFLAGS_free_f90= FPPFLAGS_fixed_F= FPPFLAGS_free_F90= BLAS_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core LAPACK_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core BLACS_LIBS=-lmkl_blacs_lp64 -lmkl_blacs_intelmpi_lp64 SCALAPACK_LIBS= -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 COMP_LIBS=dc_lapack.a NETCDF_LIBS= NETCDF_INTERFACE= LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS) #SIESTA needs an F90 interface to MPI #This will give you SIESTA's own implementation #If your compiler vendor offers an alternative, you may change #to it here. #MPI_INTERFACE=/opt/intel/impi/4.0.3.008/intel64/lib/libmpiif.a MPI_INTERFACE=libmpi_f90.a MPI_INCLUDE=. * siesta.o: In function `siesta': /home/borghi/marzari/siesta/siesta-ldau/Src/siesta.F:1: undefined reference to `__kmpc_begin' /home/borghi/marzari/siesta/siesta-ldau/Src/siesta.F:37: undefined reference to `__kmpc_end' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_ok_to_fork' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_end_single' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_ordered' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_for_static_init_8' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_thread_num' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_barrier' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_num_threads' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_num_procs' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_next_4' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_end_reduce_nowait' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_critical' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_fini_8' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_atomic_cmplx8_add' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_atomic_float4_add' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_serialized_parallel' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_end_critical' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_init_8' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `ompc_set_nested' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_nested' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so
RE: [SIESTA-L] compiling Siesta in parallel with intelmpi
Actually, at the moment the compilation problem is solved, the issue is now a segmentation fault caused by the deallocation of arrays used in ldau.F. Apparently, this problem is not dependent on parallel compilation. Thank you, Giovanni De : siesta-l-requ...@uam.es [siesta-l-requ...@uam.es] de la part de saul...@iitk.ac.in [saul...@iitk.ac.in] Envoyé : mercredi 14 mai 2014 10:00 À : siesta-l@uam.es Objet : RE: [SIESTA-L] compiling Siesta in parallel with intelmpi hi, did you do a make clean ..you may have leftovers *.mod *.o files from a previous compilation... s.auluck Thanks, I was able to compile without openmp now. The siesta-ldau bug I found is still there though (see next mail/thread). Giovanni De : siesta-l-requ...@uam.es [siesta-l-requ...@uam.es] de la part de Herbert Fruchtl [herbert.fruc...@st-andrews.ac.uk] Envoyé : mardi 13 mai 2014 14:55 À : siesta-l@uam.es Objet : Re: [SIESTA-L] compiling Siesta in parallel with intelmpi You need the sequential mkl. Here's my BLAS line: BLAS_LIBS= -L/usr/local/intel/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core Herbert On 13/05/14 13:42, Borghi Giovanni wrote: Ok below an arch.make which works, and lets siesta run in parallel, apparently -openmp is required, otherwise I get errors of the type (full output included below the arch.make) /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_fini_8' kmpc appears to be library coming with openmp. *** SIESTA_ARCH=x86_64-unknown-linux-gnu--Intel FPP= FPP_OUTPUT= FC=mpiifort RANLIB=ranlib SYS=nag SP_KIND=4 DP_KIND=8 KINDS=$(SP_KIND) $(DP_KIND) FFLAGS=-g -check all -traceback -openmp FPPFLAGS= -DMPI -DFC_HAVE_FLUSH -DFC_HAVE_ABORT LDFLAGS=-static-intel -openmp ARFLAGS_EXTRA= FCFLAGS_fixed_f= FCFLAGS_free_f90= FPPFLAGS_fixed_F= FPPFLAGS_free_F90= BLAS_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core LAPACK_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core BLACS_LIBS=-lmkl_blacs_lp64 -lmkl_blacs_intelmpi_lp64 SCALAPACK_LIBS= -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 COMP_LIBS=dc_lapack.a NETCDF_LIBS= NETCDF_INTERFACE= LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS) #SIESTA needs an F90 interface to MPI #This will give you SIESTA's own implementation #If your compiler vendor offers an alternative, you may change #to it here. #MPI_INTERFACE=/opt/intel/impi/4.0.3.008/intel64/lib/libmpiif.a MPI_INTERFACE=libmpi_f90.a MPI_INCLUDE=. * siesta.o: In function `siesta': /home/borghi/marzari/siesta/siesta-ldau/Src/siesta.F:1: undefined reference to `__kmpc_begin' /home/borghi/marzari/siesta/siesta-ldau/Src/siesta.F:37: undefined reference to `__kmpc_end' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_ok_to_fork' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_end_single' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_ordered' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_for_static_init_8' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_thread_num' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_barrier' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_num_threads' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_num_procs' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_next_4' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_end_reduce_nowait' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_critical' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_fini_8' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_atomic_cmplx8_add' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_atomic_float4_add' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_serialized_parallel' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64
Re: [SIESTA-L] compiling Siesta in parallel with intelmpi
Add -DMPI to FPPFLAGS and try again... 2014-05-13 7:48 GMT+00:00 Borghi Giovanni giovanni.bor...@epfl.ch: Dear all, I am trying to compile Siesta in parallel using intel mpi and mpiifort. However, when running with mpirun -np 6 siesta_rootdir/Obj/siesta The code starts in serial mode. Can anyone suggest me what is wrong? Does anyone have an example arch.make I may draw inspiration from? I paste the main part of the arch.make below... Thank you very much! Giovanni *** SIESTA_ARCH=x86_64-unknown-linux-gnu--unknown FPP= FPP_OUTPUT= FC=mpiifort RANLIB=ranlib SYS=nag SP_KIND=4 DP_KIND=8 KINDS=$(SP_KIND) $(DP_KIND) FFLAGS= -O2 -openmp FPPFLAGS= -DFC_HAVE_FLUSH -DFC_HAVE_ABORT LDFLAGS= -static-intel -openmp ARFLAGS_EXTRA= FCFLAGS_fixed_f= FCFLAGS_free_f90= FPPFLAGS_fixed_F= FPPFLAGS_free_F90= BLAS_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core LAPACK_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core BLACS_LIBS=-lmkl_blacs_lp64 -lmkl_blacs_intelmpi_lp64 SCALAPACK_LIBS= -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 COMP_LIBS=dc_lapack.a NETCDF_LIBS= NETCDF_INTERFACE= LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS) #SIESTA needs an F90 interface to MPI #This will give you SIESTA's own implementation #If your compiler vendor offers an alternative, you may change #to it here. #MPI_INTERFACE=/opt/intel/impi/4.0.3.008/intel64/lib/libmpiif.a MPI_INTERFACE=/usr/lib/openmpi/lib/libmpi_f90.a MPI_INCLUDE=/opt/intel/impi/4.0.3.008/intel64/include/ -- Kind regards Nick
Re: [SIESTA-L] compiling Siesta in parallel with intelmpi
And don't use openmp, as I recall it doesn't work, yet :) 2014-05-13 8:57 GMT+00:00 Nick Papior Andersen nickpap...@gmail.com: Add -DMPI to FPPFLAGS and try again... 2014-05-13 7:48 GMT+00:00 Borghi Giovanni giovanni.bor...@epfl.ch: Dear all, I am trying to compile Siesta in parallel using intel mpi and mpiifort. However, when running with mpirun -np 6 siesta_rootdir/Obj/siesta The code starts in serial mode. Can anyone suggest me what is wrong? Does anyone have an example arch.make I may draw inspiration from? I paste the main part of the arch.make below... Thank you very much! Giovanni *** SIESTA_ARCH=x86_64-unknown-linux-gnu--unknown FPP= FPP_OUTPUT= FC=mpiifort RANLIB=ranlib SYS=nag SP_KIND=4 DP_KIND=8 KINDS=$(SP_KIND) $(DP_KIND) FFLAGS= -O2 -openmp FPPFLAGS= -DFC_HAVE_FLUSH -DFC_HAVE_ABORT LDFLAGS= -static-intel -openmp ARFLAGS_EXTRA= FCFLAGS_fixed_f= FCFLAGS_free_f90= FPPFLAGS_fixed_F= FPPFLAGS_free_F90= BLAS_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core LAPACK_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core BLACS_LIBS=-lmkl_blacs_lp64 -lmkl_blacs_intelmpi_lp64 SCALAPACK_LIBS= -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 COMP_LIBS=dc_lapack.a NETCDF_LIBS= NETCDF_INTERFACE= LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS) #SIESTA needs an F90 interface to MPI #This will give you SIESTA's own implementation #If your compiler vendor offers an alternative, you may change #to it here. #MPI_INTERFACE=/opt/intel/impi/4.0.3.008/intel64/lib/libmpiif.a MPI_INTERFACE=/usr/lib/openmpi/lib/libmpi_f90.a MPI_INCLUDE=/opt/intel/impi/4.0.3.008/intel64/include/ -- Kind regards Nick -- Kind regards Nick
Re: [SIESTA-L] compiling Siesta in parallel with intelmpi
/libmkl_intel_thread.so: undefined reference to `omp_in_parallel' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_push_num_threads' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_reduce_nowait' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `omp_get_max_threads' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_for_static_init_4' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_end_serialized_parallel' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_flush' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_single' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_next_8' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_atomic_float8_add' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_dispatch_init_4' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_global_thread_num' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_end_ordered' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_fork_call' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_atomic_fixed8_add' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_for_static_fini' /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `__kmpc_atomic_cmplx4_add' De : siesta-l-requ...@uam.es [siesta-l-requ...@uam.es] de la part de Nick Papior Andersen [nickpap...@gmail.com] Envoyé : mardi 13 mai 2014 10:57 À : siesta-l@uam.es Objet : Re: [SIESTA-L] compiling Siesta in parallel with intelmpi Add -DMPI to FPPFLAGS and try again... 2014-05-13 7:48 GMT+00:00 Borghi Giovanni giovanni.bor...@epfl.ch mailto:giovanni.bor...@epfl.ch: Dear all, I am trying to compile Siesta in parallel using intel mpi and mpiifort. However, when running with mpirun -np 6 siesta_rootdir/Obj/siesta The code starts in serial mode. Can anyone suggest me what is wrong? Does anyone have an example arch.make I may draw inspiration from? I paste the main part of the arch.make below... Thank you very much! Giovanni *** SIESTA_ARCH=x86_64-unknown-linux-gnu--unknown FPP= FPP_OUTPUT= FC=mpiifort RANLIB=ranlib SYS=nag SP_KIND=4 DP_KIND=8 KINDS=$(SP_KIND) $(DP_KIND) FFLAGS= -O2 -openmp FPPFLAGS= -DFC_HAVE_FLUSH -DFC_HAVE_ABORT LDFLAGS= -static-intel -openmp ARFLAGS_EXTRA= FCFLAGS_fixed_f= FCFLAGS_free_f90= FPPFLAGS_fixed_F= FPPFLAGS_free_F90= BLAS_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core LAPACK_LIBS=-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core BLACS_LIBS=-lmkl_blacs_lp64 -lmkl_blacs_intelmpi_lp64 SCALAPACK_LIBS= -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 COMP_LIBS=dc_lapack.a NETCDF_LIBS= NETCDF_INTERFACE= LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS) #SIESTA needs an F90 interface to MPI #This will give you SIESTA's own implementation #If your compiler vendor offers an alternative, you may change #to it here. #MPI_INTERFACE=/opt/intel/impi/4.0.3.008/intel64/lib/libmpiif.a http://4.0.3.008/intel64/lib/libmpiif.a MPI_INTERFACE=/usr/lib/openmpi/lib/libmpi_f90.a MPI_INCLUDE=/opt/intel/impi/4.0.3.008/intel64/include/http: //4.0.3.008/intel64/include/ -- Kind regards Nick -- Herbert Fruchtl Senior Scientific Computing Officer School of Chemistry, School of Mathematics and Statistics University of St Andrews -- The University of St Andrews is a charity registered in Scotland: No SC013532 -- Kind regards Nick
