Hi barry, could you please point me to a package that execute that kind of small test to see how is it done. Thanks
----- Original Message ----- From: "Barry Smith" <[email protected]> To: "Jose David Bermeol" <[email protected]> Cc: "petsc-dev" <[email protected]> Sent: Sunday, October 20, 2013 3:08:49 PM Subject: Re: [petsc-dev] Compiling Petsc with Intel mpi safe thread library You will need to have mkl-cpardiso.py create and run a simple test program to make sure that the MPI supports funneled, see for example http://www.mcs.anl.gov/research/projects/mpi/mpi-standard/mpi-report-2.0/node165.htm Based on my reading this can only be determined by actually running a MPI program and checking the provided flag to see if the MPI is providing the funneled. unfortunately in the batch world this is a problem ….. Barry On Oct 19, 2013, at 11:13 PM, Jose David Bermeol <[email protected]> wrote: > many thakns, I didn't know that using -mt_mpi half way wouldn't work. > > Yes cpardiso need this flag if not, it compiles, but when I run it sends the > following error: > MPI_THREAD_FUNNELED level is not supported! > Exit... > > So there is a way to check in mkl-cpardiso.py that I'm using the flag -mt_mpi > > Thanks > ----- Original Message ----- > From: "Satish Balay" <[email protected]> > To: "Jose David Bermeol" <[email protected]> > Cc: [email protected] > Sent: Sunday, October 20, 2013 12:04:19 AM > Subject: Re: [petsc-dev] Compiling Petsc with Intel mpi safe thread library > > sounds like you are inserting -mt_mpi half way through the configure > process via mkl-cpardiso.py. > > This won't work. > > is mkl-cpardiso tied to [thread safe variant of] intel mpi? > i.e it won't work with regular intel-mpi or other MPI like mpich? > > To use thread safe mpi - you specify thread safe mpi compilers to > petsc configure. > > i.e something like: > > --with-cc='mpicc -mt_mpi' or --with-cc='mpicc' CFALGS='-mt_mpi'. > > Satish > > On Sat, 19 Oct 2013, Jose David Bermeol wrote: > >> Hi I'm working again with the solver mkl-cpardiso. I did my implementation, >> the configuration works well, the problem is during compilation. The linking >> flags petsc is using are the followings: >> >> -Wl,-rpath,/home/jbermeol/software/test/arch-linux2-c-opt/lib >> -L/home/jbermeol/software/test/arch-linux2-c-opt/lib -lpetsc >> -Wl,--start-group -L/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 >> -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -lpthread -lm >> -liomp5 -mt_mpi -Wl,--start-group >> -L/home/jbermeol/testPetscSolvers/intel_mkl_cpardiso/lib/intel64 >> -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group >> -lcpardiso_lp64 -lcpardiso_mpi_lp64 -lpthread -lm -liomp5 -lifcore >> -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 >> -L/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 -lmkl_intel_lp64 >> -lmkl_sequential -lmkl_core -lpthread -lm -lX11 -lpthread >> -Wl,-rpath,/apps/rhel6/intel/impi/4.1.0.030/intel64/lib >> -L/apps/rhel6/intel/impi/4.1.0.030/intel64/lib >> -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/compiler/lib/intel64 >> -L/apps/rhel6/intel/composer_xe_2013.3.163/compiler/lib/intel64 >> -Wl,-rpath,/apps/rhel6/intel/compos > > er_xe_2013.3.163/ipp/lib/intel64 > -L/apps/rhel6/intel/composer_xe_2013.3.163/ipp/lib/intel64 > -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/tbb/lib/intel64 > -L/apps/rhel6/intel/composer_xe_2013.3.163/tbb/lib/intel64 > -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.4.7 > -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 > -Wl,-rpath,/home/jbermeol/software/test/-Xlinker > -Wl,-rpath,/opt/intel/mpi-rt/4.1 -lifport -lifcore -lm -lm -lmpigc4 -ldl > -lmpigf -lmpi -lmpigi -lrt -lpthread -limf -lsvml -lirng -lipgo -ldecimal > -lcilkrts -lstdc++ -lgcc_s -lirc -lirc_s -ldl >> >> The problem is with flag "-mt_mpi", this flag is telling to the Intel >> compiler, please link with the library libmpi_mt.so(this is a thread safe >> library for intel mpi). However petsc adds the flag "-lmpi" that links with >> libmpi.so. Because of this I'm gettig the following error: >> ld: MPIR_Thread: TLS definition in >> /apps/rhel6/intel/impi/4.1.0.030/intel64/lib/libmpi_mt.so section .tbss >> mismatches non-TLS definition in >> /apps/rhel6/intel/impi/4.1.0.030/intel64/lib/libmpi.so section .bss >> >> So the first question would be where is this "-lmpi" library flag added?? >> Second how can I compile using the thread save mpi library from intel?? >> Is there a way to set a dependency such that the solver cpardiso is not >> installed if petsc is compiled without this thread safe library?? >> I don't know yet how to check if petsc is been configure with mkl blas/lapack >> >> Thanks >>
