Re: [Wien] MKL error: Cannot load symbol MKLMPI_Get_wrappers
Thank you all for your hints. The links Professor Mark gave in his last Mail solved the problem (but gave me a new one to chew on). It was a missing libmkl_blacs_lp64.so that caused the problem. Linking with lmkl_intelmpi_lp64 solved it. Obviously there are some differences in the mkl between the version we used at TU Wien and the version I’m using now. Now lapw1 is working and I’m fighting a „** testerror“ in lapw2, but I’ve got a several hours train-ride ahead to do so. ;-) Thanks for your help! Cheers, Walid -- Dr.techn. Walid Hetaba Fritz-Haber-Institut der Max-Planck-Gesellschaft Department of Inorganic Chemistry Faradayweg 4-6, 14195 Berlin, Germany T: +49 30 8413-4412 het...@fhi-berlin.mpg.de http://www.fhi-berlin.mpg.de > Am 28.04.2016 um 07:59 schrieb Peter Blaha : > > Don't know if this is the problem here, but very often the problem is the > blacks-library. There are several versions of blacks in the mkl and they must > be chosen according to your mpi. Of course there is a version for intel-mpi, > but it must be the correct library for each mpi. > > > Am 27.04.2016 um 16:42 schrieb Walid Hetaba: >> Dear colleagues, >> >> I'm trying to get the parallel version of WIEN2k 14.2 running on the >> institute's cluster, but I'm stuck now. >> I searched the mailing list for the error message I receive, but at >> first glance I did not find relevant postings. >> >> This is the problem I face: >> The serial version of WIEN2k is running fine (also when using qsub and >> the submission system). >> However, when trying to use the parallel version I get the following >> error when lapw1 is invoked in parallel mode: >> "Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers." >> >> Here the facts on hard- and software: >> I'm using WIEN2k 14.2 to run the example TiC case from the UG as a test >> to see whether all works out fine. >> I compiled the sofware using the Intel Cluster Studio with the Intel >> Fortran Compiler 16.0, Intel C++ Compiler 16.0, Intel MKL 11.3 and Intel >> MPI. >> I compiled it using FFTW3 and the MKL lapack, scalapack and blacs. >> >> The cluster consists of 37 nodes with 20 CPUs each. It is running a >> CentOS Linux distribution and it is using SGE as submission system. In >> the submission script orte is used as the parallel environment. >> >> This is what I've tried so far: >> I don't know why it can't find things from the MKL as I export all my >> environmental variables using the submission script, which I also >> checked by printing them. >> As additional information I include the submission script >> (WIEN2k-test-p.sh, which is still a bit messy as I adopted it from >> another program for testing reasons), the :parallel file with the error >> messages (which I renamed to .parallel so that Thunderbird on Windows >> can handle it), the output-file (TiC.o14540) and the .machines file. >> >> I get the same error messages when using OpenMPI instead of the >> IntelMPI, but the output is a bit different (see TiC.o14541). >> >> I did try different ways to export my environmental variables (to get >> rid of unnecessary stuff), also tried mpi as the parallel environment in >> the submission script instead of orte. I also tried to request a >> different number of slots,... >> But nothing off this got me rid of this error message. >> >> Has anybody experienced a similar problem or are there any hints what >> else I can check? >> >> Thanks for your help. >> All the best, >> Walid >> >> -- >> Dr.techn. Walid Hetaba >> >> Fritz-Haber-Institut der Max-Planck-Gesellschaft >> Department of Inorganic Chemistry >> Faradayweg 4-6, 14195 Berlin, Germany >> T: +49 30 8413-4412 >> het...@fhi-berlin.mpg.de >> http://www.fhi-berlin.mpg.de >> >> >> >> ___ >> Wien mailing list >> Wien@zeus.theochem.tuwien.ac.at >> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien >> SEARCH the MAILING-LIST at: >> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html >> > > -- > -- > Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna > Phone: +43-1-58801-165300 FAX: +43-1-58801-165982 > Email: bl...@theochem.tuwien.ac.atWIEN2k: http://www.wien2k.at > WWW: http://www.imc.tuwien.ac.at/staff/tc_group_e.php > -- > ___ > Wien mailing list > Wien@zeus.theochem.tuwien.ac.at > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien > SEARCH the MAILING-LIST at: > http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html ___ Wien mailing list Wien@zeus.theochem.tuwien.ac.at http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
Re: [Wien] MKL error: Cannot load symbol MKLMPI_Get_wrappers
Don't know if this is the problem here, but very often the problem is the blacks-library. There are several versions of blacks in the mkl and they must be chosen according to your mpi. Of course there is a version for intel-mpi, but it must be the correct library for each mpi. Am 27.04.2016 um 16:42 schrieb Walid Hetaba: Dear colleagues, I'm trying to get the parallel version of WIEN2k 14.2 running on the institute's cluster, but I'm stuck now. I searched the mailing list for the error message I receive, but at first glance I did not find relevant postings. This is the problem I face: The serial version of WIEN2k is running fine (also when using qsub and the submission system). However, when trying to use the parallel version I get the following error when lapw1 is invoked in parallel mode: "Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers." Here the facts on hard- and software: I'm using WIEN2k 14.2 to run the example TiC case from the UG as a test to see whether all works out fine. I compiled the sofware using the Intel Cluster Studio with the Intel Fortran Compiler 16.0, Intel C++ Compiler 16.0, Intel MKL 11.3 and Intel MPI. I compiled it using FFTW3 and the MKL lapack, scalapack and blacs. The cluster consists of 37 nodes with 20 CPUs each. It is running a CentOS Linux distribution and it is using SGE as submission system. In the submission script orte is used as the parallel environment. This is what I've tried so far: I don't know why it can't find things from the MKL as I export all my environmental variables using the submission script, which I also checked by printing them. As additional information I include the submission script (WIEN2k-test-p.sh, which is still a bit messy as I adopted it from another program for testing reasons), the :parallel file with the error messages (which I renamed to .parallel so that Thunderbird on Windows can handle it), the output-file (TiC.o14540) and the .machines file. I get the same error messages when using OpenMPI instead of the IntelMPI, but the output is a bit different (see TiC.o14541). I did try different ways to export my environmental variables (to get rid of unnecessary stuff), also tried mpi as the parallel environment in the submission script instead of orte. I also tried to request a different number of slots,... But nothing off this got me rid of this error message. Has anybody experienced a similar problem or are there any hints what else I can check? Thanks for your help. All the best, Walid -- Dr.techn. Walid Hetaba Fritz-Haber-Institut der Max-Planck-Gesellschaft Department of Inorganic Chemistry Faradayweg 4-6, 14195 Berlin, Germany T: +49 30 8413-4412 het...@fhi-berlin.mpg.de http://www.fhi-berlin.mpg.de ___ Wien mailing list Wien@zeus.theochem.tuwien.ac.at http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html -- -- Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna Phone: +43-1-58801-165300 FAX: +43-1-58801-165982 Email: bl...@theochem.tuwien.ac.atWIEN2k: http://www.wien2k.at WWW: http://www.imc.tuwien.ac.at/staff/tc_group_e.php -- ___ Wien mailing list Wien@zeus.theochem.tuwien.ac.at http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
Re: [Wien] MKL error: Cannot load symbol MKLMPI_Get_wrappers
See https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/590302 If that does not work, search https://www.google.com/search?q=Cannot+load+symbol+MKLMPI_Get_wrappers&oq=Cannot+load+symbol+MKLMPI_Get_wrappers&aqs=chrome..69i57j69i61.1094j0j4&sourceid=chrome&ie=UTF-8 Google searching of bugs/problems is normally very effective. On Wed, Apr 27, 2016 at 1:21 PM, Walid Hetaba wrote: > Hi, > > thanks for the hints but unfortunately it didn’t solve the problem. > > Cheers, > Walid > > > -- > Dr.techn. Walid Hetaba > > Fritz-Haber-Institut der Max-Planck-Gesellschaft > Department of Inorganic Chemistry > Faradayweg 4-6, 14195 Berlin, Germany > T: +49 30 8413-4412 > het...@fhi-berlin.mpg.de > http://www.fhi-berlin.mpg.de > > > Am 27.04.2016 um 16:52 schrieb Laurence Marks >: > > > > I am not certain if this will work -- worth a try. > > > > Instead of exporting LD_LIBRARY_PATH etc yourself, use the scripts that > Intel provides. These are safer since they keep changing them. For > instance, on my cluster I have > > > > VERS=2015 > > source /opt/intel/composer$VERS/bin/compilervars.sh intel64 > > source /opt/intel/composer$VERS/mkl/bin/mklvars.sh intel64 > > > > # This should work always > > export MKLPATH=$MKLROOT/lib/intel64 > > export MKLINC=$MKLROOT/include > > > > Make sure that you are appropriately compiling, and check what version > of VERS to use > > > > > > On Wed, Apr 27, 2016 at 9:42 AM, Walid Hetaba > wrote: > > Dear colleagues, > > > > I'm trying to get the parallel version of WIEN2k 14.2 running on the > institute's cluster, but I'm stuck now. > > I searched the mailing list for the error message I receive, but at > first glance I did not find relevant postings. > > > > This is the problem I face: > > The serial version of WIEN2k is running fine (also when using qsub and > the submission system). > > However, when trying to use the parallel version I get the following > error when lapw1 is invoked in parallel mode: > > "Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers." > > > > Here the facts on hard- and software: > > I'm using WIEN2k 14.2 to run the example TiC case from the UG as a test > to see whether all works out fine. > > I compiled the sofware using the Intel Cluster Studio with the Intel > Fortran Compiler 16.0, Intel C++ Compiler 16.0, Intel MKL 11.3 and Intel > MPI. > > I compiled it using FFTW3 and the MKL lapack, scalapack and blacs. > > > > The cluster consists of 37 nodes with 20 CPUs each. It is running a > CentOS Linux distribution and it is using SGE as submission system. In the > submission script orte is used as the parallel environment. > > > > This is what I've tried so far: > > I don't know why it can't find things from the MKL as I export all my > environmental variables using the submission script, which I also checked > by printing them. > > As additional information I include the submission script > (WIEN2k-test-p.sh, which is still a bit messy as I adopted it from another > program for testing reasons), the :parallel file with the error messages > (which I renamed to .parallel so that Thunderbird on Windows can handle > it), the output-file (TiC.o14540) and the .machines file. > > > > I get the same error messages when using OpenMPI instead of the > IntelMPI, but the output is a bit different (see TiC.o14541). > > > > I did try different ways to export my environmental variables (to get > rid of unnecessary stuff), also tried mpi as the parallel environment in > the submission script instead of orte. I also tried to request a different > number of slots,... > > But nothing off this got me rid of this error message. > > > > Has anybody experienced a similar problem or are there any hints what > else I can check? > > > > Thanks for your help. > > All the best, > > Walid > > > > -- > > Dr.techn. Walid Hetaba > > > > Fritz-Haber-Institut der Max-Planck-Gesellschaft > > Department of Inorganic Chemistry > > Faradayweg 4-6, 14195 Berlin, Germany > > T: +49 30 8413-4412 > > > > het...@fhi-berlin.mpg.de > > http://www.fhi-berlin.mpg.de > > > > > > > > -- > > Professor Laurence Marks > > "Research is to see what everybody else has seen, and to think what > nobody else has thought", Albert Szent-Gyorgi > > www.numis.northwestern.edu ; Corrosion in 4D: > MURI4D.numis.northwestern.edu > > Partner of the CFW 100% program for gender equity, > www.cfw.org/100-percent > > Co-Editor, Acta Cryst A > > ___ > > Wien mailing list > > Wien@zeus.theochem.tuwien.ac.at > > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien > > SEARCH the MAILING-LIST at: > http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html > > ___ > Wien mailing list > Wien@zeus.theochem.tuwien.ac.at > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien > SEARCH the MAILING-LIST at: > http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html > -- Professor Laure
Re: [Wien] MKL error: Cannot load symbol MKLMPI_Get_wrappers
Hi, thanks for the hints but unfortunately it didn’t solve the problem. Cheers, Walid -- Dr.techn. Walid Hetaba Fritz-Haber-Institut der Max-Planck-Gesellschaft Department of Inorganic Chemistry Faradayweg 4-6, 14195 Berlin, Germany T: +49 30 8413-4412 het...@fhi-berlin.mpg.de http://www.fhi-berlin.mpg.de > Am 27.04.2016 um 16:52 schrieb Laurence Marks : > > I am not certain if this will work -- worth a try. > > Instead of exporting LD_LIBRARY_PATH etc yourself, use the scripts that Intel > provides. These are safer since they keep changing them. For instance, on my > cluster I have > > VERS=2015 > source /opt/intel/composer$VERS/bin/compilervars.sh intel64 > source /opt/intel/composer$VERS/mkl/bin/mklvars.sh intel64 > > # This should work always > export MKLPATH=$MKLROOT/lib/intel64 > export MKLINC=$MKLROOT/include > > Make sure that you are appropriately compiling, and check what version of > VERS to use > > > On Wed, Apr 27, 2016 at 9:42 AM, Walid Hetaba > wrote: > Dear colleagues, > > I'm trying to get the parallel version of WIEN2k 14.2 running on the > institute's cluster, but I'm stuck now. > I searched the mailing list for the error message I receive, but at first > glance I did not find relevant postings. > > This is the problem I face: > The serial version of WIEN2k is running fine (also when using qsub and the > submission system). > However, when trying to use the parallel version I get the following error > when lapw1 is invoked in parallel mode: > "Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers." > > Here the facts on hard- and software: > I'm using WIEN2k 14.2 to run the example TiC case from the UG as a test to > see whether all works out fine. > I compiled the sofware using the Intel Cluster Studio with the Intel Fortran > Compiler 16.0, Intel C++ Compiler 16.0, Intel MKL 11.3 and Intel MPI. > I compiled it using FFTW3 and the MKL lapack, scalapack and blacs. > > The cluster consists of 37 nodes with 20 CPUs each. It is running a CentOS > Linux distribution and it is using SGE as submission system. In the > submission script orte is used as the parallel environment. > > This is what I've tried so far: > I don't know why it can't find things from the MKL as I export all my > environmental variables using the submission script, which I also checked by > printing them. > As additional information I include the submission script (WIEN2k-test-p.sh, > which is still a bit messy as I adopted it from another program for testing > reasons), the :parallel file with the error messages (which I renamed to > .parallel so that Thunderbird on Windows can handle it), the output-file > (TiC.o14540) and the .machines file. > > I get the same error messages when using OpenMPI instead of the IntelMPI, but > the output is a bit different (see TiC.o14541). > > I did try different ways to export my environmental variables (to get rid of > unnecessary stuff), also tried mpi as the parallel environment in the > submission script instead of orte. I also tried to request a different number > of slots,... > But nothing off this got me rid of this error message. > > Has anybody experienced a similar problem or are there any hints what else I > can check? > > Thanks for your help. > All the best, > Walid > > -- > Dr.techn. Walid Hetaba > > Fritz-Haber-Institut der Max-Planck-Gesellschaft > Department of Inorganic Chemistry > Faradayweg 4-6, 14195 Berlin, Germany > T: +49 30 8413-4412 > > het...@fhi-berlin.mpg.de > http://www.fhi-berlin.mpg.de > > > > -- > Professor Laurence Marks > "Research is to see what everybody else has seen, and to think what nobody > else has thought", Albert Szent-Gyorgi > www.numis.northwestern.edu ; Corrosion in 4D: MURI4D.numis.northwestern.edu > Partner of the CFW 100% program for gender equity, www.cfw.org/100-percent > Co-Editor, Acta Cryst A > ___ > Wien mailing list > Wien@zeus.theochem.tuwien.ac.at > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien > SEARCH the MAILING-LIST at: > http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html ___ Wien mailing list Wien@zeus.theochem.tuwien.ac.at http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
Re: [Wien] MKL error: Cannot load symbol MKLMPI_Get_wrappers
I am not certain if this will work -- worth a try. Instead of exporting LD_LIBRARY_PATH etc yourself, use the scripts that Intel provides. These are safer since they keep changing them. For instance, on my cluster I have VERS=2015 source /opt/intel/composer$VERS/bin/compilervars.sh intel64 source /opt/intel/composer$VERS/mkl/bin/mklvars.sh intel64 # This should work always export MKLPATH=$MKLROOT/lib/intel64 export MKLINC=$MKLROOT/include Make sure that you are appropriately compiling, and check what version of VERS to use On Wed, Apr 27, 2016 at 9:42 AM, Walid Hetaba wrote: > Dear colleagues, > > I'm trying to get the parallel version of WIEN2k 14.2 running on the > institute's cluster, but I'm stuck now. > I searched the mailing list for the error message I receive, but at first > glance I did not find relevant postings. > > This is the problem I face: > The serial version of WIEN2k is running fine (also when using qsub and the > submission system). > However, when trying to use the parallel version I get the following error > when lapw1 is invoked in parallel mode: > "Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers." > > Here the facts on hard- and software: > I'm using WIEN2k 14.2 to run the example TiC case from the UG as a test to > see whether all works out fine. > I compiled the sofware using the Intel Cluster Studio with the Intel > Fortran Compiler 16.0, Intel C++ Compiler 16.0, Intel MKL 11.3 and Intel > MPI. > I compiled it using FFTW3 and the MKL lapack, scalapack and blacs. > > The cluster consists of 37 nodes with 20 CPUs each. It is running a CentOS > Linux distribution and it is using SGE as submission system. In the > submission script orte is used as the parallel environment. > > This is what I've tried so far: > I don't know why it can't find things from the MKL as I export all my > environmental variables using the submission script, which I also checked > by printing them. > As additional information I include the submission script > (WIEN2k-test-p.sh, which is still a bit messy as I adopted it from another > program for testing reasons), the :parallel file with the error messages > (which I renamed to .parallel so that Thunderbird on Windows can handle > it), the output-file (TiC.o14540) and the .machines file. > > I get the same error messages when using OpenMPI instead of the IntelMPI, > but the output is a bit different (see TiC.o14541). > > I did try different ways to export my environmental variables (to get rid > of unnecessary stuff), also tried mpi as the parallel environment in the > submission script instead of orte. I also tried to request a different > number of slots,... > But nothing off this got me rid of this error message. > > Has anybody experienced a similar problem or are there any hints what else > I can check? > > Thanks for your help. > All the best, > Walid > > -- > Dr.techn. Walid Hetaba > > Fritz-Haber-Institut der Max-Planck-Gesellschaft > Department of Inorganic Chemistry > Faradayweg 4-6, 14195 Berlin, Germany > T: +49 30 8413-4412het...@fhi-berlin.mpg.dehttp://www.fhi-berlin.mpg.de > > -- Professor Laurence Marks "Research is to see what everybody else has seen, and to think what nobody else has thought", Albert Szent-Gyorgi www.numis.northwestern.edu ; Corrosion in 4D: MURI4D.numis.northwestern.edu Partner of the CFW 100% program for gender equity, www.cfw.org/100-percent Co-Editor, Acta Cryst A ___ Wien mailing list Wien@zeus.theochem.tuwien.ac.at http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
[Wien] MKL error: Cannot load symbol MKLMPI_Get_wrappers
Dear colleagues,
I'm trying to get the parallel version of WIEN2k 14.2 running on the
institute's cluster, but I'm stuck now.
I searched the mailing list for the error message I receive, but at
first glance I did not find relevant postings.
This is the problem I face:
The serial version of WIEN2k is running fine (also when using qsub and
the submission system).
However, when trying to use the parallel version I get the following
error when lapw1 is invoked in parallel mode:
"Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers."
Here the facts on hard- and software:
I'm using WIEN2k 14.2 to run the example TiC case from the UG as a test
to see whether all works out fine.
I compiled the sofware using the Intel Cluster Studio with the Intel
Fortran Compiler 16.0, Intel C++ Compiler 16.0, Intel MKL 11.3 and Intel
MPI.
I compiled it using FFTW3 and the MKL lapack, scalapack and blacs.
The cluster consists of 37 nodes with 20 CPUs each. It is running a
CentOS Linux distribution and it is using SGE as submission system. In
the submission script orte is used as the parallel environment.
This is what I've tried so far:
I don't know why it can't find things from the MKL as I export all my
environmental variables using the submission script, which I also
checked by printing them.
As additional information I include the submission script
(WIEN2k-test-p.sh, which is still a bit messy as I adopted it from
another program for testing reasons), the :parallel file with the error
messages (which I renamed to .parallel so that Thunderbird on Windows
can handle it), the output-file (TiC.o14540) and the .machines file.
I get the same error messages when using OpenMPI instead of the
IntelMPI, but the output is a bit different (see TiC.o14541).
I did try different ways to export my environmental variables (to get
rid of unnecessary stuff), also tried mpi as the parallel environment in
the submission script instead of orte. I also tried to request a
different number of slots,...
But nothing off this got me rid of this error message.
Has anybody experienced a similar problem or are there any hints what
else I can check?
Thanks for your help.
All the best,
Walid
--
Dr.techn. Walid Hetaba
Fritz-Haber-Institut der Max-Planck-Gesellschaft
Department of Inorganic Chemistry
Faradayweg 4-6, 14195 Berlin, Germany
T: +49 30 8413-4412
het...@fhi-berlin.mpg.de
http://www.fhi-berlin.mpg.de
#!/bin/bash
#$ -S /bin/bash
#$ -hard
#$ -cwd
#$ -j y
#$ -m bea
#$ -l h_cpu=900
#$ -M het...@fhi-berlin.mpg.de
#$ -N TiC
$ -v MPIR_HOME=/share/apps/openmpi/bin
#
#$ -V
#
#$ -pe orte 24
#
# for orca NUMPROC must be aligned to "%pal nprocs 400 end" in input file
# and to the above statement -pe orte xxx
export NUMPROC=24
# use single threaded processes with open mpi
export OMP_NUM_THREADS=1
# on yfhix only ssh is supported
export RSH_COMMAND=ssh
export WIENROOT=/home/hetaba/WIEN2k
export PATH="$WIENROOT:$PATH"
#export SCRATCH=/scratch/hetaba
# Remnants of original submission file
#
# orca input output file
# export INPUT_FILE=${JOB_NAME}.inp
# export OUTPUT_FILE=${JOB_NAME}.out
#-- MAIN PROGRAM NAME -
#
# export PROGRAM=/home/hetaba/ORCA/orca_3_0_3_linux_x86-64/orca
#
#-- Set Environment ---
#
# Do we need this when using -V ?
#
# export PATH=/share/apps/openmpi/bin:${PATH}
# This is needed to use OpenMPI instead of IntelMPI when using -V
#
# export LD_LIBRARY_PATH=/home/hetaba/FFTW3/lib:$LD_LIBRARY_PATH
# Maybe this is needed for FFTW3, check it later
#
export
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/share/apps/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/share/apps/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64
# export LIBRARY_PATH=${LIBRARY_PATH}
#
# just to learn which SGE environment variables are available
echo " SGE_ROOT : ${SGE_ROOT}"
echo " SGE_JOB_SPOOL_DIR : ${SGE_JOB_SPOOL_DIR}"
echo " SGE_O_HOME : ${SGE_O_HOME}"
echo " SGE_O_HOST : ${SGE_O_HOST}"
echo " SGE_O_LOGNAME : ${SGE_O_LOGNAME}"
echo " SGE_O_MAIL : ${SGE_O_MAIL}"
echo " SGE_O_PATH : ${SGE_O_PATH}"
echo " SGE_O_SHELL : ${SGE_O_SHELL}"
echo " SGE_O_WORKDIR : ${SGE_O_WORKDIR}"
echo " SGE_TASK_ID : ${SGE_TASK_ID}"
echo " HOME : ${HOME}"
echo " HOSTNAME : ${HOSTNAME}"
echo " JOB_ID : ${JOB_ID}"
echo " JOB_NAME : ${JOB_NAME}"
echo " NHOSTS : ${NHOSTS}"
echo " NSLOTS : ${NSLOTS}"
echo " PE_HOSTFILE : ${PE_HOSTFILE}"
echo " hosts : `cat ${PE_HOSTFILE}`"
#echo " LD_LIBRARY_PATH: ${LD_LIBRARY_PATH}"
#echo " TMPDIR : ${TMPDIR}"
#- Generating .machines -
echo " Generating .machines file"
#
# remove old file
rm .machines
#
# For details see WIEN2k User Guide p.77
#
# Load balancing and k-point distribution
echo "granularity:1" >> .machines
echo "extrafine:1" >> .machines
#
# nodes for parallel lapw1/2
#
for host in `cat $PE_HOSTFILE | awk '{print $1 ":" $2}'`; do
echo "1:$host" >> .machines
done
#
# nodes for parallel lapw0
#
# deactivat
