Hi Pavel, Many thanks for your insights. As you know, I am not an expert on how to compile codes, for me, this is sadly a trial and error adventure.
I tried to compile it against the openblas library, but although the compilation ends without any errors, I get a segmentation fault when running lapw1 (on the test case from http://www.wien2k.at/reg_user/benchmark/). The current setting are: L Linker Flags: $(FOPT) -L/usr/lib/x86_64-linux-gnu/openblas64-openmp R R_LIBS (LAPACK+BLAS): /usr/lib/x86_64-linux-gnu/openblas64-openmp/libopenblas64.so.0 -lpthread (The rest is as I wrote in my first email.) Here is the list of linked libraries: $ ldd lapw1 linux-vdso.so.1 (0x00007ffea57d6000) *libopenblas64.so.0 => /lib/x86_64-linux-gnu/libopenblas64.so.0 (0x000014fe2b2e5000) * libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000014fe2b2c2000) libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x000014fe2affa000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000014fe2aeab000) libmvec.so.1 => /lib/x86_64-linux-gnu/libmvec.so.1 (0x000014fe2ae7f000) libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x000014fe2ae3d000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000014fe2ac49000) /lib64/ld-linux-x86-64.so.2 (0x000014fe2d4d3000) libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x000014fe2abff000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000014fe2abe4000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x000014fe2abde000) And here is the stacking fault (it doesn't tell me anything): $x lapw1 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x148fb48b8d21 in ??? #1 0x148fb48b7ef5 in ??? #2 0x148fb452d20f in ??? #3 0x148fb54495fb in ??? Segmentation fault 0.949u 0.472s 0:00.84 167.8% 0+0k 14400+10992io 31pf+0w Any idea? With the settings I reported yesterday, everything works just fine (though probably not very efficiently - but this is not a problem for me as this binary is not a "production" binary on any HPC): $ ldd lapw1 linux-vdso.so.1 (0x00007ffc60765000) *libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x0000150cec90f000) liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x0000150cec26b000) * libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x0000150cec248000) libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x0000150cebf80000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x0000150cebe31000) libmvec.so.1 => /lib/x86_64-linux-gnu/libmvec.so.1 (0x0000150cebe05000) libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x0000150cebdc1000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000150cebbcf000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x0000150cebbb4000) /lib64/ld-linux-x86-64.so.2 (0x0000150cec9fc000) libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x0000150cebb6a000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x0000150cebb64000) and: $ x lapw1 STOP LAPW1 END 162.045u 0.918s 2:31.25 107.7% 0+0k 8976+37856io 35pf+0w $ grep HORB *output1* test_case.output1: TIME HAMILT (CPU) = 16.3, HNS = 20.0, HORB = 0.0, DIAG = 125.9, SYNC = 0.0 test_case.output1: TIME HAMILT (WALL) = 4.4, HNS = 20.0, HORB = 0.0, DIAG = 126.0, SYNC = 0.0 (I am using it on my 10-year old "type writer": $lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Vendor ID: GenuineIntel CPU family: 6 Model: 26 Model name: Intel(R) Xeon(R) CPU W3550 @ 3.07GHz ) --- *Dr David Holec* *Computational Materials Science group* Department of Materials Science Montanuniversität Leoben Franz-Josef-Strasse 18, A-8700 Leoben, Austria tel. +43-(0)3842-4024211 fax. +43-(0)3842-4024202 materials.unileoben.ac.at cms.unileoben.ac.at ________________________________ WHERE RESEARCH MEETS FUTURE On Wed, 24 Nov 2021 at 08:27, Pavel Ondračka <[email protected]> wrote: > Hi David, > > as you said it works for you, so feel free to ignore, but I have some > further tips if you are interested. Ubuntu switches between the > different blas and lapack using the "alternatives", so its difficult to > say if you actually link with the correct one. > > "ldd lapw1" in WIENROOT should show which one is actually linked, what > you want to have is the openmp openblas > /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so > /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so > or alternatively > /usr/lib/x86_64-linux-gnu/openblas-openmp/libopenblas.so > It looks like you linked with the pthread one. This is not a problem > when running at single thread but at higher thread number this might > lead to oversubscription and slowdowns as the pthreaded openblas > doesn't respect the OMP_NUM_THREADS set by Wien2k. So I would recommend > to relink with the openmp OpenBLAS. BTW it is usually safer to link > with OpenBLAS explicitly using the -lopenblas instead of the -llapack - > lblas to be sure you don't accidentally link the netlib one > (libopenblas is just the libblas and libblapack provided by OpenBLAS > merged together). > > In general easy way how to check performance is to run the serial > test_case from http://www.wien2k.at/reg_user/benchmark/ On modern CPUs > (at least avx2) the runtime should be around 15-25 seconds at single > thread. > > I see total runtime of ~18seconds on Fedora 35 with gfortran 11.2.1 > OpenBLAS and AMD Ryzen 9 3900X 12-Core Processor. > Also look for the following line in test_case.output1, this is what I > have: > TIME HAMILT (WALL) = 2.2, HNS = 1.7, HORB = 0.0, DIAG = > 14.0, SYNC = 0.0 > The time in HAMILT mostly depends on you compiler and vectorizing > settings, while the DIAG is 99% lapack/blas related, so this can help > with the diagnostics if things are slow. > > You might also get extra speedup of the HAMILT part by adding "- > DHAVE_LIBMVEC" to the Compiler options. > > Best regards > Pavel > > On Tue, 2021-11-23 at 11:07 +0100, David Holec wrote: > > Dear all, > > > > I have just spent some time making Wien2k run on my single machine > > running Ubuntu 20.04 with gfortran/gcc. Since I am not an expert, it > > was a trial and error, but it seems that I found a working combination > > (sadly, the default parameters didn't work for me). Maybe this will > > help someone. Here are the settings that did the job for me: > > > > M OpenMP switch: -fopenmp > > O Compiler options: -ffree-form -O2 -ftree-vectorize - > > march=native -ffree-line-length-none -ffpe-summary=none > > L Linker Flags: $(FOPT) -L/usr/lib/x86_64-linux-gnu > > P Preprocessor flags '-DParallel' > > R R_LIBS (LAPACK+BLAS): -lblas -llapack -lpthread > > F FFTW options: -DFFTW3 -DFFTW_OMP -I/usr/include > > FFTW-LIBS: -L/usr/lib/x86_64-linux-gnu -lfftw3 - > > lfftw3_omp > > > > where the FFTW options were: > > > > R FFTWROOT: /usr/ > > V FFTW_VERSION: FFTW3 > > L FFTW_LIB: lib/x86_64-linux-gnu > > N FFTW_LIBNAME: fftw3 > > > > Compiler versions: > > $ gcc --version > > gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 > > gfortran --version > > GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 > > > > And I used the generic lapack and openblas packages provides by Ubuntu > > repos: > > liblapack-dev/focal,now 3.9.0-1build1 amd64 [installed] > > liblapack3/focal,now 3.9.0-1build1 amd64 [installed,automatic] > > > > liblapack64-3/focal,now 3.9.0-1build1 amd64 [installed,automatic] > > liblapack64-dev/focal,now 3.9.0-1build1 amd64 [installed] > > > > libblas-dev/focal,now 3.9.0-1build1 amd64 [installed] > > libblas3/focal,now 3.9.0-1build1 amd64 [installed,automatic] > > libblas64-3/focal,now 3.9.0-1build1 amd64 [installed,automatic] > > libblas64-dev/focal,now 3.9.0-1build1 amd64 [installed,automatic] > > > > libopenblas64-0/focal-updates,now 0.3.8+ds-1ubuntu0.20.04.1 amd64 > > [installed] > > libopenblas64-0-openmp/focal-updates,now 0.3.8+ds-1ubuntu0.20.04.1 > > amd64 [installed] > > libopenblas64-0-pthread/focal-updates,now 0.3.8+ds-1ubuntu0.20.04.1 > > amd64 [installed,automatic] > > > > (I am not totally sure if I need all the libraries above, but > > certainly, with these, the compilation seems to work and I am able to > > run SCF cycles & Telnes calculations without errors :-) > > > > All the best, > > David > > --- > > Dr David Holec > > Computational Materials Science group > > Department of Materials Science > > Montanuniversität Leoben > > > > > > > > Franz-Josef-Strasse 18, A-8700 Leoben, Austria > > tel. +43-(0)3842-4024211 > > fax. +43-(0)3842-4024202 > > materials.unileoben.ac.at > > cms.unileoben.ac.at > > ________________________________ > > WHERE RESEARCH MEETS FUTURE > > _______________________________________________ > > Wien mailing list > > [email protected] > > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien > > SEARCH the MAILING-LIST at: > > http://www.mail-archive.com/[email protected]/index.html > > _______________________________________________ > Wien mailing list > [email protected] > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien > SEARCH the MAILING-LIST at: > http://www.mail-archive.com/[email protected]/index.html >
_______________________________________________ Wien mailing list [email protected] http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/[email protected]/index.html
