On 5/24/22 17:37, Robert MIJAKOVIC wrote:
I tried to specify the location of fftw3.f03 file by setting INCLUDE_FFTW
variable
should be "FFTW_INCLUDE"
mpif90 -mp -cuda -gpu=cc80,cuda11.0 -acc -o pw.x \
pwscf.o libpw.a ../../KS_Solvers/libks_solvers.a ../../dft-d3/libdftd3qe.a /mnt/tier2/users/apps/scratchpad/qe-7.0//Modules/libqemod.a /mnt/tier2/users/apps/scratchpad/qe-7.0//upflib/libupf.a /mnt/tier2/users/apps/scratchpad/qe-7.0//XClib/xc_lib.a /mnt/tier2/users/apps/scratchpad/qe-7.0//FFTXlib/libqefft.a /mnt/tier2/users/apps/scratchpad/qe-7.0//LAXlib/libqela.a /mnt/tier2/users/apps/scratchpad/qe-7.0//UtilXlib/libutil.a /mnt/tier2/users/apps/scratchpad/qe-7.0//MBD/libmbd.a -cudalib=cufft,cublas,cusolver,curand /mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src/libdevXlib.a -L/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src -ldevXlib -L/apps/USE/easybuild/staging/2021.1/software/NVHPC/22.3/Linux_x86_64/22.3/compilers/lib -llapack -L/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/lib -lFoX_dom -lFoX_sax -lFoX_wxml -lFoX_common -lFoX_utils -lFoX_fsys /apps/USE/easybuild/staging/2021.1/software/FFTW/3.3.9-nvompic-2021a/lib
should be "-L/apps/..."
/apps/USE/easybuild/staging/2021.1/software/binutils/2.36.1-GCCcore-10.3.0/bin/ld:
read in flex scanner failed
maybe it is a consequence of the above point? if not, there is something
(very) strange in your software
Paolo
As you can see, there I reach another issue.
I need to search through the users mailing list to see if somebody else
experienced something similar.
In case you have some suggestions, I'm happy to hear them.
Best regards,
Robert
-----Original Message-----
From: Paolo Giannozzi <[email protected]>
Sent: Monday, 23 May 2022 21:46
To: Quantum ESPRESSO users Forum <[email protected]>; Robert MIJAKOVIC
<[email protected]>
Subject: Re: [QE-users] [QE 7.0/GPU] Build configured with Autotools fails
Here some explanations:
https://www.mail-archive.com/[email protected]/msg41494.html
https://www.mail-archive.com/[email protected]/msg40215.html
The path to the fftw3.f03 file, needed by fftw3, must be specified in
INCLUDE_FFTW or similar variables. The latest development version (soon to be
released) has somewhat better FFTW3 auto-detection
Paolo
On 23/05/2022 17:55, Robert MIJAKOVIC wrote:
# Summary
QE 7.0/GPU compilation with Autotools fails with "Can't find include
file fftw3.f03 (fft_scalar.FFTW3.f90: 40)"
# Version
qe-7.0-ReleasePack.tgz
# Environment
## Hardware
1. 2xAMD EPYC 7452
2. 4xNVIDIA A100
3. 512 GB RAM
## Software
1. OS: Rocky Linux release 8.5 (Green Obsidian)
2. NVHPC 22.3
3. OpenMPI 4.1.3 built with NVHPC 22.3
4. CUDA 11.3.1 with Driver 470.82.01
5. libxc 5.1.5
6. Autotools 20210128
# Steps to reproduce
## Configured with:
`./configure --prefix=$HOME/install/qe-7.0 --enable-openmp
--with-scalapack=no --enable-cuda-env-check=no
--with-cuda=$EBROOTNVHPC/Linux_x86_64/22.3/cuda/11.0
--with-cuda-runtime=11.0 --with-cuda-cc=80`
## Prebuild options
`cp $EBROOTLIBXC/include/*.mod Modules/mod/qe_modules && export
FPP='nvfortran -Mpreprocess -E' && export CPP='cpp -E' && export
FCPP='cpp -E' && `
## make options
`make all epw`
# Observed behavior
Compilation fails with:
```
make[1]: Entering directory
'/mnt/tier2/users/apps/scratchpad/qe-7.0/FFTXlib'
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_param.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_support.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
stick_base.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_types.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
scatter_mod.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_scatter_2d.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_buffers.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_scatter_gpu.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_scatter_2d_gpu.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_ggen.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_scalar.DFTI.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_scalar.ESSL.f90
NVFORTRAN-W-0006-Input file empty (fft_scalar.ESSL.f90)
NVFORTRAN/x86-64 Linux 22.3-0: compilation completed with warnings
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fftw_interfaces.f90
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_scalar.FFTW.f90
NVFORTRAN-W-0006-Input file empty (fft_scalar.FFTW.f90)
NVFORTRAN/x86-64 Linux 22.3-0: compilation completed with warnings
mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI
-D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI -cuda -gpu=cc80,cuda11.0
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include
-acc -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//include
-I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude -c
fft_scalar.FFTW3.f90
NVFORTRAN-F-0906-Can't find include file fftw3.f03
(fft_scalar.FFTW3.f90: 40)
NVFORTRAN/x86-64 Linux 22.3-0: compilation aborted
make[1]: *** [../make.inc:16: fft_scalar.FFTW3.o] Error 2
make[1]: Leaving directory '/mnt/tier2/users/apps/scratchpad/qe-7.0/FFTXlib'
make: *** [Makefile:196: libfft] Error 1
```
# Questions
1. What do I do wrong?
2. Why configure didn't fail if FFTW is mandatory?
3. Is it possible to configure the build system to use cuFFTinstead?
Dr. rer. nat. Robert Mijaković | HPC System Software Architect
*Lux**Provide*
3, Op der Poukewiss | L-7795 Bissen
Grand-Duchy of Luxembourg
M (+352) 691 396 474
[email protected]
<mailto:[email protected]> |www.luxprovide.lu
<http://www.luxprovide.lu/>
_______________________________________________
The Quantum ESPRESSO community stands by the Ukrainian people and
expresses its concerns about the devastating effects that the Russian
military offensive has on their country and on the free and peaceful
scientific, cultural, and economic cooperation amongst peoples
_______________________________________________
Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing
list [email protected]
https://lists.quantum-espresso.org/mailman/listinfo/users
--
Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche, Univ. Udine,
via delle Scienze 208, 33100 Udine, Italy Phone +39-0432-558216, fax
+39-0432-558222
--
Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
Univ. Udine, via delle Scienze 206, 33100 Udine, Italy
Phone +39-0432-558216, fax +39-0432-558222
_______________________________________________
The Quantum ESPRESSO community stands by the Ukrainian
people and expresses its concerns about the devastating
effects that the Russian military offensive has on their
country and on the free and peaceful scientific, cultural,
and economic cooperation amongst peoples
_______________________________________________
Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
users mailing list [email protected]
https://lists.quantum-espresso.org/mailman/listinfo/users
