Re: [gmx-users] nvcc fatal: 'avx512f': expected a number
Hi,
On Wed, Oct 24, 2018 at 11:50 AM Ehsan Moravveji
wrote:
> Dear Mark,
>
> Sincere thanks for your swift reply, and apologies for the duplicate
> message; I realized that my original message had no subject, so, I thought
> it is more proper to send the message with a meaningful topic;
>
OK
> Regarding your comment, I am building GROMACS from source, and without
> intervention from EasyBuild. In order to be transparent, I also put my full
> cmake command arguments in the original message as well. Since some other
> dependencies are installed by EasyBuild as modules, I am setting some of
> the paths in a compatible way on our system, hence e.g. EBROOTCUDA, etc.
>
Then you will have a easier time if you start with a simple invocation,
rather than manually configuring a whole host of things, including CUDA
flags. If e.g. those flags are wrongly escaped for the different wrappers,
then you will get mysterious issues like this :-) So I suggest you start
with
cmake -DGMX_SIMD=AVX_512 -DCUDA_TOOLKIT_ROOT_DIR=whatever
and gradually add the complexity that you think is needed (e.g. MKL, MPI,
NVML...)
To double check your feedback, let me ask the following:
> I am explicitly setting the flag -DGMX_SIMD='AVX_512'. Is it equivalent to
>
That's a GROMACS cmake flag.
> the flag you proposed (-mavx512f), or I have to add the latter to the
>
That's a flag for the compiler. They're not the same. But when the GROMACS
build tries to target AVX512, it checks what flags works and organizes to
use them. I assume that the way you're adding things manually to the CUDA
flags is breaking e.g. the tricks that FindCUDA.cmake has to do in order to
invoke the CUDA compiler.
Mark
> argument list to cmake, too?
>
> Sincere thanks,
> Ehsan
>
>
> On Wed, Oct 24, 2018 at 10:42 AM Mark Abraham
> wrote:
>
> > Hi,
> >
> > You sent this email yesterday. Please be a little more patient.
> >
> > Unfortunately, I have no idea how EasyBuild works, and suspect that the
> > problem lies in the way it is wrapping arguments that then get passed to
> > nvcc. If you can reproduce this issue with a simple call to cmake with
> > minimal extra stuff, then we can explore whether there's an issue with
> the
> > GROMACS build system. But since -mavx512f is a valid compiler flag that
> > GROMACS checked was valid before using it, I suspect the problem isn't
> > ours.
> >
> > Mark
> >
> > On Wed, Oct 24, 2018 at 9:55 AM Ehsan Moravveji
> > wrote:
> >
> > > Dear GMX users,
> > >
> > > I would like to share a difficulty installing GROMACS/2018.3 with you,
> > > and kindly ask for your wisdom.
> > >
> > > I use GCC/6.4.0 on a Linux cluster with CentOS 7.5 kernel and GPU
> > > nodes equipped with 4 x P100 Nvidia devices.
> > >
> > >
> > > Below is the cmake configuration that I use, followed by the error (in
> > > boldface font) that I
> > > receive. I am not sure if there is still an extra flag to set to
> resolve
> > > this issue, or some other steps must be taken?
> > >
> > > I appreciate your input.
> > >
> > > --
> > > Kind regards,
> > > Ehsan Moravveji
> > >
> > > ###
> > > cmake "${cwd}/${dir_src}" \
> > > -DGMX_SIMD='AVX_512' \
> > > -DBLAS_mkl_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> > > -DLAPACK_mkl_lapack_LIBRARY="DIR_MKL_LIB/libmkl_sequential.so" \
> > > -DBLAS_guide_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> > > -DCMAKE_C_COMPILER="${GMX_MPICC}"
> -DCMAKE_CXX_COMPILER="${GMX_MPICXX}"
> > \
> > > -DMPI_C_COMPILER="${GMX_MPICC}" -DMPI_CXX_COMPILER="${GMX_MPICXX}" \
> > > -DCMAKE_MPI_C_COMPILER="${GMX_MPICC}"
> > > -DCMAKE_MPI_CXX_COMPILER="${GMX_MPICXX}" \
> > > -DCMAKE_INSTALL_PREFIX="${dir_install}" \
> > > -DCMAKE_PREFIX_PATH="${DIR_MKL_LIB}" \
> > > -DCMAKE_VERBOSE_MAKEFILE=ON \
> > > -DGMX_MPI=ON \
> > > -DGMX_GPU=ON \
> > > -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF -DEXTRA_CXX_FLAGS="-std=c++11" \
> > > -DGMX_USE_NVML=ON -DGMX_BUILD_OWN_FFTW=ON \
> > > -DHWLOC_INCLUDE_DIRS="${EBROOTHWLOC}/include"
> > > -DHWLOC_LIBRARIES="${EBROOTHWLOC}/lib/libhwloc.so" \
> > > -DCUDA_HOST_COMPILER="${PATH_NVCC}"
> > > -DNVML_INCLUDE_DIR="${EBROOTCUDA}/include" \
> > > -DNVML_LIBRARY="${EBROOTCUDA}/lib64/stubs/libnvidia-ml.so" \
> > > -DCUDA_TOOLKIT_ROOT_DIR="${EBROOTCUDA}"
> > > -DCUDA_SDK_ROOT_DIR="${EBROOTCUDA}" \
> > > -DCUDA_PROPAGATE_HOST_FLAGS=ON -DCUDA_VERBOSE_BUILD=ON \
> > > -DGMX_CUDA_TARGET_SM="${SM_VER}"
> -DGMX_CUDA_TARGET_COMPUTE="${SM_VER}"
> > \
> > > -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF \
> > > -DCUDA_CUDART_LIBRARY="${EBROOTCUDA}/lib64/libcudart.so"
> > > -DCUDA_CUDA_LIBRARY="${EBROOTCUDA}/lib64/libcuda.so" \
> > >
> > >
> > >
> >
> -DCUDA_NVCC_FLAGS="-gencode;arch=compute_${SM_VER},code=compute_${SM_VER};-use_fast_math;-D_FORCE_INLINES"
> > > \
> > > -DGMX_PREFER_STATIC_LIBS=OFF -DGMX_X11=OFF
> > >
> > > ###
> > >
> > > -- Generating dependency file:
> > >
> > >
> >
>
Re: [gmx-users] nvcc fatal: 'avx512f': expected a number
Dear Mark,
Sincere thanks for your swift reply, and apologies for the duplicate
message; I realized that my original message had no subject, so, I thought
it is more proper to send the message with a meaningful topic;
Regarding your comment, I am building GROMACS from source, and without
intervention from EasyBuild. In order to be transparent, I also put my full
cmake command arguments in the original message as well. Since some other
dependencies are installed by EasyBuild as modules, I am setting some of
the paths in a compatible way on our system, hence e.g. EBROOTCUDA, etc.
To double check your feedback, let me ask the following:
I am explicitly setting the flag -DGMX_SIMD='AVX_512'. Is it equivalent to
the flag you proposed (-mavx512f), or I have to add the latter to the
argument list to cmake, too?
Sincere thanks,
Ehsan
On Wed, Oct 24, 2018 at 10:42 AM Mark Abraham
wrote:
> Hi,
>
> You sent this email yesterday. Please be a little more patient.
>
> Unfortunately, I have no idea how EasyBuild works, and suspect that the
> problem lies in the way it is wrapping arguments that then get passed to
> nvcc. If you can reproduce this issue with a simple call to cmake with
> minimal extra stuff, then we can explore whether there's an issue with the
> GROMACS build system. But since -mavx512f is a valid compiler flag that
> GROMACS checked was valid before using it, I suspect the problem isn't
> ours.
>
> Mark
>
> On Wed, Oct 24, 2018 at 9:55 AM Ehsan Moravveji
> wrote:
>
> > Dear GMX users,
> >
> > I would like to share a difficulty installing GROMACS/2018.3 with you,
> > and kindly ask for your wisdom.
> >
> > I use GCC/6.4.0 on a Linux cluster with CentOS 7.5 kernel and GPU
> > nodes equipped with 4 x P100 Nvidia devices.
> >
> >
> > Below is the cmake configuration that I use, followed by the error (in
> > boldface font) that I
> > receive. I am not sure if there is still an extra flag to set to resolve
> > this issue, or some other steps must be taken?
> >
> > I appreciate your input.
> >
> > --
> > Kind regards,
> > Ehsan Moravveji
> >
> > ###
> > cmake "${cwd}/${dir_src}" \
> > -DGMX_SIMD='AVX_512' \
> > -DBLAS_mkl_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> > -DLAPACK_mkl_lapack_LIBRARY="DIR_MKL_LIB/libmkl_sequential.so" \
> > -DBLAS_guide_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> > -DCMAKE_C_COMPILER="${GMX_MPICC}" -DCMAKE_CXX_COMPILER="${GMX_MPICXX}"
> \
> > -DMPI_C_COMPILER="${GMX_MPICC}" -DMPI_CXX_COMPILER="${GMX_MPICXX}" \
> > -DCMAKE_MPI_C_COMPILER="${GMX_MPICC}"
> > -DCMAKE_MPI_CXX_COMPILER="${GMX_MPICXX}" \
> > -DCMAKE_INSTALL_PREFIX="${dir_install}" \
> > -DCMAKE_PREFIX_PATH="${DIR_MKL_LIB}" \
> > -DCMAKE_VERBOSE_MAKEFILE=ON \
> > -DGMX_MPI=ON \
> > -DGMX_GPU=ON \
> > -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF -DEXTRA_CXX_FLAGS="-std=c++11" \
> > -DGMX_USE_NVML=ON -DGMX_BUILD_OWN_FFTW=ON \
> > -DHWLOC_INCLUDE_DIRS="${EBROOTHWLOC}/include"
> > -DHWLOC_LIBRARIES="${EBROOTHWLOC}/lib/libhwloc.so" \
> > -DCUDA_HOST_COMPILER="${PATH_NVCC}"
> > -DNVML_INCLUDE_DIR="${EBROOTCUDA}/include" \
> > -DNVML_LIBRARY="${EBROOTCUDA}/lib64/stubs/libnvidia-ml.so" \
> > -DCUDA_TOOLKIT_ROOT_DIR="${EBROOTCUDA}"
> > -DCUDA_SDK_ROOT_DIR="${EBROOTCUDA}" \
> > -DCUDA_PROPAGATE_HOST_FLAGS=ON -DCUDA_VERBOSE_BUILD=ON \
> > -DGMX_CUDA_TARGET_SM="${SM_VER}" -DGMX_CUDA_TARGET_COMPUTE="${SM_VER}"
> \
> > -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF \
> > -DCUDA_CUDART_LIBRARY="${EBROOTCUDA}/lib64/libcudart.so"
> > -DCUDA_CUDA_LIBRARY="${EBROOTCUDA}/lib64/libcuda.so" \
> >
> >
> >
> -DCUDA_NVCC_FLAGS="-gencode;arch=compute_${SM_VER},code=compute_${SM_VER};-use_fast_math;-D_FORCE_INLINES"
> > \
> > -DGMX_PREFER_STATIC_LIBS=OFF -DGMX_X11=OFF
> >
> > ###
> >
> > -- Generating dependency file:
> >
> >
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend
> > /software/CUDA/9.2.148/bin/nvcc -M -D__CUDACC__
> > /work/GROMACS/gromacs-2018.3/src/gromacs/mdlib/nbnxn_cuda/nbnxn_cuda.cu
> -o
> >
> >
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend
> > -ccbin /software/CUDA/9.2.148/bin/nvcc -m64 --std
> > c++11 -Dlibgromacs_EXPORTS -DGMX_DOUBLE=0 -DHAVE_CONFIG_H
> > -DUSE_STD_INTTYPES_H -Xcompiler
> >
> >
> ,\"-mavx512f\",\"-mfma\",\"-fPIC\",\"-O3\",\"-DNDEBUG\",\"-funroll-all-loops\",\"-fexcess-precision=fast\"
> > -gencode arch=compute_60,code=sm_60 -gencode
> > arch=compute_60,code=compute_60 -use_fast_math -gencode
> > arch=compute_60,code=compute_60 -use_fast_math -D_FORCE_INLINES -DNVCC
> > -I/software/CUDA/9.2.148/include
> > -I/work/GROMACS/gromacs-2018.3/src/external/lmfit
> >
> >
> -I/work/GROMACS/gromacs-2018.3-build/src/contrib/fftw/fftwBuild-prefix/include
> > -I/work/GROMACS/gromacs-2018.3-build/src
> >
Re: [gmx-users] nvcc fatal: 'avx512f': expected a number
Hi,
You sent this email yesterday. Please be a little more patient.
Unfortunately, I have no idea how EasyBuild works, and suspect that the
problem lies in the way it is wrapping arguments that then get passed to
nvcc. If you can reproduce this issue with a simple call to cmake with
minimal extra stuff, then we can explore whether there's an issue with the
GROMACS build system. But since -mavx512f is a valid compiler flag that
GROMACS checked was valid before using it, I suspect the problem isn't ours.
Mark
On Wed, Oct 24, 2018 at 9:55 AM Ehsan Moravveji
wrote:
> Dear GMX users,
>
> I would like to share a difficulty installing GROMACS/2018.3 with you,
> and kindly ask for your wisdom.
>
> I use GCC/6.4.0 on a Linux cluster with CentOS 7.5 kernel and GPU
> nodes equipped with 4 x P100 Nvidia devices.
>
>
> Below is the cmake configuration that I use, followed by the error (in
> boldface font) that I
> receive. I am not sure if there is still an extra flag to set to resolve
> this issue, or some other steps must be taken?
>
> I appreciate your input.
>
> --
> Kind regards,
> Ehsan Moravveji
>
> ###
> cmake "${cwd}/${dir_src}" \
> -DGMX_SIMD='AVX_512' \
> -DBLAS_mkl_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> -DLAPACK_mkl_lapack_LIBRARY="DIR_MKL_LIB/libmkl_sequential.so" \
> -DBLAS_guide_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> -DCMAKE_C_COMPILER="${GMX_MPICC}" -DCMAKE_CXX_COMPILER="${GMX_MPICXX}" \
> -DMPI_C_COMPILER="${GMX_MPICC}" -DMPI_CXX_COMPILER="${GMX_MPICXX}" \
> -DCMAKE_MPI_C_COMPILER="${GMX_MPICC}"
> -DCMAKE_MPI_CXX_COMPILER="${GMX_MPICXX}" \
> -DCMAKE_INSTALL_PREFIX="${dir_install}" \
> -DCMAKE_PREFIX_PATH="${DIR_MKL_LIB}" \
> -DCMAKE_VERBOSE_MAKEFILE=ON \
> -DGMX_MPI=ON \
> -DGMX_GPU=ON \
> -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF -DEXTRA_CXX_FLAGS="-std=c++11" \
> -DGMX_USE_NVML=ON -DGMX_BUILD_OWN_FFTW=ON \
> -DHWLOC_INCLUDE_DIRS="${EBROOTHWLOC}/include"
> -DHWLOC_LIBRARIES="${EBROOTHWLOC}/lib/libhwloc.so" \
> -DCUDA_HOST_COMPILER="${PATH_NVCC}"
> -DNVML_INCLUDE_DIR="${EBROOTCUDA}/include" \
> -DNVML_LIBRARY="${EBROOTCUDA}/lib64/stubs/libnvidia-ml.so" \
> -DCUDA_TOOLKIT_ROOT_DIR="${EBROOTCUDA}"
> -DCUDA_SDK_ROOT_DIR="${EBROOTCUDA}" \
> -DCUDA_PROPAGATE_HOST_FLAGS=ON -DCUDA_VERBOSE_BUILD=ON \
> -DGMX_CUDA_TARGET_SM="${SM_VER}" -DGMX_CUDA_TARGET_COMPUTE="${SM_VER}" \
> -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF \
> -DCUDA_CUDART_LIBRARY="${EBROOTCUDA}/lib64/libcudart.so"
> -DCUDA_CUDA_LIBRARY="${EBROOTCUDA}/lib64/libcuda.so" \
>
>
> -DCUDA_NVCC_FLAGS="-gencode;arch=compute_${SM_VER},code=compute_${SM_VER};-use_fast_math;-D_FORCE_INLINES"
> \
> -DGMX_PREFER_STATIC_LIBS=OFF -DGMX_X11=OFF
>
> ###
>
> -- Generating dependency file:
>
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend
> /software/CUDA/9.2.148/bin/nvcc -M -D__CUDACC__
> /work/GROMACS/gromacs-2018.3/src/gromacs/mdlib/nbnxn_cuda/nbnxn_cuda.cu -o
>
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend
> -ccbin /software/CUDA/9.2.148/bin/nvcc -m64 --std
> c++11 -Dlibgromacs_EXPORTS -DGMX_DOUBLE=0 -DHAVE_CONFIG_H
> -DUSE_STD_INTTYPES_H -Xcompiler
>
> ,\"-mavx512f\",\"-mfma\",\"-fPIC\",\"-O3\",\"-DNDEBUG\",\"-funroll-all-loops\",\"-fexcess-precision=fast\"
> -gencode arch=compute_60,code=sm_60 -gencode
> arch=compute_60,code=compute_60 -use_fast_math -gencode
> arch=compute_60,code=compute_60 -use_fast_math -D_FORCE_INLINES -DNVCC
> -I/software/CUDA/9.2.148/include
> -I/work/GROMACS/gromacs-2018.3/src/external/lmfit
>
> -I/work/GROMACS/gromacs-2018.3-build/src/contrib/fftw/fftwBuild-prefix/include
> -I/work/GROMACS/gromacs-2018.3-build/src
> -I/work/GROMACS/gromacs-2018.3/src/external/thread_mpi/include
> -I/work/GROMACS/gromacs-2018.3/src
> -I/software/hwloc/1.11.11-GCCcore-6.4.0/include
> -I/work/GROMACS/gromacs-2018.3/src/external/tng_io/include
> -I/work/GROMACS/gromacs-2018.3-build/tng/include*nvcc fatal :
> 'avx512f': expected a number
> CMake Error at libgromacs_generated_nbnxn_cuda.cu.o.Release.cmake:219
> (message):
> Error generating
>
>
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/./libgromacs_generated_nbnxn_cuda.cu.o*
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-requ...@gromacs.org.
>
--
Gromacs Users mailing list
* Please search the archive at
http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
* Can't post? Read
