Hi, On Wed, Oct 24, 2018 at 11:50 AM Ehsan Moravveji <[email protected]> wrote:
> Dear Mark, > > Sincere thanks for your swift reply, and apologies for the duplicate > message; I realized that my original message had no subject, so, I thought > it is more proper to send the message with a meaningful topic; > OK > Regarding your comment, I am building GROMACS from source, and without > intervention from EasyBuild. In order to be transparent, I also put my full > cmake command arguments in the original message as well. Since some other > dependencies are installed by EasyBuild as modules, I am setting some of > the paths in a compatible way on our system, hence e.g. EBROOTCUDA, etc. > Then you will have a easier time if you start with a simple invocation, rather than manually configuring a whole host of things, including CUDA flags. If e.g. those flags are wrongly escaped for the different wrappers, then you will get mysterious issues like this :-) So I suggest you start with cmake -DGMX_SIMD=AVX_512 -DCUDA_TOOLKIT_ROOT_DIR=whatever and gradually add the complexity that you think is needed (e.g. MKL, MPI, NVML...) To double check your feedback, let me ask the following: > I am explicitly setting the flag -DGMX_SIMD='AVX_512'. Is it equivalent to > That's a GROMACS cmake flag. > the flag you proposed (-mavx512f), or I have to add the latter to the > That's a flag for the compiler. They're not the same. But when the GROMACS build tries to target AVX512, it checks what flags works and organizes to use them. I assume that the way you're adding things manually to the CUDA flags is breaking e.g. the tricks that FindCUDA.cmake has to do in order to invoke the CUDA compiler. Mark > argument list to cmake, too? > > Sincere thanks, > Ehsan > > > On Wed, Oct 24, 2018 at 10:42 AM Mark Abraham <[email protected]> > wrote: > > > Hi, > > > > You sent this email yesterday. Please be a little more patient. > > > > Unfortunately, I have no idea how EasyBuild works, and suspect that the > > problem lies in the way it is wrapping arguments that then get passed to > > nvcc. If you can reproduce this issue with a simple call to cmake with > > minimal extra stuff, then we can explore whether there's an issue with > the > > GROMACS build system. But since -mavx512f is a valid compiler flag that > > GROMACS checked was valid before using it, I suspect the problem isn't > > ours. > > > > Mark > > > > On Wed, Oct 24, 2018 at 9:55 AM Ehsan Moravveji <[email protected]> > > wrote: > > > > > Dear GMX users, > > > > > > I would like to share a difficulty installing GROMACS/2018.3 with you, > > > and kindly ask for your wisdom. > > > > > > I use GCC/6.4.0 on a Linux cluster with CentOS 7.5 kernel and GPU > > > nodes equipped with 4 x P100 Nvidia devices. > > > > > > > > > Below is the cmake configuration that I use, followed by the error (in > > > boldface font) that I > > > receive. I am not sure if there is still an extra flag to set to > resolve > > > this issue, or some other steps must be taken? > > > > > > I appreciate your input. > > > > > > -- > > > Kind regards, > > > Ehsan Moravveji > > > > > > ########################################################### > > > cmake "${cwd}/${dir_src}" \ > > > -DGMX_SIMD='AVX_512' \ > > > -DBLAS_mkl_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \ > > > -DLAPACK_mkl_lapack_LIBRARY="DIR_MKL_LIB/libmkl_sequential.so" \ > > > -DBLAS_guide_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \ > > > -DCMAKE_C_COMPILER="${GMX_MPICC}" > -DCMAKE_CXX_COMPILER="${GMX_MPICXX}" > > \ > > > -DMPI_C_COMPILER="${GMX_MPICC}" -DMPI_CXX_COMPILER="${GMX_MPICXX}" \ > > > -DCMAKE_MPI_C_COMPILER="${GMX_MPICC}" > > > -DCMAKE_MPI_CXX_COMPILER="${GMX_MPICXX}" \ > > > -DCMAKE_INSTALL_PREFIX="${dir_install}" \ > > > -DCMAKE_PREFIX_PATH="${DIR_MKL_LIB}" \ > > > -DCMAKE_VERBOSE_MAKEFILE=ON \ > > > -DGMX_MPI=ON \ > > > -DGMX_GPU=ON \ > > > -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF -DEXTRA_CXX_FLAGS="-std=c++11" \ > > > -DGMX_USE_NVML=ON -DGMX_BUILD_OWN_FFTW=ON \ > > > -DHWLOC_INCLUDE_DIRS="${EBROOTHWLOC}/include" > > > -DHWLOC_LIBRARIES="${EBROOTHWLOC}/lib/libhwloc.so" \ > > > -DCUDA_HOST_COMPILER="${PATH_NVCC}" > > > -DNVML_INCLUDE_DIR="${EBROOTCUDA}/include" \ > > > -DNVML_LIBRARY="${EBROOTCUDA}/lib64/stubs/libnvidia-ml.so" \ > > > -DCUDA_TOOLKIT_ROOT_DIR="${EBROOTCUDA}" > > > -DCUDA_SDK_ROOT_DIR="${EBROOTCUDA}" \ > > > -DCUDA_PROPAGATE_HOST_FLAGS=ON -DCUDA_VERBOSE_BUILD=ON \ > > > -DGMX_CUDA_TARGET_SM="${SM_VER}" > -DGMX_CUDA_TARGET_COMPUTE="${SM_VER}" > > \ > > > -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF \ > > > -DCUDA_CUDART_LIBRARY="${EBROOTCUDA}/lib64/libcudart.so" > > > -DCUDA_CUDA_LIBRARY="${EBROOTCUDA}/lib64/libcuda.so" \ > > > > > > > > > > > > -DCUDA_NVCC_FLAGS="-gencode;arch=compute_${SM_VER},code=compute_${SM_VER};-use_fast_math;-D_FORCE_INLINES" > > > \ > > > -DGMX_PREFER_STATIC_LIBS=OFF -DGMX_X11=OFF > > > > > > ########################################################### > > > > > > -- Generating dependency file: > > > > > > > > > /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend > > > /software/CUDA/9.2.148/bin/nvcc -M -D__CUDACC__ > > > /work/GROMACS/gromacs-2018.3/src/gromacs/mdlib/nbnxn_cuda/ > nbnxn_cuda.cu > > -o > > > > > > > > > /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend > > > -ccbin /software/CUDA/9.2.148/bin/nvcc -m64 --std > > > c++11 -Dlibgromacs_EXPORTS -DGMX_DOUBLE=0 -DHAVE_CONFIG_H > > > -DUSE_STD_INTTYPES_H -Xcompiler > > > > > > > > > ,\"-mavx512f\",\"-mfma\",\"-fPIC\",\"-O3\",\"-DNDEBUG\",\"-funroll-all-loops\",\"-fexcess-precision=fast\" > > > -gencode arch=compute_60,code=sm_60 -gencode > > > arch=compute_60,code=compute_60 -use_fast_math -gencode > > > arch=compute_60,code=compute_60 -use_fast_math -D_FORCE_INLINES -DNVCC > > > -I/software/CUDA/9.2.148/include > > > -I/work/GROMACS/gromacs-2018.3/src/external/lmfit > > > > > > > > > -I/work/GROMACS/gromacs-2018.3-build/src/contrib/fftw/fftwBuild-prefix/include > > > -I/work/GROMACS/gromacs-2018.3-build/src > > > -I/work/GROMACS/gromacs-2018.3/src/external/thread_mpi/include > > > -I/work/GROMACS/gromacs-2018.3/src > > > -I/software/hwloc/1.11.11-GCCcore-6.4.0/include > > > -I/work/GROMACS/gromacs-2018.3/src/external/tng_io/include > > > -I/work/GROMACS/gromacs-2018.3-build/tng/include*nvcc fatal : > > > 'avx512f': expected a number > > > CMake Error at libgromacs_generated_nbnxn_cuda.cu.o.Release.cmake:219 > > > (message): > > > Error generating > > > > > > > > > > > > /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/./libgromacs_generated_nbnxn_cuda.cu.o* > > > -- > > > Gromacs Users mailing list > > > > > > * Please search the archive at > > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > > posting! > > > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > > > * For (un)subscribe requests visit > > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > > send a mail to [email protected]. > > > > > -- > > Gromacs Users mailing list > > > > * Please search the archive at > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > posting! > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > * For (un)subscribe requests visit > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > send a mail to [email protected]. > > > > > -- > Kind regards, > Ehsan Moravveji > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to [email protected]. > -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to [email protected].
