Re: [gmx-users] Problem with CUDA
Thanks. That was already set by cuda module. Didn't help me.
Chris
-Original Message-
From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se
[mailto:gromacs.org_gmx-users-boun...@maillist.sys.kth.se] On Behalf Of Szilárd
Páll
Sent: Saturday, April 07, 2018 11:28 AM
To: Discussion list for GROMACS users <gmx-us...@gromacs.org>
Cc: gromacs.org_gmx-users@maillist.sys.kth.se
Subject: Re: [gmx-users] Problem with CUDA
Sorry, forgot to mention that for dynamic linking you (might?) need export
CRAYPE_LINK_TYPE=dynamic
--
Szilárd
On Sat, Apr 7, 2018 at 12:09 AM, Borchert, Christopher B ERDC-RDE-ITL-MS
Contractor <christopher.b.borch...@erdc.dren.mil>
wrote:
> Unfortunately using your shortened cmake args, I still get fpic
> errors. But it does complete the build statically with
> -DGMX_BUILD_SHARED_EXE=OFF
>
> CC=cc CXX=CC cmake ../ -DGMX_SIMD=AVX2_256 -DGMX_MPI=ON -DGMX_GPU=ON
> -DCMAKE_PREFIX_PATH=${FFTW_DIR}/..
>
> /usr/bin/ld:
> CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_
> cuda.cu.o: relocation R_X86_64_32 against
> `_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_at
> omdata10cu_nbparam8cu_plistb' can not be used when making a shared
> object; recompile with -fPIC
>
> Thanks,
> Chris
>
> -Original Message-
> From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se
> [mailto:gromacs.org_gmx-users-boun...@maillist.sys.kth.se] On Behalf
> Of Szilárd Páll
> Sent: Friday, April 06, 2018 2:40 PM
> To: Discussion list for GROMACS users <gmx-us...@gromacs.org>
> Cc: gromacs.org_gmx-users@maillist.sys.kth.se
> Subject: Re: [gmx-users] Problem with CUDA
>
> On CSCS Piz Daint I use the following command line (assuming
> PrgEnv-gnu) where everything in "[]" is optional and compilation should work
> just fine without.
>
> CC=cc CXX=CC cmake ../
> -DGMX_SIMD=THE_RIGHT_SIMD_FLAVOR
> -DGMX_MPI=ON
> -DGMX_GPU=ON
> -DCMAKE_PREFIX_PATH=${FFTW_DIR}/.. \
> [ -DGMX_FFT_LIBRARY=fftw3
> -DGMX_CUDA_TARGET_SM=60
> -DGMX_PREFER_STATIC_LIBS=ON
> -DBUILD_SHARED_LIBS=OFF
> -DGMX_BUILD_MDRUN_ONLY=ON
> -DGMX_EXTERNAL_BLAS=OFF -DGMX_EXTERNAL_LAPACK=OFF ]
>
> In fact, other than the "-DCMAKE_PREFIX_PATH=${FFTW_DIR}/.."and
> setting the right compiler wrappers, the rest is most of the time unnecessary
> for a "vanilla" build.
>
> Cheers,
> --
> Szilárd
>
>
> On Fri, Apr 6, 2018 at 8:32 PM, Borchert, Christopher B
> ERDC-RDE-ITL-MS Contractor <christopher.b.borch...@erdc.dren.mil>
> wrote:
>> You are trying to give me a hint. :) My cmake args are taken from a
>> co-worker, and the statement syntax is from the CMakeCache.txt file. On a
>> Cray you force cc/CC and all the module libraries/headers should be
>> automatically found. Strangely it didn’t find fftw without help. Regardless
>> I get the fpic error. But you've given me a path to investigate. Thanks.
>>
>> cmake ..
>> -DGMX_GPU=ON
>> -DCMAKE_C_COMPILER:FILEPATH=`which cc`
>> -DCMAKE_CXX_COMPILER:FILEPATH=`which CC`
>> -DGMX_FFT_LIBRARY=fftw3
>> -DFFTWF_LIBRARY:FILEPATH=${FFTW_DIR}/libfftw3f.so
>> -DFFTWF_INCLUDE_DIR:PATH=$FFTW_INC
>>
>> /usr/bin/ld:
>> CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn
>> _
>> cuda.cu.o: relocation R_X86_64_32 against
>> `_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_a
>> t omdata10cu_nbparam8cu_plistb' can not be used when making a shared
>> object; recompile with -fPIC
>>
>> Chris
>>
>> -Original Message-
>> From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se
>> [mailto:gromacs.org_gmx-users-boun...@maillist.sys.kth.se] On Behalf
>> Of Szilárd Páll
>> Sent: Friday, April 06, 2018 12:05 PM
>> To: Discussion list for GROMACS users <gmx-us...@gromacs.org>
>> Cc: gromacs.org_gmx-users@maillist.sys.kth.se
>> Subject: Re: [gmx-users] Problem with CUDA
>>
>> FYI: not even a my vanilla (non-CRAY) local build which does work otherwise
>> succeeds with cmake . -DCUDA_NVCC_FLAGS:STRING="-rdc=true"
>> so as I guessed, that's the culprit.
>>
>> Out of curiosity I wonder: what's the reason for the "inventive" use of
>> CMake options, none of which are needed?
>>
>> --
>> Szilárd
>>
>>
>> On Fri, Apr 6, 2018 at 6:57 PM, Szilárd Páll <pall.szil...@gmail.com> wrote:
>>> I think the fpic errors can't be caused by missing rdc=true because
>>> the latter refers to the GPU _device_ code, but GROMACS does not
>>> need relocatable device code, so that sho
Re: [gmx-users] Problem with CUDA
Unfortunately using your shortened cmake args, I still get fpic errors. But it
does complete the build statically with -DGMX_BUILD_SHARED_EXE=OFF
CC=cc CXX=CC cmake ../ -DGMX_SIMD=AVX2_256 -DGMX_MPI=ON -DGMX_GPU=ON
-DCMAKE_PREFIX_PATH=${FFTW_DIR}/..
/usr/bin/ld:
CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o:
relocation R_X86_64_32 against
`_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_atomdata10cu_nbparam8cu_plistb'
can not be used when making a shared object; recompile with -fPIC
Thanks,
Chris
-Original Message-
From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se
[mailto:gromacs.org_gmx-users-boun...@maillist.sys.kth.se] On Behalf Of Szilárd
Páll
Sent: Friday, April 06, 2018 2:40 PM
To: Discussion list for GROMACS users <gmx-us...@gromacs.org>
Cc: gromacs.org_gmx-users@maillist.sys.kth.se
Subject: Re: [gmx-users] Problem with CUDA
On CSCS Piz Daint I use the following command line (assuming
PrgEnv-gnu) where everything in "[]" is optional and compilation should work
just fine without.
CC=cc CXX=CC cmake ../
-DGMX_SIMD=THE_RIGHT_SIMD_FLAVOR
-DGMX_MPI=ON
-DGMX_GPU=ON
-DCMAKE_PREFIX_PATH=${FFTW_DIR}/.. \
[ -DGMX_FFT_LIBRARY=fftw3
-DGMX_CUDA_TARGET_SM=60
-DGMX_PREFER_STATIC_LIBS=ON
-DBUILD_SHARED_LIBS=OFF
-DGMX_BUILD_MDRUN_ONLY=ON
-DGMX_EXTERNAL_BLAS=OFF -DGMX_EXTERNAL_LAPACK=OFF ]
In fact, other than the "-DCMAKE_PREFIX_PATH=${FFTW_DIR}/.."and
setting the right compiler wrappers, the rest is most of the time unnecessary
for a "vanilla" build.
Cheers,
--
Szilárd
On Fri, Apr 6, 2018 at 8:32 PM, Borchert, Christopher B ERDC-RDE-ITL-MS
Contractor <christopher.b.borch...@erdc.dren.mil>
wrote:
> You are trying to give me a hint. :) My cmake args are taken from a
> co-worker, and the statement syntax is from the CMakeCache.txt file. On a
> Cray you force cc/CC and all the module libraries/headers should be
> automatically found. Strangely it didn’t find fftw without help. Regardless I
> get the fpic error. But you've given me a path to investigate. Thanks.
>
> cmake ..
> -DGMX_GPU=ON
> -DCMAKE_C_COMPILER:FILEPATH=`which cc`
> -DCMAKE_CXX_COMPILER:FILEPATH=`which CC`
> -DGMX_FFT_LIBRARY=fftw3
> -DFFTWF_LIBRARY:FILEPATH=${FFTW_DIR}/libfftw3f.so
> -DFFTWF_INCLUDE_DIR:PATH=$FFTW_INC
>
> /usr/bin/ld:
> CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_
> cuda.cu.o: relocation R_X86_64_32 against
> `_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_at
> omdata10cu_nbparam8cu_plistb' can not be used when making a shared
> object; recompile with -fPIC
>
> Chris
>
> -Original Message-
> From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se
> [mailto:gromacs.org_gmx-users-boun...@maillist.sys.kth.se] On Behalf
> Of Szilárd Páll
> Sent: Friday, April 06, 2018 12:05 PM
> To: Discussion list for GROMACS users <gmx-us...@gromacs.org>
> Cc: gromacs.org_gmx-users@maillist.sys.kth.se
> Subject: Re: [gmx-users] Problem with CUDA
>
> FYI: not even a my vanilla (non-CRAY) local build which does work otherwise
> succeeds with cmake . -DCUDA_NVCC_FLAGS:STRING="-rdc=true"
> so as I guessed, that's the culprit.
>
> Out of curiosity I wonder: what's the reason for the "inventive" use of CMake
> options, none of which are needed?
>
> --
> Szilárd
>
>
> On Fri, Apr 6, 2018 at 6:57 PM, Szilárd Páll <pall.szil...@gmail.com> wrote:
>> I think the fpic errors can't be caused by missing rdc=true because
>> the latter refers to the GPU _device_ code, but GROMACS does not need
>> relocatable device code, so that should not be necessary.
>> --
>> Szilárd
>>
>>
>> On Fri, Apr 6, 2018 at 6:33 PM, Borchert, Christopher B
>> ERDC-RDE-ITL-MS Contractor <christopher.b.borch...@erdc.dren.mil>
>> wrote:
>>> Thanks Szilárd. My understanding is rdc is nvcc's equivalent of fpic. I get
>>> fpic errors without it. In fact I get fpic errors without including fpic
>>> explicitly in the C/CXX flags.
>>>
>>> /usr/bin/ld:
>>> CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnx
>>> n
>>> _cuda.cu.o: relocation R_X86_64_32 against
>>> `_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_
>>> a tomdata10cu_nbparam8cu_plistb' can not be used when making a
>>> shared object; recompile with -fPIC
>>>
>>> So I removed boost, avx2, mpi, and dynamic but get the same result. What
>>> else should I remove?
>>>
>>> cmake ..
>>> -DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE
>>> -DCMAKE_C_COMPILER:FILEPATH=`which cc` -DCMAKE_C_FLAGS:STRING=-fPIC
>>> -DCMAKE_CXX_CO
Re: [gmx-users] Problem with CUDA
You are trying to give me a hint. :) My cmake args are taken from a co-worker,
and the statement syntax is from the CMakeCache.txt file. On a Cray you force
cc/CC and all the module libraries/headers should be automatically found.
Strangely it didn’t find fftw without help. Regardless I get the fpic error.
But you've given me a path to investigate. Thanks.
cmake ..
-DGMX_GPU=ON
-DCMAKE_C_COMPILER:FILEPATH=`which cc`
-DCMAKE_CXX_COMPILER:FILEPATH=`which CC`
-DGMX_FFT_LIBRARY=fftw3
-DFFTWF_LIBRARY:FILEPATH=${FFTW_DIR}/libfftw3f.so
-DFFTWF_INCLUDE_DIR:PATH=$FFTW_INC
/usr/bin/ld:
CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o:
relocation R_X86_64_32 against
`_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_atomdata10cu_nbparam8cu_plistb'
can not be used when making a shared object; recompile with -fPIC
Chris
-Original Message-
From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se
[mailto:gromacs.org_gmx-users-boun...@maillist.sys.kth.se] On Behalf Of Szilárd
Páll
Sent: Friday, April 06, 2018 12:05 PM
To: Discussion list for GROMACS users <gmx-us...@gromacs.org>
Cc: gromacs.org_gmx-users@maillist.sys.kth.se
Subject: Re: [gmx-users] Problem with CUDA
FYI: not even a my vanilla (non-CRAY) local build which does work otherwise
succeeds with cmake . -DCUDA_NVCC_FLAGS:STRING="-rdc=true"
so as I guessed, that's the culprit.
Out of curiosity I wonder: what's the reason for the "inventive" use of CMake
options, none of which are needed?
--
Szilárd
On Fri, Apr 6, 2018 at 6:57 PM, Szilárd Páll <pall.szil...@gmail.com> wrote:
> I think the fpic errors can't be caused by missing rdc=true because
> the latter refers to the GPU _device_ code, but GROMACS does not need
> relocatable device code, so that should not be necessary.
> --
> Szilárd
>
>
> On Fri, Apr 6, 2018 at 6:33 PM, Borchert, Christopher B
> ERDC-RDE-ITL-MS Contractor <christopher.b.borch...@erdc.dren.mil>
> wrote:
>> Thanks Szilárd. My understanding is rdc is nvcc's equivalent of fpic. I get
>> fpic errors without it. In fact I get fpic errors without including fpic
>> explicitly in the C/CXX flags.
>>
>> /usr/bin/ld:
>> CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn
>> _cuda.cu.o: relocation R_X86_64_32 against
>> `_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_a
>> tomdata10cu_nbparam8cu_plistb' can not be used when making a shared
>> object; recompile with -fPIC
>>
>> So I removed boost, avx2, mpi, and dynamic but get the same result. What
>> else should I remove?
>>
>> cmake ..
>> -DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE
>> -DCMAKE_C_COMPILER:FILEPATH=`which cc` -DCMAKE_C_FLAGS:STRING=-fPIC
>> -DCMAKE_CXX_COMPILER:FILEPATH=`which CC`
>> -DCMAKE_CXX_FLAGS:STRING=-fPIC -DCMAKE_INSTALL_PREFIX:PATH=$PREFIX
>> -DGMX_FFT_LIBRARY=fftw3
>> -DFFTWF_LIBRARY:FILEPATH=${FFTW_DIR}/libfftw3f.so
>> -DFFTWF_INCLUDE_DIR:PATH=$FFTW_INC
>> -DGMX_GPU=ON
>> -DCUDA_NVCC_FLAGS:STRING="-rdc=true"
>>
>> /opt/cray/pe/craype/2.5.13/bin/CC-march=core-avx2 -fPIC -std=c++11
>> -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
>> CMakeFiles/template.dir/template.cpp.o -o ../../bin/template
>> -Wl,-rpath,/p/work/borchert/gromacs-2018.1/build/lib
>> ../../lib/libgromacs.so.3.1.0 -fopenmp -lm
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_59_tmpxft_01e3__21_pmalloc_cuda_compute_61_cpp1_ii_63d60154'
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_57_tmpxft_a64f__21_pme_spread_compute_61_cpp1_ii_d982d3ad'
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_71_tmpxft_03a4__21_cuda_version_information_compute_61_cpp1_ii_8ab8dc1d'
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_58_tmpxft_a80b__21_pme_timings_compute_61_cpp1_ii_75ae0e44'
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_56_tmpxft_a10b__21_pme_3dfft_compute_61_cpp1_ii_79dff388'
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_57_tmpxft_9bd7__21_nbnxn_cuda_compute_61_cpp1_ii_f147f02c'
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_50_tmpxft_a9c8__21_pme_compute_61_cpp1_ii_6dbf966c'
>> ../../lib/libgromacs.so.3.1.0: undefined reference to
>> `__cudaRegisterLinkedBinary_56_tmpxft_a490__21_pme_solve_compute_61_cpp1_ii_06051a94'
>> ../../
Re: [gmx-users] Problem with CUDA
Thanks Szilárd. My understanding is rdc is nvcc's equivalent of fpic. I get
fpic errors without it. In fact I get fpic errors without including fpic
explicitly in the C/CXX flags.
/usr/bin/ld:
CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o:
relocation R_X86_64_32 against
`_Z58nbnxn_kernel_ElecEwQSTabTwinCut_VdwLJEwCombLB_F_prune_cuda11cu_atomdata10cu_nbparam8cu_plistb'
can not be used when making a shared object; recompile with -fPIC
So I removed boost, avx2, mpi, and dynamic but get the same result. What else
should I remove?
cmake ..
-DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE
-DCMAKE_C_COMPILER:FILEPATH=`which cc`
-DCMAKE_C_FLAGS:STRING=-fPIC
-DCMAKE_CXX_COMPILER:FILEPATH=`which CC`
-DCMAKE_CXX_FLAGS:STRING=-fPIC
-DCMAKE_INSTALL_PREFIX:PATH=$PREFIX
-DGMX_FFT_LIBRARY=fftw3
-DFFTWF_LIBRARY:FILEPATH=${FFTW_DIR}/libfftw3f.so
-DFFTWF_INCLUDE_DIR:PATH=$FFTW_INC
-DGMX_GPU=ON
-DCUDA_NVCC_FLAGS:STRING="-rdc=true"
/opt/cray/pe/craype/2.5.13/bin/CC-march=core-avx2 -fPIC -std=c++11 -O3
-DNDEBUG -funroll-all-loops -fexcess-precision=fast
CMakeFiles/template.dir/template.cpp.o -o ../../bin/template
-Wl,-rpath,/p/work/borchert/gromacs-2018.1/build/lib
../../lib/libgromacs.so.3.1.0 -fopenmp -lm
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_59_tmpxft_01e3__21_pmalloc_cuda_compute_61_cpp1_ii_63d60154'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_57_tmpxft_a64f__21_pme_spread_compute_61_cpp1_ii_d982d3ad'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_71_tmpxft_03a4__21_cuda_version_information_compute_61_cpp1_ii_8ab8dc1d'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_58_tmpxft_a80b__21_pme_timings_compute_61_cpp1_ii_75ae0e44'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_a10b__21_pme_3dfft_compute_61_cpp1_ii_79dff388'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_57_tmpxft_9bd7__21_nbnxn_cuda_compute_61_cpp1_ii_f147f02c'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_50_tmpxft_a9c8__21_pme_compute_61_cpp1_ii_6dbf966c'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_a490__21_pme_solve_compute_61_cpp1_ii_06051a94'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_ab85__21_cudautils_compute_61_cpp1_ii_25933dd5'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_54_tmpxft_aefc__21_pinning_compute_61_cpp1_ii_5d0f4aae'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_ad42__21_gpu_utils_compute_61_cpp1_ii_70828085'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_57_tmpxft_a2d0__21_pme_gather_compute_61_cpp1_ii_a7a2f9c7'
../../lib/libgromacs.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_67_tmpxft_9f4e__21_nbnxn_cuda_data_mgmt_compute_61_cpp1_ii_a1eafeba'
Chris
-Original Message-
From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se
[mailto:gromacs.org_gmx-users-boun...@maillist.sys.kth.se] On Behalf Of Szilárd
Páll
Sent: Friday, April 06, 2018 10:17 AM
To: Discussion list for GROMACS users <gmx-us...@gromacs.org>
Cc: gromacs.org_gmx-users@maillist.sys.kth.se
Subject: Re: [gmx-users] Problem with CUDA
Hi,
What is the reason for using the custom CMake options? What's the -rdc=true for
-- I don't think it's needed and it can very well be causing the issue. Have
you tried to actually do an as-vanilla-as-possible build?
--
Szilárd
On Thu, Apr 5, 2018 at 6:52 PM, Borchert, Christopher B ERDC-RDE-ITL-MS
Contractor <christopher.b.borch...@erdc.dren.mil>
wrote:
> Hello. I'm taking a working build from a co-worker and trying to add GPU
> support on a Cray XC. CMake works but make fails. Both 2016 and 2018 die at
> the same point -- can't find gromac's own routines.
>
> 2016.5:
> /opt/cray/pe/craype/2.5.13/bin/CC-march=core-avx2 -O2 -fPIC -dynamic
> -std=c++0x -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
> -dynamic CMakeFiles/template.dir/template.cpp.o -o ../../bin/template
> -Wl,-rpath,/p/work/cots/gromacs-2016.5/build/lib:/opt/nvidia/cudatoolkit8.0/8.0.54_2.3.12_g180d272-2.2/lib64/stubs
> -dynamic ../../lib/libgromacs_mpi.so.2.5.0 -fopenmp -lcudart
> /opt/nvidia/cudatoolkit8.0/8.0.54_2.3.12_g180d272-2.2/lib64/stubs/libnvidia-ml.so
> -lhwloc -lz -ldl -lrt -lm -lfftw3f
> ../../lib/libgromacs_mpi.so.2.5.0: undefined reference to
> `__cudaRegisterLinkedBinary_59_tmpxft_0001bc78__21_pmalloc_cuda_compute_61_
[gmx-users] Problem with CUDA
Hello. I'm taking a working build from a co-worker and trying to add GPU
support on a Cray XC. CMake works but make fails. Both 2016 and 2018 die at the
same point -- can't find gromac's own routines.
2016.5:
/opt/cray/pe/craype/2.5.13/bin/CC-march=core-avx2 -O2 -fPIC -dynamic
-std=c++0x -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
-dynamic CMakeFiles/template.dir/template.cpp.o -o ../../bin/template
-Wl,-rpath,/p/work/cots/gromacs-2016.5/build/lib:/opt/nvidia/cudatoolkit8.0/8.0.54_2.3.12_g180d272-2.2/lib64/stubs
-dynamic ../../lib/libgromacs_mpi.so.2.5.0 -fopenmp -lcudart
/opt/nvidia/cudatoolkit8.0/8.0.54_2.3.12_g180d272-2.2/lib64/stubs/libnvidia-ml.so
-lhwloc -lz -ldl -lrt -lm -lfftw3f
../../lib/libgromacs_mpi.so.2.5.0: undefined reference to
`__cudaRegisterLinkedBinary_59_tmpxft_0001bc78__21_pmalloc_cuda_compute_61_cpp1_ii_63d60154'
../../lib/libgromacs_mpi.so.2.5.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_0001bac2__21_gpu_utils_compute_61_cpp1_ii_d70ebee0'
../../lib/libgromacs_mpi.so.2.5.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_0001b90b__21_cudautils_compute_61_cpp1_ii_24d20763'
../../lib/libgromacs_mpi.so.2.5.0: undefined reference to
`__cudaRegisterLinkedBinary_71_tmpxft_0001c016__21_cuda_version_information_compute_61_cpp1_ii_e35285be'
../../lib/libgromacs_mpi.so.2.5.0: undefined reference to
`__cudaRegisterLinkedBinary_57_tmpxft_0001b592__21_nbnxn_cuda_compute_61_cpp1_ii_6e47f057'
../../lib/libgromacs_mpi.so.2.5.0: undefined reference to
`__cudaRegisterLinkedBinary_67_tmpxft_0001b754__21_nbnxn_cuda_data_mgmt_compute_61_cpp1_ii_a1eafeba'
collect2: error: ld returned 1 exit status
2018.1:
/opt/cray/pe/craype/2.5.13/bin/CC-march=core-avx2 -O2 -fPIC -dynamic
-std=c++11 -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
-dynamic CMakeFiles/template.dir/template.cpp.o -o ../../bin/template
-Wl,-rpath,/p/work/cots/gromacs-2018.1/build/lib
../../lib/libgromacs_mpi.so.3.1.0 -fopenmp -lm
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_68a5__21_pme_3dfft_compute_61_cpp1_ii_79dff388'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_67_tmpxft_6621__21_nbnxn_cuda_data_mgmt_compute_61_cpp1_ii_a1eafeba'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_57_tmpxft_6f47__21_pme_spread_compute_61_cpp1_ii_d982d3ad'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_6d70__21_pme_solve_compute_61_cpp1_ii_06051a94'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_59_tmpxft_8da7__21_pmalloc_cuda_compute_61_cpp1_ii_63d60154'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_50_tmpxft_7930__21_pme_compute_61_cpp1_ii_6dbf966c'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_58_tmpxft_7382__21_pme_timings_compute_61_cpp1_ii_75ae0e44'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_57_tmpxft_6b11__21_pme_gather_compute_61_cpp1_ii_a7a2f9c7'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_7f9f__21_cudautils_compute_61_cpp1_ii_25933dd5'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_54_tmpxft_88f9__21_pinning_compute_61_cpp1_ii_5d0f4aae'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_57_tmpxft_39b7__21_nbnxn_cuda_compute_61_cpp1_ii_f147f02c'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_71_tmpxft_91d4__21_cuda_version_information_compute_61_cpp1_ii_8ab8dc1d'
../../lib/libgromacs_mpi.so.3.1.0: undefined reference to
`__cudaRegisterLinkedBinary_56_tmpxft_8407__21_gpu_utils_compute_61_cpp1_ii_70828085'
collect2: error: ld returned 1 exit status
BUILD INSTRUCTIONS:
module swap PrgEnv-cray PrgEnv-gnu
module swap gcc gcc/5.3.0
export CRAYPE_LINK_TYPE=dynamic
module load cudatoolkit/8.0.54_2.3.12_g180d272-2.2
module load cmake/gcc-6.3.0/3.7.2
module load fftw/3.3.4.11
export BOOST_DIR=/app/unsupported/boost/1.64.0-gcc-6.3.0
export PREFIX=/app/unsupported/gromacs/201x.x
mkdir $PREFIX
cmake .. \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE \
-DCMAKE_C_COMPILER:FILEPATH=`which cc` \
-DCMAKE_C_FLAGS:STRING="-O2 -fPIC -dynamic" \
-DCMAKE_CXX_COMPILER:FILEPATH=`which CC` \
-DCMAKE_CXX_FLAGS:STRING="-O2 -fPIC -dynamic" \
-DCMAKE_INSTALL_PREFIX:PATH=$PREFIX \
-DGMX_FFT_LIBRARY=fftw3 \
-DCMAKE_EXE_LINKER_FLAGS:STRING=-dynamic \
-DGMX_SIMD=AVX2_256 \
-DFFTWF_LIBRARY:FILEPATH=${FFTW_DIR}/libfftw3f.so \
-DFFTWF_INCLUDE_DIR:PATH=$FFTW_INC \
