> you need to have host and device compiler either both using c++17 or neither
> using c++17.
I will create a fixup MR for this that tries to address it in configure. I had
previously come across this behavior in CI, but was never able to reproduce the
linker errors on multiple machines (summit included!).
Oddly enough the above rule isn’t even always required. Some compiler
combinations seem to work perfectly fine mixing c++11/14 with c++17… It doesn’t
scale with compiler age either, I tested linking with a compiler far older than
the one breaking in CI and it compiled and linked without a problem.
Best regards,
Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
> On Aug 30, 2021, at 13:02, Jacob Faibussowitsch <jacob....@gmail.com> wrote:
>
>> No luck with C++14
>
> TL;DR: you need to have host and device compiler either both using c++17 or
> neither using c++17.
>
> Long version:
> C++17 among other things changed how static constexpr member variables for
> classes worked. Previously if I had a class with a static constexpr member
> variable I would have to not only declare it inline within the class, but
> also define it within an executable otherwise the variable would not actually
> have any physical memory address:
>
> // foo.hpp
> class foo
> {
> static constexpr int bar = 5;
> };
>
> // foo.cpp
> int foo::bar;
>
> In c++17 however this changed because you can have static “inline” variables.
> All this does is force the compiler define the variable for you instead. The
> issue of course is that static constexpr implicitly makes that variable
> inline in c++17. So to sum it up:
>
> 1. The c++17 compiler (nvcc) sees the static constexpr variable, goes “hmm ok
> I will define this in some undefined location”.
> 2. The c++11/14 compiler comes along, sees your hand-coded definition of the
> variable and goes “ah but I think I’ve seen this before, I’ll ignore it”.
> This silent rejection is due to the hand-coded definition idiom being
> deprecated from c++17 onwards. Stupid, I know.
> 2. The linker (driven by the c++11/14 compiler since PETSc links using the
> host compiler) comes along and now suddenly cannot find the literal
> definition, because it doesn’t know what the c++17 did. Disaster!
>
> Best regards,
>
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
>
>> On Aug 30, 2021, at 10:12, Mark Adams <mfad...@lbl.gov
>> <mailto:mfad...@lbl.gov>> wrote:
>>
>> No luck with C++14
>>
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/vec/is/sf/impls/basic/cuda/sfcuda.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/vec/is/sf/impls/basic/cuda/sfcuda.o
>> CLINKER arch-summit-hypre-cuda-dbg/lib/libpetsc.so.3.015.3
>> arch-summit-hypre-cuda-dbg/obj/sys/objects/device/impls/cupm/cuda/cupmcontext.o:(.rodata._ZN5Petsc13CUPMInterfaceILNS_14CUPMDeviceKindE0EE21cupmStreamNonBlockingE[_ZN5Petsc13CUPMInterfaceILNS_14CUPMDeviceKindE0EE21cupmStreamNonBlockingE]+0x0):
>> multiple definition of
>> `Petsc::CUPMInterface<(Petsc::CUPMDeviceKind)0>::cupmStreamNonBlocking'
>> arch-summit-hypre-cuda-dbg/obj/sys/objects/device/interface/cupminterface.o:(.rodata+0x44):
>> first defined here
>> /usr/bin/ld: link errors found, deleting executable
>> `arch-summit-hypre-cuda-dbg/lib/libpetsc.so.3.015.3'
>> collect2: error: ld returned 1 exit status
>> gmake[3]: *** [gmakefile:113:
>> arch-summit-hypre-cuda-dbg/lib/libpetsc.so.3.015.3] Error 1
>> gmake[2]: ***
>> [/gpfs/alpine/csc314/scratch/adams/petsc2/lib/petsc/conf/rules:50: libs]
>> Error 2
>> **************************ERROR*************************************
>> Error during compile, check
>> arch-summit-hypre-cuda-dbg/lib/petsc/conf/make.log
>> Send it and arch-summit-hypre-cuda-dbg/lib/petsc/conf/configure.log to
>> petsc-ma...@mcs.anl.gov <mailto:petsc-ma...@mcs.anl.gov>
>> ********************************************************************
>> gmake[1]: *** [makefile:40: all] Error 1
>>
>> On Mon, Aug 30, 2021 at 10:50 AM Mark Adams <mfad...@lbl.gov
>> <mailto:mfad...@lbl.gov>> wrote:
>> Stefano suggested C++14 in configure. I was using C++11.
>>
>> On Mon, Aug 30, 2021 at 10:46 AM Junchao Zhang <junchao.zh...@gmail.com
>> <mailto:junchao.zh...@gmail.com>> wrote:
>> Petsc::CUPMInterface
>> @Jacob Faibussowitsch <mailto:jacob....@gmail.com>
>> --Junchao Zhang
>>
>>
>> On Mon, Aug 30, 2021 at 9:35 AM Mark Adams <mfad...@lbl.gov
>> <mailto:mfad...@lbl.gov>> wrote:
>> I was running fine this AM and am bouncing between modules to help two apps
>> (ECP milestone season) at the same time and something broke. I did update
>> main and I get the same error in main and a hypre branch of Stefano's.
>> I started with a clean build and checked my modules...
>> Any ideas?
>>
>> Thanks,
>> Mark
>>
>> CC arch-summit-hypre-cuda-dbg/obj/tao/interface/taosolver.o
>> CC arch-summit-hypre-cuda-dbg/obj/ts/interface/ts.o
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/dense/seq/cuda/densecuda.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/dense/seq/cuda/densecuda.o
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/aij/seq/seqcusparse/aijcusparseband.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/aij/seq/seqcusparse/aijcusparseband.o
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/ts/utils/dmplexlandau/cuda/landaucu.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/ts/utils/dmplexlandau/cuda/landaucu.o
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/vec/vec/impls/seq/seqcuda/veccuda2.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/vec/vec/impls/seq/seqcuda/veccuda2.o
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/aij/mpi/mpicusparse/mpiaijcusparse.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/aij/mpi/mpicusparse/mpiaijcusparse.o
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/aij/seq/seqcusparse/aijcusparse.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/mat/impls/aij/seq/seqcusparse/aijcusparse.o
>> CUDAC
>> arch-summit-hypre-cuda-dbg/obj/vec/is/sf/impls/basic/cuda/sfcuda.o
>> CUDAC.dep
>> arch-summit-hypre-cuda-dbg/obj/vec/is/sf/impls/basic/cuda/sfcuda.o
>> CLINKER arch-summit-hypre-cuda-dbg/lib/libpetsc.so.3.015.3
>> arch-summit-hypre-cuda-dbg/obj/sys/objects/device/impls/cupm/cuda/cupmcontext.o:(.rodata._ZN5Petsc13CUPMInterfaceILNS_14CUPMDeviceKindE0EE21cupmStreamNonBlockingE[_ZN5Petsc13CUPMInterfaceILNS_14CUPMDeviceKindE0EE21cupmStreamNonBlockingE]+0x0):
>> multiple definition of
>> `Petsc::CUPMInterface<(Petsc::CUPMDeviceKind)0>::cupmStreamNonBlocking'
>> arch-summit-hypre-cuda-dbg/obj/sys/objects/device/interface/cupminterface.o:(.rodata+0x44):
>> first defined here
>> /usr/bin/ld: link errors found, deleting executable
>> `arch-summit-hypre-cuda-dbg/lib/libpetsc.so.3.015.3'
>> collect2: error: ld returned 1 exit status
>> gmake[3]: *** [gmakefile:113:
>> arch-summit-hypre-cuda-dbg/lib/libpetsc.so.3.015.3] Error 1
>> gmake[2]: ***
>> [/gpfs/alpine/csc314/scratch/adams/petsc2/lib/petsc/conf/rules:50: libs]
>> Error 2
>> **************************ERROR*************************************
>> Error during compile, check
>> arch-summit-hypre-cuda-dbg/lib/petsc/conf/make.log
>> Send it and arch-summit-hypre-cuda-dbg/lib/petsc/conf/configure.log to
>> petsc-ma...@mcs.anl.gov <mailto:petsc-ma...@mcs.anl.gov>
>> ********************************************************************
>> gmake[1]: *** [makefile:40: all] Error 1
>> make: *** [GNUmakefile:9: all] Error 2
>
