> On Apr 5, 2024, at 4:04 PM, Frank Bramkamp <[email protected]> wrote: > > Hi Barry, > > The problem is that libnvJitlink.so.12 also appears in the libraries in > petsclib.so (when I check with ldd). > But there, libnvJitlink.no > <https://urldefense.us/v3/__http://libnvjitlink.no/__;!!G_uCfscf7eWS!bS26tKsQ1ISdECsLypFMo2kft9mzG57JQkvudkUd5J9ja4oPFPbS7nunTNVXArezMHasPwVhH7BSoPfWAEckhs8$ > >.12 is stated as not found. And then I cannot execute my program, since this > Library is not found. We typically do not want to add LD_LIBRARY_PATH.
Ok, it is a bit confusing. Contents /software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64: ['libnppif.so.12', 'libcudart.so.12', 'libnppitc.so.12.1.1.14', 'libcuinj64.so.12.2', 'libnppc.so.12.1.1.14', 'libcuinj64.so.12.2.60', 'libnppim.so.12.1.1.14', 'libnppig.so', 'libnvjpeg.so.12.1.1.14', 'libnppidei.so.12.1.1.14', 'libnppidei.so.12', 'libnppial.so.12.1.1.14', 'libnppisu_static.a', 'libcudadevrt.a', 'libnppim.so', 'libnvToolsExt.so.1.0.0', 'libnpps.so.12.1.1.14', 'libaccinj64.so.12.2', 'libnppidei_static.a', 'libnvJitLink_static.a', 'libnppif.so', 'libnppial.so', 'libnvrtc-builtins.so', 'libcufilt.a', 'libnppig.so.12.1.1.14', 'libnvjpeg.so', 'libnvjpeg.so.12', 'libnppial_static.a', 'libnppicc_static.a', 'libnppidei.so', 'libnppist.so.12', 'cmake', 'libOpenCL.so.1.0', 'libnppisu.so', 'libnvrtc-builtins.so.12.2.91', 'libnpps.so', 'libnvrtc.so.12', 'libnppim_static.a', 'libcudart_static.a', 'libcuinj64.so', 'libnppicc.so.12.1.1.14', 'libOpenCL.so', 'libnvToolsExt.so', 'libnppig_static.a', 'libOpenCL.so.1.0.0', 'libnppc.so', 'libnppif_static.a', 'libaccinj64.so', 'libnvrtc_static.a', 'stubs', 'libnvJitLink.so.12.2.91', 'libnvrtc.so', 'libnvrtc-builtins_static.a', 'libnppig.so.12', 'libnppist.so', 'libnppif.so.12.1.1.14', 'libnppicc.so.12', 'libnppitc_static.a', 'libnppial.so.12', 'libnppim.so.12', 'libnppitc.so', 'libnpps.so.12', 'libnppist_static.a', 'libnpps_static.a', 'libnppist.so.12.1.1.14', 'libnppc.so.12', 'libcudart.so.12.2.53', 'libnvrtc-builtins.so.12.2', 'libnppisu.so.12.1.1.14', 'libculibos.a', 'libnvJitLink.so', 'libnvptxcompiler_static.a', 'libaccinj64.so.12.2.60', 'libnppitc.so.12', 'libnppisu.so.12', 'libOpenCL.so.1', 'libnvToolsExt.so.1', 'libnvrtc.so.12.2.91', 'libnppicc.so', 'libnppc_static.a', 'libcudart.so', 'libnvJitLink.so.12', 'libnvjpeg_static.a'] Executing: mpicc -o /tmp/petsc-h7tpd5_s/config.packages.BlasLapack/conftest -KPIC -O2 -g /tmp/petsc-h7tpd5_s/config.packages.BlasLapack/conftest.o -Wl,-rpath,/proj/nsc/users/bramkamp/petsc_install/petsc_barry_fix_nvclib_with_cuda_fixed_module/lib -L/proj/nsc/users/bramkamp/petsc_install/petsc_barry_fix_nvclib_with_cuda_fixed_module/lib -lflapack -Wl,-rpath,/proj/nsc/users/bramkamp/petsc_install/petsc_barry_fix_nvclib_with_cuda_fixed_module/lib -L/proj/nsc/users/bramkamp/petsc_install/petsc_barry_fix_nvclib_with_cuda_fixed_module/lib -lfblas -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64 -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64 -lcudart -lnvToolsExt -lcufft -lcublas -lcusparse -lcusolver -lcurand -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64/stubs -lcuda -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/mpi/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/mpi/lib -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/mpi/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/FFTW/3.3.10/nv23.7/hpc1/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/FFTW/3.3.10/nv23.7/hpc1/lib -L/software/sse2/tetralith_el9/manual/FFTW/3.3.10/nv23.7/hpc1/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/nvshmem/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/nvshmem/lib -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/nvshmem/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/nccl/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/nccl/lib -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/nccl/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/math_libs/lib64 -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/math_libs/lib64 -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/math_libs/lib64 -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/compilers/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/compilers/lib -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/compilers/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/compilers/extras/qd/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/compilers/extras/qd/lib -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/compilers/extras/qd/lib -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/extras/CUPTI/lib64 -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/extras/CUPTI/lib64 -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/extras/CUPTI/lib64 -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64 -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64 -L/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64 -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/11 -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/11 -L/usr/lib/gcc/x86_64-redhat-linux/11 -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/comm_libs/mpi/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/compilers/lib -lnvf -lnvomp -ldl -lnvhpcatm -latomic -lpthread -lnvcpumath -lnsnvc -lrt -lgcc_s -lm -lquadmath We see that libnvJitLink.so.12 is in /software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64 Then whenit links the executable (above) it passes -Wl,-rpath,/software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64 to the linker so that at run time, it should be able to find all the libraries in /software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64. Is /software/sse2/tetralith_el9/manual/nvhpc/23.7/Linux_x86_64/23.7/cuda/12.2/lib64/libnvJitLink.so.12 a link to something that actually exists. > > At the moment I am not sure where it tries to find that library. Therefore I > thought that maybe the problem > is that BlasLapack could put some path in the library, which does not exist. > At the beginning of configure.log > it mentions libnvJitlink.so.12, but then it seems to get lost somewhere. > > I have to see again if there is already a problem when I make petsc check, or > if it is just in my program later. > Not quite sure anymore. > > > I will write back next week, Frank > > > > > >> On 5 Apr 2024, at 19:47, Barry Smith <[email protected] >> <mailto:[email protected]>> wrote: >> >> >> Thanks for the configure.log Send the configure.log for the failed >> nvJitlink problem. >> >> >>> On Apr 5, 2024, at 12:58 PM, Frank Bramkamp <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi Barry, >>> >>> Here comes the latest configure.log file >>> >>> My cuda nvJitlink problem unfortunately still exists. >>> I will try it on a different cluster to see if this a specific problem of >>> the actual nvhpc installation. >>> >>> >>> Have a nice weekend, Frank >>> >>> >>> >>> <configure.log> >> >
