Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-10 Thread Junchao Zhang
On Mon, Oct 10, 2022 at 8:13 AM Rob Kudyba wrote: > OK, let's walk back and don't use -DCMAKE_C_COMPILER=/path/to/mpicc >> > Will do > > >> libompitrace.so.40.30.0 is not the OpenMP library; it is the tracing >> library for OpenMPI, https://github.com/open-mpi/ompi/issues/10036 >> > Does that

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-10 Thread Rob Kudyba
OK so I missed the OpenMP vs OpenMPI with incorrectly setting -DOpenMP_libomp_LIBRARY="/path/to/openmpi-4.1.1_ucx_cuda_11.0.3_support/lib//libompitrace.so.40.30.0 So I changed it to point to /cm/local/apps/gcc/10.2.0/lib/libgomp.so.1.0.0 -- Found PETSc 3.18.0 CMake Error at

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-10 Thread Rob Kudyba
> > OK, let's walk back and don't use -DCMAKE_C_COMPILER=/path/to/mpicc > Will do > libompitrace.so.40.30.0 is not the OpenMP library; it is the tracing > library for OpenMPI, https://github.com/open-mpi/ompi/issues/10036 > Does that mean I should remove this option in the cmake command? > In

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-09 Thread Junchao Zhang
OK, let's walk back and don't use -DCMAKE_C_COMPILER=/path/to/mpicc libompitrace.so.40.30.0 is not the OpenMP library; it is the tracing library for OpenMPI, https://github.com/open-mpi/ompi/issues/10036 In your previous email, there was /path/to/cmake/cmake-3.22.1-linux-x86_64/bin/cmake -E

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-09 Thread Rob Kudyba
I did have -DMPI_CXX_COMPILER set, so I added -DCMAKE_C_COMPILER and now get these errors: [ 25%] Linking CXX shared library librichdem.so /lib/../lib64/crt1.o: In function `_start': (.text+0x24): undefined reference to `main' CMakeFiles/richdem.dir/src/random.cpp.o: In function

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-09 Thread Junchao Zhang
In the last link step to generate the executable /cm/local/apps/gcc/10.2.0/bin/c++ -isystem -O3 -g -Wall -Wextra -pedantic -Wshadow CMakeFiles/wtm.x.dir/src/WTM.cpp.o -o wtm.x -Wl,-rpath,/path/to/WTM/build/common/richdem:/path/to/ gdal-3.3.0/lib:/path/to/openmpi-4.1.1_ucx_cuda_11.0.3_

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-08 Thread Rob Kudyba
> > Perhaps we can back one step: > Use your mpicc to build a "hello world" mpi test, then run it on a compute > node (with GPU) to see if it works. > If no, then your MPI environment has problems; > If yes, then use it to build petsc (turn on petsc's gpu support, > --with-cuda

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-08 Thread Barry Smith
True, but when users send reports back to us they will never have used the VERBOSE=1 option, so it requires one more round trip of email to get this additional information. > On Oct 8, 2022, at 6:48 PM, Jed Brown wrote: > > Barry Smith writes: > >> I hate these kinds of make rules

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-08 Thread Jed Brown
Barry Smith writes: >I hate these kinds of make rules that hide what the compiler is doing (in > the name of having less output, I guess) it makes it difficult to figure out > what is going wrong. You can make VERBOSE=1 with CMake-generated makefiles.

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-08 Thread Junchao Zhang
Perhaps we can back one step: Use your mpicc to build a "hello world" mpi test, then run it on a compute node (with GPU) to see if it works. If no, then your MPI environment has problems; If yes, then use it to build petsc (turn on petsc's gpu support, --with-cuda --with-cudac=nvcc), and then

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-08 Thread Barry Smith
I hate these kinds of make rules that hide what the compiler is doing (in the name of having less output, I guess) it makes it difficult to figure out what is going wrong. Anyways, either some of the MPI libraries are missing from the link line or they are in the wrong order and thus it

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-07 Thread Rob Kudyba
The error changes now and at an earlier place, 66% vs 70%: make LDFLAGS="-Wl,--copy-dt-needed-entries" Consolidate compiler generated dependencies of target fmt [ 12%] Built target fmt Consolidate compiler generated dependencies of target richdem [ 37%] Built target richdem Consolidate compiler

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-07 Thread Rob Kudyba
> Thanks for the quick reply. I added these options to make and make check >> still produce the warnings so I used the command like this: >> make PETSC_DIR=/path/to/petsc PETSC_ARCH=arch-linux-c-debug >> MPIEXEC="mpiexec -mca orte_base_help_aggregate 0 --mca >> opal_warn_on_missing_libcuda 0 -mca

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-07 Thread Junchao Zhang
On Fri, Oct 7, 2022 at 1:08 PM Rob Kudyba wrote: > Thanks for the quick reply. I added these options to make and make check > still produce the warnings so I used the command like this: > make PETSC_DIR=/path/to/petsc PETSC_ARCH=arch-linux-c-debug > MPIEXEC="mpiexec -mca

Re: [petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-07 Thread Rob Kudyba
Thanks for the quick reply. I added these options to make and make check still produce the warnings so I used the command like this: make PETSC_DIR=/path/to/petsc PETSC_ARCH=arch-linux-c-debug MPIEXEC="mpiexec -mca orte_base_help_aggregate 0 --mca opal_warn_on_missing_libcuda 0 -mca pml ucx --mca

[petsc-users] suppress CUDA warning & choose MCA parameter for mpirun during make PETSC_ARCH=arch-linux-c-debug check

2022-10-07 Thread Rob Kudyba
We are on RHEL 8, using modules that we can load/unload various version of packages/libraries, and I have OpenMPI 4.1.1 with CUDA aware loaded along with GDAL 3.3.0, GCC 10.2.0, and cmake 3.22.1 make PETSC_DIR=/path/to/petsc PETSC_ARCH=arch-linux-c-debug check fails with the below errors, Running