Il giorno ven 5 gen 2024 alle ore 16:21 Junchao Zhang < [email protected]> ha scritto:
> > > > On Fri, Jan 5, 2024 at 4:40 AM Edoardo Centofanti < > [email protected]> wrote: > >> Dear all, >> >> I have a code running both on GPU and CPU. This code has both cuda >> kernels and calls to PETSc KSP and related PC. In particular, I am trying >> to perform tests with Hypre BoomerAMG both on CPU and GPU. In order to do >> that, on CPU I am running the code with -mat_type mpiaij and -vec_type mpi, >> while on GPU I am using respectively aijcusparse and cuda. >> >> The configuration ran for PETSc (version is 3.20) is >> ./configure PETSC_ARCH=arch-linux-cuda --with-cuda --download-mumps >> --download-hypre --with-debugging=0 --download-scalapack >> --download-parmetis --download-metis --download-fblaslapack=1 >> --download-mpich --download-make --download-cmake >> >> My problem is that when I try to run my code on GPU it works well, while >> on CPU with mat_type mpiaij and -vec_type mpi it works regularly until the >> call to Hypre, then freezes (I have to kill it myself), while with GAMG it >> works on CPU with the same configuration (and the same code, just PC is >> changed). >> > Probably because if hypre is configured with GPU support, even petsc is > running on CPU (with -mat_type mpiaij and -vec_type mpi), hypre is still > trying to run on GPU, causing the hanging problem. The petsc/hypre > interface is not able to support both CPU/GPU with the same build. > > >> On another machine running PETSc version 3.17 everything worked smoothly >> with the same code and the same configuration, also on Hypre. >> > I guess with petsc-3.17, you also used a different hypre version > (downloaded by petsc). > Yes, this is exactly what I concluded after a bit of debugging and recompiling. In the new version it seems that CPU/GPU with the same build is not supported anymore. Since the CUDA kernels I have in my code do not interfere with Hypre, the only workaround I found was removing kernels from my code and performing an only-CPU run. It slows down a bit overall, but nothing dramatic. Anyway, in another test with version 3.18 I encountered the same problem, but the program crashes and throws "Invalid options" error, while in 3.20 it just hangs. Thanks, Edoardo > > > >> Do you have any insights on what is happening? >> >> Best regards, >> Edoardo >> >> >
