CUPM is half baked, and only supports Vec and MATDENSE. --Junchao Zhang
On Mon, Aug 4, 2025 at 7:31 PM Barry Smith <bsm...@petsc.dev> wrote: > > I thought the CUPM code was intended to allow us to have common code > between NVIDIA and AMD? > > > On Jul 31, 2025, at 11:05 AM, Junchao Zhang <junchao.zh...@gmail.com> > wrote: > > What would embarrass me more is to copy the same code to MatGetDiagonal_ > SeqAIJHIPSPARSE. > > --Junchao Zhang > > On Wed, Jul 30, 2025 at 1:34 PM Barry Smith <bsm...@petsc.dev> wrote: > >> >> We absolutely should have a MatGetDiagonal_SeqAIJCUSPARSE(). >> It's somewhat embarrassing that we don't provide this. >> >> I have found some potential code at >> https://urldefense.us/v3/__https://stackoverflow.com/questions/60311408/how-to-get-the-diagonal-of-a-sparse-matrix-in-cusparse__;!!G_uCfscf7eWS!azrBQb7GZ-x-aytrnryDJoGQom1TGFZk4Q0BN-DKMOXzaNODwCt5or8ssxWJZb9IxgnQ-uKCTdvZvBX0xN2oESk0qS74$ >> >> >> Barry >> >> >> >> >> On Jul 28, 2025, at 11:43 AM, Junchao Zhang <junchao.zh...@gmail.com> >> wrote: >> >> Yes, MatGetDiagonal_SeqAIJCUSPARSE hasn't been implemented. petsc/cuda >> and petsc/kokkos backends are separate code. >> If petsc/kokkos meet your needs, then just use them. For petsc users, we >> hope it will be just a difference of extra --download-kokkos >> --download-kokkos-kernels in configuration. >> >> --Junchao Zhang >> >> >> On Mon, Jul 28, 2025 at 2:51 AM LEDAC Pierre <pierre.le...@cea.fr> wrote: >> >>> Hello all, >>> >>> >>> We are solving with PETSc a linear system updated every time step >>> (constant stencil but coefficients changing). >>> >>> >>> The matrix is preallocated once with MatSetPreallocationCOO() then >>> filled each time step with MatSetValuesCOO() and we use device pointers >>> for coo_i, coo_j, and coefficients values. >>> >>> >>> It is working fine with a GMRES Ksp solver and PC Jacobi but we are >>> surprised to see that every time step, during PCSetUp, >>> MatGetDiagonal_SeqAIJ is called whereas the matrix is on the device. >>> Looking at the API, it seems there is no MatGetDiagonal_SeqAIJCUSPARSE() >>> but a MatGetDiagonal_SeqAIJKOKKOS(). >>> >>> >>> Does it mean we should use Kokkos backend in PETSc to have Jacobi >>> preconditioner built directly on device ? Or I am doing something wrong ? >>> >>> NB: Gmres is running well on device. >>> >>> >>> I could use -ksp_reuse_preconditioner to avoid Jacobi being recreated >>> each solve on host but it increases significantly the number of iterations. >>> >>> >>> Thanks, >>> >>> >>> <pastedImage.png> >>> >>> >>> >>> Pierre LEDAC >>> Commissariat à l’énergie atomique et aux énergies alternatives >>> Centre de SACLAY >>> DES/ISAS/DM2S/SGLS/LCAN >>> Bâtiment 451 – point courrier n°41 >>> F-91191 Gif-sur-Yvette >>> +33 1 69 08 04 03 >>> +33 6 83 42 05 79 >>> >> >> >
