Jorge, I found multiple problems in petsc with your example. I have a fix at https://gitlab.com/petsc/petsc/-/merge_requests/2575 If everything goes well, it will be in maint and master in a few days. Thanks for reporting the problem. --Junchao Zhang
On Fri, Feb 28, 2020 at 12:13 PM Junchao Zhang <[email protected]> wrote: > I will take a look at it and get back to you. Thanks. > > On Fri, Feb 28, 2020, 7:29 AM jordic <[email protected]> wrote: > >> Dear all, >> >> the following simple program: >> >> >> ////////////////////////////////////////////////////////////////////////////////////// >> >> #include <petscmat.h> >> >> PetscInt ierr=0; >> int main(int argc,char **argv) >> { >> MPI_Comm comm; >> PetscMPIInt rank,size; >> >> PetscInitialize(&argc,&argv,NULL,help);if (ierr) return ierr; >> comm = PETSC_COMM_WORLD; >> MPI_Comm_rank(comm,&rank); >> MPI_Comm_size(comm,&size); >> >> Mat A; >> MatCreate(comm, &A); >> MatSetSizes(A, 1, 1, PETSC_DETERMINE, PETSC_DETERMINE); >> MatSetFromOptions(A); >> PetscInt dnz=1, onz=0; >> MatMPIAIJSetPreallocation(A, 0, &dnz, 0, &onz); >> MatSetOption(A, MAT_NO_OFF_PROC_ENTRIES, PETSC_TRUE); >> MatSetOption(A, MAT_IGNORE_ZERO_ENTRIES, PETSC_TRUE); >> PetscInt igid=rank, jgid=rank; >> PetscScalar value=rank+1.0; >> >> // for(int i=0; i<10; ++i) >> for(;;) //infinite loop >> { >> MatSetValue(A, igid, jgid, value, INSERT_VALUES); >> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); >> MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); >> } >> MatDestroy(&A); >> PetscFinalize(); >> return ierr; >> } >> >> >> ////////////////////////////////////////////////////////////////////////////////////// >> >> creates a simple diagonal matrix with one value per mpi-core. If the type >> of the matrix is "mpiaij" (-mat_type mpiaij) there is no problem but >> with "mpiaijcusparse" (-mat_type mpiaijcusparse) the memory usage at the >> GPU grows with every iteration of the infinite loop. The only solution that >> I found is to destroy and create the matrix every time that it needs to be >> updated. Is there a better way to avoid this problem? >> >> I am using Petsc Release Version 3.12.2 with this configure options: >> >> Configure options --package-prefix-hash=/home_nobck/user/petsc-hash-pkgs >> --with-debugging=0 --with-fc=0 CC=gcc CXX=g++ --COPTFLAGS="-g -O3" >> --CXXOPTFLAGS="-g -O3" --CUDAOPTFLAGS="-D_FORCE_INLINES -g -O3" >> --with-mpi-include=/usr/lib/openmpi/include >> --with-mpi-lib="-L/usr/lib/openmpi/lib -lmpi_cxx -lmpi" --with-cuda=1 >> --with-precision=double --with-cuda-include=/usr/include >> --with-cuda-lib="-L/usr/lib/x86_64-linux-gnu -lcuda -lcudart -lcublas >> -lcufft -lcusparse -lcusolver" PETSC_ARCH=arch-ci-linux-opt-cxx-cuda-double >> >> Thanks for your help, >> >> Jorge >> >
