On Mon, Nov 30, 2015 at 5:20 PM, Andrey Ovsyannikov <aovsyanni...@lbl.gov> wrote:
> Dear PETSc team, > > I am working on optimization of Chombo-Crunch CFD code for next-generation > supercomputer architectures at NERSC (Berkeley Lab) and we use PETSc AMG > solver. During memory analysis study I faced with a difficulty to get > memory usage data from PETSc for all MPI ranks. I am looking for memory > dump function to get a detailed information on memory usage (not only > resident size and virtual memory but allso allocation by Vec, Mat, etc). > There is PetscMallocDumpLog() function but it is a collective function and > it always provides a log for 0 rank. I am wondering if it is possible to > include in PETSc a modification of PetscMallocDumpLog() which dumps the > similar log but for all MPI ranks. > > I am attaching an example of my own memory function which uses PETSc > non-collective functions and it provides a resident set size and virtual > memory for all ranks. Perhaps in a similar way it is possible to modify > PetscMallocDumpLog. > You could walk the heap if you use the debugging malloc infrastructure in PETSc. However, I would really recommend trying out Massif from the valgrind toolset. Its designed for this and really nice. Thanks, Matt > Thank you, > > void petscMemoryLog(const char prefix[]) > { > FILE* fd; > char fname[PETSC_MAX_PATH_LEN]; > PetscMPIInt rank; > > MPI_Comm_rank(Chombo_MPI::comm,&rank); > > PetscLogDouble allocated; > PetscLogDouble resident; > PetscMallocGetCurrentUsage(&allocated); > PetscMemoryGetCurrentUsage(&resident); > PetscSNPrintf(fname,sizeof(fname),"%s.%d",prefix,rank); > PetscFOpen(PETSC_COMM_SELF,fname,"a",&fd); > > PetscFPrintf(PETSC_COMM_SELF,fd,"### PETSc memory footprint for rank %d > \n",rank); > PetscFPrintf(PETSC_COMM_SELF,fd,"[%d] Memory allocated by PetscMalloc() > %.0f bytes\n",rank,allocated); > PetscFPrintf(PETSC_COMM_SELF,fd,"[%d] RSS usage by entire process %.0f > KB\n",rank,resident); > PetscFClose(PETSC_COMM_SELF,fd); > } > > Best regards, > Andrey Ovsyannikov, Ph.D. > Postdoctoral Fellow > NERSC Division > Lawrence Berkeley National Laboratory > 510-486-7880 > aovsyanni...@lbl.gov > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener