Hello PETSc developers I'm doing some scaling benchmarks and I found that the parallel asm preconditioner, my favorite preconditioner, has a limit in the number of cores it can handle.
I am doing a numerical experiment where I scale up the size of my matrix by roughly the same factor as the number of CPUs employed. When I look at which function used how much memory using PETSc's routine PetscMallocDumpLog, I see the following: Function name N=300 N=600 increase ====================================================================== MatGetSubMatrices_MPIAIJ_Local 75'912'016 134'516'928 1.77 MatIncreaseOverlap_MPIAIJ_Once 168'288'288 346'870'832 2.06 MatIncreaseOverlap_MPIAIJ_Receive 2'918'960 5'658'160 1.94 The matrix sizes are 6'899'904 and 14'224'896, respectively. Above N~5000 CPUs I am running out of memory. Here's my question now: Is the asm preconditioner limited from the algorithm point of view, or is it the implementation? I thought that 'only' the local matrices, plus some constant overlap with neighbors, are solved, so that memory consumption should stay constant when I scale up with a constant number of rows per process. Best Sebastian
