Sometimes when we do not have exact counts, we need to overestimate sizes. This is especially true in sparse MatMat.
Matt On Mon, Nov 16, 2015 at 9:38 AM, Eric Chamberland < [email protected]> wrote: > Hi, > > we just tried this morning to solve a 1.7 billion dofs problem (on 1060 > processors task) . > > Unfortunately, petsc-3.5.3 didn't succeed within a MatMatMult product with > the following backtrace: > > [0]PETSC ERROR: Out of memory. This could be due to allocating > [0]PETSC ERROR: too large an object or bleeding by not properly > [0]PETSC ERROR: destroying unneeded objects. > [0]PETSC ERROR: Memory allocated 0 Memory used by process 4126371840 > [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info. > [0]PETSC ERROR: Memory requested 18446744070461249536 > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.5.3, Jan, 31, 2015 > [0]PETSC ERROR: /rap/jsf-051-aa/ericc/GIREF/bin/probGD.opt on a > linux-gnu-intel named r101-n31 by > [0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu-intel CFLAGS="-O3 > -xHost -mkl -fPIC -m64 -n > [0]PETSC ERROR: #1 PetscMallocAlign() line 46 in > /software6/src/petsc-3.5.3/src/sys/memory/mal.c > [0]PETSC ERROR: #2 PetscLLCondensedCreate_Scalable() line 1327 in > /software6/src/petsc-3.5.3/inclu > [0]PETSC ERROR: #3 MatMatMultSymbolic_MPIAIJ_MPIAIJ() line 741 in > /software6/src/petsc-3.5.3/src/m > [0]PETSC ERROR: #4 MatMatMult_MPIAIJ_MPIAIJ() line 33 in > /software6/src/petsc-3.5.3/src/mat/impls/ > [0]PETSC ERROR: #5 MatMatMult() line 8713 in > /software6/src/petsc-3.5.3/src/mat/interface/matrix.c > > The amount of memory requested is, apparently, an overflow of a PetscInt > which is a 32 bit signed int. > > Bad user I am, I didn't expected that to solve a problem with less than > 2^31 unknowns, it could be mandatory to compile petsc with 64bit indices... > > Is compiling with "--with-64-bit-indices" the only solution to my problem? > > Is it a known limitation/bug with a "Petsc-32bit-indices"? > > Thanks, > > Eric > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
