> 1) Could you be calling MatGetInfo() with the wrong matrix? Nice try, but I only have one matrix ;-) I call it A. We get along.
> 2) Can you use -info to check the numbers? > 3) Can you try MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, 1)? Yes and yes. In fact, I even wrote a small test app just for this purpose which is revealing some interesting behavior. Results: Additional memory is being malloced; however, MatGetInfo does not appear to be reporting this. Here I've grepped the output for "malloc": $ rjq ./make -info | grep malloc [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. [1] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 [0] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 1 [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 [1] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 1 [ 1] mallocs 0 memory 64 [ 0] mallocs 0 memory 64 The initial messages are from -info, and the last two are from my MatGetInfo snippet. Clearly two mallocs are happening that are not being reported. Maybe this is reason: those mallocs don't happen in MatAssembly_MPIAIJ? The reason for this is that each processor only computes the rows it's reponsible for. Thoughts? -Andrew On Mon, Jun 30, 2008 at 1:38 PM, Matthew Knepley <knepley at gmail.com> wrote: > 1) Could you be calling MatGetInfo() with the wrong matrix? > > 2) Can you use -info to check the numbers? > > 3) Can you try MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, 1)? > > Matt > > On Mon, Jun 30, 2008 at 12:25 PM, Andrew Colombi <acolombi at gmail.com> > wrote: >> I am a little confused by some behavior. >> >> I've been trying to understand some poor performance numbers and I >> noticed that if I doubled the preallocation numbers (presently I'm >> setting d_nz and o_nz to constants, no effort has been made to get the >> numbers _right_) performance improved dramatically (about 20X). >> >> This isn't so surprising except for the report from MatGetInfo. Even >> with the lower preallocation numbers I'm getting 0 in info.mallocs for >> >> MatInfo info; >> MatGetInfo(pc_A, MAT_LOCAL, &info); // done and reported on each >> machine individually >> >> So, my question is.... well.. why? If I'm seeing 0 mallocs before >> doubling prealloation shouldn't that mean I've preallocated enough? >> Or are their some switches I need to use to enable malloc counting? >> Also you can see (according to the same call to MetGetInfo) I'm >> wasting a lot of memory: >> >> // after doubling preallocation >> nz_alloc 6.2704e+07 nz_used 1.9125e+07 nz_unneed 4.3579e+07 >> >> (these are print outs of info.nz_allocated, info.nz_used and >> info.nzunneeded). >> >> Any thoughts? For now memory is not a bottleneck, so I guess I'll be >> satisfied with guessing big numbers for d_nz and o_nz. Still, I spent >> a lot of time scratching my head since guessing higher numbers didn't >> seem likely to have an effect. >> >> Thanks, >> -Andrew > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > >
