Re: [petsc-dev] Errors in MatSetValuesBlockedLocal() when using AIJ, but not BAIJ!

[Smith, Barry F. via petsc-dev]

> On Feb 20, 2019, at 12:50 PM, Mills, Richard Tran via petsc-dev 
> <petsc-dev@mcs.anl.gov> wrote:
> 
> Folks,
> 
> I'm working with some PFLOTRAN examples, and I've been scratching my head for 
> a while at errors that are being generated in MatSetValuesBlockedLocal(), 
> related to new nonzeros requiring a malloc. E.g.,
> 
> [0]PETSC ERROR: Argument out of range
> [0]PETSC ERROR: New nonzero at (2,0) caused a malloc
> 
> The strange thing is that all of these examples run fine when I use the 
> PFLOTRAN default BAIJ matrix type (with blocks column-oriented) -- I only see 
> the issue when I switch to using AIJ. If I set the matrix option 
> MAT_NEW_NONZERO_ALLOCATION_ERR to PETSC_FALSE to allow things to run with 
> AIJ, everything seems to work perfectly. (And once the Jacobian matrix is 
> constructed the first time, the memory usage is not growing according to what 
> I see from -malloc_log.)
> 
> I've spent some time poking around with a debugger in the variants of 
> MatSetValues() and MatXAIJSetPreallocation(), and I haven't been able make 
> much progress in determining why I'm seeing this in AIJ vs. BAIJ -- it's easy 
> to get confused looking at this code, and it's going to take me more time to 
> follow what is going on than I've had to spare so far. So, I've got a few 
> questions. First, should it even be possible to see what I am seeing? That 
> is, if the MatSetValuesBlocked() routine is not causing new allocations when 
> using BAIJ, should this be possible with AIJ?


    Sure, because the preallocation for BAIJ matrix is by block, while for AIJ 
it is by point (even if you provide a block size to AIJ). So long as the new 
entry is within an allocated block it will not trigger an error with BAIJ but 
may with AIJ.

   Barry

> (Clearly it *is* possible, but should it be?) I'm still trying to figure out 
> if there is something wrong with what PFLOTRAN is doing, vs. something going 
> wrong somewhere inside PETSc.
> 
> Any hints about what to look at from someone with more familiarity with this 
> code would be appreciated.
> 
> --Richard