Hi Barry, hi Dmitry, I set the matrix to BAIJ and back to AIJ, and the code got a bit further. But I now run into the error pasted below (Note that I'm now using "--with-debugging=1"):
PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- PETSC ERROR: Petsc has generated inconsistent data PETSC ERROR: MPIAIJ Matrix was assembled but is missing garray PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 PETSC ERROR: ./example-dbg on a arch-linux2-c-debug named david-Lenovo by dknez Mon Feb 23 08:05:44 2015 PETSC ERROR: Configure options --with-shared-libraries=1 --with-debugging=1 --download-suitesparse=1 --download-parmetis=1 --download-blacs=1 --download-scalapack=1 --download-mumps=1 --download-metis --download-superlu_dist --prefix=/home/dknez/software/libmesh_install/dbg_real/petsc --download-hypre PETSC ERROR: #1 MatCreateColmap_MPIAIJ_Private() line 361 in /home/dknez/software/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c PETSC ERROR: #2 MatSetValues_MPIAIJ() line 538 in /home/dknez/software/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c PETSC ERROR: #3 MatSetValues() line 1136 in /home/dknez/software/petsc-3.5.2/src/mat/interface/matrix.c PETSC ERROR: #4 add_matrix() line 765 in /home/dknez/software/libmesh-src/src/numerics/petsc_matrix.C -------------------------------------------------------------------------- This occurs when I try to set some entries of the matrix. Do you have any suggestions on how I can resolve this? Thanks! David On Sun, Feb 22, 2015 at 10:22 PM, Dmitry Karpeyev <[email protected]> wrote: > > > On Sun Feb 22 2015 at 9:15:22 PM Barry Smith <[email protected]> wrote: > >> >> > On Feb 22, 2015, at 9:09 PM, David Knezevic <[email protected]> >> wrote: >> > >> > Hi Dmitry, >> > >> > Thanks for the suggestion. I tried MatSetType(mat,MATMPIAIJ) followed >> by MatXAIJSetPreallocation(...), but unfortunately this still gives me the >> same error as before: "nnz cannot be greater than row length: local row 168 >> value 24 rowlength 0". >> > >> > I gather that the idea here is that MatSetType builds a new matrix >> object, and then I should be able to pre-allocate for that new matrix >> however I like, right? Was I supposed to clear the matrix object somehow >> before calling MatSetType? (I didn't do any sort of clear operation.) >> >> If the type doesn't change then MatSetType() won't do anything. You can >> try setting the type to BAIJ and then setting the type back to AIJ. This >> may/should clear out the matrix. >> > Ah, yes. If the type is the same as before it does quit early, but > changing the type and then back will clear out and rebuild the matrix. We > need > something like MatReset() to do the equivalent thing. > >> >> > >> > As I said earlier, I'll make a dbg PETSc build, so hopefully that will >> help shed some light on what's going wrong for me. >> > I think it's always a good idea to have a dbg build of PETSc when you > doing things like these. > > Dmitry. > >> >> Don't bother, what I suggested won't work. >> >> Barry >> >> >> > >> > Thanks, >> > David >> > >> > >> > >> > >> > On Sun, Feb 22, 2015 at 6:02 PM, Dmitry Karpeyev <[email protected]> >> wrote: >> > David, >> > It might be easier to just rebuild the whole matrix from scratch: you >> would in effect be doing all that with disassembling and resetting the >> preallocation. >> > MatSetType(mat,MATMPIAIJ) >> > or >> > PetscObjectGetType((PetscObject)mat,&type); >> > MatSetType(mat,type); >> > followed by >> > MatXAIJSetPreallocation(...); >> > should do. >> > Dmitry. >> > >> > >> > On Sun Feb 22 2015 at 4:45:46 PM Barry Smith <[email protected]> >> wrote: >> > >> > Do not call for SeqAIJ matrix. Do not call before the first time you >> have preallocated and put entries in the matrix and done the >> MatAssemblyBegin/End() >> > >> > If it still crashes you'll need to try the debugger >> > >> > Barry >> > >> > > On Feb 22, 2015, at 4:09 PM, David Knezevic < >> [email protected]> wrote: >> > > >> > > Hi Barry, >> > > >> > > Thanks for your help, much appreciated. >> > > >> > > I added a prototype for MatDisAssemble_MPIAIJ: >> > > PETSC_INTERN PetscErrorCode MatDisAssemble_MPIAIJ(Mat); >> > > >> > > and I added a call to MatDisAssemble_MPIAIJ before >> MatMPIAIJSetPreallocation. However, I get a segfault on the call to >> MatDisAssemble_MPIAIJ. The segfault occurs in both serial and parallel. >> > > >> > > FYI, I'm using Petsc 3.5.2, and I'm not using a non-debug build >> (though I could rebuild PETSc in debug mode if you think that would help >> figure out what's happening here). >> > > >> > > Thanks, >> > > David >> > > >> > > >> > > >> > > On Sun, Feb 22, 2015 at 1:13 PM, Barry Smith <[email protected]> >> wrote: >> > > >> > > David, >> > > >> > > This is an obscure little feature of MatMPIAIJ, each time you >> change the sparsity pattern before you call the MatMPIAIJSetPreallocation >> you need to call MatDisAssemble_MPIAIJ(Mat mat). This is a private >> PETSc function so you need to provide your own prototype for it above the >> function you use it in. >> > > >> > > Let us know if this resolves the problem. >> > > >> > > Barry >> > > >> > > We never really intended that people would call >> MatMPIAIJSetPreallocation() AFTER they had already used the matrix. >> > > >> > > >> > > > On Feb 22, 2015, at 6:50 AM, David Knezevic < >> [email protected]> wrote: >> > > > >> > > > Hi all, >> > > > >> > > > I've implemented a solver for a contact problem using SNES. The >> sparsity pattern of the jacobian matrix needs to change at each nonlinear >> iteration (because the elements which are in contact can change), so I >> tried to deal with this by calling MatSeqAIJSetPreallocation and >> MatMPIAIJSetPreallocation during each iteration in order to update the >> preallocation. >> > > > >> > > > This seems to work fine in serial, but with two or more MPI >> processes I run into the error "nnz cannot be greater than row length", >> e.g.: >> > > > nnz cannot be greater than row length: local row 528 value 12 >> rowlength 0 >> > > > >> > > > This error is from the call to >> > > > MatSeqAIJSetPreallocation(b->B,o_nz,o_nnz); in >> MatMPIAIJSetPreallocation_MPIAIJ. >> > > > >> > > > Any guidance on what the problem might be would be most >> appreciated. For example, I was wondering if there is a problem with >> calling SetPreallocation on a matrix that has already been preallocated? >> > > > >> > > > Some notes: >> > > > - I'm using PETSc via libMesh >> > > > - The code that triggers this issue is available as a PR on the >> libMesh github repo, in case anyone is interested: >> https://github.com/libMesh/libmesh/pull/460/ >> > > > - I can try to make a minimal pure-PETSc example that reproduces >> this error, if that would be helpful. >> > > > >> > > > Many thanks, >> > > > David >> > > > >> > > >> > > >> > >> > >> >>
