On Fri, Jan 20, 2017 at 11:55 AM, Barry Smith <[email protected]> wrote:
> > > On Jan 20, 2017, at 11:49 AM, Matthew Knepley <[email protected]> wrote: > > > > On Fri, Jan 20, 2017 at 10:49 AM, Hong <[email protected]> wrote: > > Matt, > > By default, the flag C->submat_singleis = false. > > In PCSetUp_ASM(), we set it as 'true' to use MatGetSubMatrices_MPIAIJ_ > SingleIS(). > > > > Can you check the value of this flag in your case? > > > > The problem is the following: > > > > 1) We use MatGetSubMatrices() to extract small matrices in order to > form a preconditioner > > > > 2) We do this at each Newton iteration > > > > 3) We use ASM as a preconditioner for the eventual Newton solve > > > > 4) The second time we call MatGetSubMatrices(), it has this flag set, > even though we are using multiple ISes > > > > Solution: ALSO check that the user is in fact passing a single IS. > > > This requires a communication. I am confused, is the number of IS > changing each time? If not why is the flag set? ASM sets this flag because it knows that IT is going to call MatGetSubMatrices() later, but it unsafe if any user calls MatGetSubMatrices() as well. I think overall its a fragile design and should be scrapped. Matt > > Barry > > > Matt > > > > Hong > > > > It comes from here: > > > > https://bitbucket.org/petsc/petsc/commits/ > c10200c1442b553b7ad65c70101560db4fa22e78 > > > > If we ask for more than 1 matrix, it dispatches to > > > > MatGetSubMatrices_MPIAIJ_SingleIS() > > > > but then fails here > > > > https://bitbucket.org/petsc/petsc/annotate/ > 2e559809f9aee9c95ee79eb0939630cfe5502c8d/src/mat/impls/aij/ > mpi/mpiov.c?at=master&fileviewer=file-view-default#mpiov.c-1306 > > > > because ismax > 1. I think the ismax check needs to move up to here > > > > https://bitbucket.org/petsc/petsc/annotate/ > 2e559809f9aee9c95ee79eb0939630cfe5502c8d/src/mat/impls/aij/ > mpi/mpiov.c?at=master&fileviewer=file-view-default#mpiov.c-2012 > > > > but I don't know for sure. Please fix this since it is breaking PyLith. > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
