> On Jan 20, 2017, at 12:45 PM, Matthew Knepley <[email protected]> wrote: > > On Fri, Jan 20, 2017 at 11:55 AM, Barry Smith <[email protected]> wrote: > > > On Jan 20, 2017, at 11:49 AM, Matthew Knepley <[email protected]> wrote: > > > > On Fri, Jan 20, 2017 at 10:49 AM, Hong <[email protected]> wrote: > > Matt, > > By default, the flag C->submat_singleis = false. > > In PCSetUp_ASM(), we set it as 'true' to use > > MatGetSubMatrices_MPIAIJ_SingleIS(). > > > > Can you check the value of this flag in your case? > > > > The problem is the following: > > > > 1) We use MatGetSubMatrices() to extract small matrices in order to form > > a preconditioner > > > > 2) We do this at each Newton iteration > > > > 3) We use ASM as a preconditioner for the eventual Newton solve > > > > 4) The second time we call MatGetSubMatrices(), it has this flag set, > > even though we are using multiple ISes > > > > Solution: ALSO check that the user is in fact passing a single IS. > > > This requires a communication. I am confused, is the number of IS > changing each time? If not why is the flag set? > > ASM sets this flag because it knows that IT is going to call > MatGetSubMatrices() later, but it unsafe if any user calls > MatGetSubMatrices() as well. I think overall its a fragile design and should > be scrapped.
You mean the user calls the SAME MatGetSubMatrices with the same matrices, right? Not a completely different unrelated MatGetSubMatrices() which should not be affect by the previous unrelated call. Barry > > Matt > > > Barry > > > Matt > > > > Hong > > > > It comes from here: > > > > > > https://bitbucket.org/petsc/petsc/commits/c10200c1442b553b7ad65c70101560db4fa22e78 > > > > If we ask for more than 1 matrix, it dispatches to > > > > MatGetSubMatrices_MPIAIJ_SingleIS() > > > > but then fails here > > > > > > https://bitbucket.org/petsc/petsc/annotate/2e559809f9aee9c95ee79eb0939630cfe5502c8d/src/mat/impls/aij/mpi/mpiov.c?at=master&fileviewer=file-view-default#mpiov.c-1306 > > > > because ismax > 1. I think the ismax check needs to move up to here > > > > > > https://bitbucket.org/petsc/petsc/annotate/2e559809f9aee9c95ee79eb0939630cfe5502c8d/src/mat/impls/aij/mpi/mpiov.c?at=master&fileviewer=file-view-default#mpiov.c-2012 > > > > but I don't know for sure. Please fix this since it is breaking PyLith. > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which their > > experiments lead. > > -- Norbert Wiener > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which their > > experiments lead. > > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener
