> On Feb 21, 2015, at 11:23 AM, Karl Rupp <[email protected]> wrote: > > Hi Dominic, > > yes, it is due to the changes for block-Jacobi, but I haven't found the exact > cause. It is definitely related to the following code lines in bjacobi.c, > around lines 820: > > /* > The reason we need to generate these vectors is to serve > as the right-hand side and solution vector for the solve on the > block. We do not need to allocate space for the vectors since > that is provided via VecPlaceArray() just before the call to > KSPSolve() on the block. > */ > MatGetSize(pmat,&m,&m); > VecCreateSeqWithArray(PETSC_COMM_SELF,1,m,NULL,&bjac->x); > VecCreateSeqWithArray(PETSC_COMM_SELF,1,m,NULL,&bjac->y); > #ifdef PETSC_HAVE_CUSP > VecSetType(bjac->x,VECCUSP); > VecSetType(bjac->y,VECCUSP); > #endif > > I don't think that it is a good idea to unconditionally set the vector type > to VECCUSP here (and at a few other locations in bjacobi.c). Particularly as > this causes a test to leak memory which does not even use CUSP vectors. We > may not have the 'correct' vector information available at this point, so we > can only guess which type is correct.
Would a call to MatGetVecs() here tell you the type of the vectors you need? Yeah you create and then immediately destroy a vector but that is ok. > > One way to circumvent this is to set the correct (GPU) vector type only when > applying the preconditioner, where the vector type is known. This would, > however, require some tweaks to VecSetType in order to avoid repeated memory > allocations and the like. Any other ideas? > > Best regards, > Karli > > > On 02/19/2015 11:13 PM, Dominic Meiser wrote: >> On 02/19/2015 02:31 PM, Karl Rupp wrote: >>> Hi, >>> >>> > Have you had a chance to look into this? >>> >>> sorry, no, I was busy with a project proposal. I'll look into it >>> tomorrow. >> I've been really swamped, too. I'm afraid these failures might be >> related to the changes we made for asm and bjacobi with GPU matrices. >> Feel free to ping me via google chat or skype if you'd like me to have a >> look at something or to go over some code together. >> >> Cheers, >> Dominic >> >>> >>> Best regards, >>> Karli >>> >>> >>> > >>>> On 02/19/2015 12:46 PM, Barry Smith wrote: >>>>> http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/02/18/examples_master_arch-cuda-double_bb-proxy.log >>>>> >>>>> >>>>> >>>>> >>>>> ========================================= >>>>> 1,2c1,150 >>>>> < Linear stokes_ solve converged due to CONVERGED_RTOL iterations 16 >>>>> < 9.0909e-02 1.9273e-03 2.9842e-04 4.8732e-03 >>>>> --- >>>>>> Linear stokes_ solve converged due to CONVERGED_ATOL iterations 0 >>>>>> 9.0909e-02 7.9577e-02 1.7911e-02 2.9249e-01 >>>>>> [0]Total space allocated 64 bytes >>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [0] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [0] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [0] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [0] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [1]Total space allocated 64 bytes >>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [1] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [1] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [1] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [1] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [2]Total space allocated 64 bytes >>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [2] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [2] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [2] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [2] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [3]Total space allocated 64 bytes >>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [3] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [3] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [3] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [3] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials >>>>> Possible problem with ex43_bjacobi, diffs above >>>>> ========================================= >>>>> 1,2c1,150 >>>>> < Linear stokes_ solve converged due to CONVERGED_RTOL iterations 16 >>>>> < 9.0909e-02 1.9273e-03 2.9842e-04 4.8732e-03 >>>>> --- >>>>>> Linear stokes_ solve converged due to CONVERGED_ATOL iterations 0 >>>>>> 9.0909e-02 7.9577e-02 1.7911e-02 2.9249e-01 >>>>>> [0]Total space allocated 64 bytes >>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [0] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [0] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [0] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [0] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [0] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [0] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [0] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [0] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [0] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [0] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [0] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [1]Total space allocated 64 bytes >>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [1] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [1] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [1] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [1] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [1] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [1] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [1] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [1] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [1] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [1] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [1] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [2]Total space allocated 64 bytes >>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [2] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [2] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [2] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [2] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [2] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [2] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [2] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [2] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [2] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [2] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [2] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [3]Total space allocated 64 bytes >>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [3] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [3] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_Seq_Private() line 1250 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c >>>>>> [3] VecCreate_SeqCUSP() line 1942 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] VecCreate_CUSP() line 242 in >>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu >>>>>> [3] VecSetType() line 38 in >>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c >>>>>> [3] PCSetUp_BJacobi_Singleblock() line 783 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp_BJacobi() line 24 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCSetUp() line 882 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSPSetUp() line 245 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] PCApply_BJacobi_Multiproc() line 1178 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c >>>>>> [3] PCApply() line 430 in >>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c >>>>>> [3] KSP_PCApply() line 233 in >>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h >>>>>> [3] KSPInitialResidual() line 44 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c >>>>>> [3] KSPSolve_GMRES() line 224 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c >>>>>> [3] KSPSolve() line 492 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c >>>>>> [3] solve_stokes_2d_coupled() line 1143 in >>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c >>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials >>>>> Possible problem with ex43_bjacobi_baij, diffs above >>>>> ========================================= >>>>> 46c46 >>>>> < 44 KSP Residual norm 5.15089e-05 >>>>> --- >>>>>> 44 KSP Residual norm 5.1509e-05 >>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials >>>>> Possible problem with ex49_1, diffs above >>>>> ========================================= >>>>> 2,14c2,16 >>>>> < 0 KSP Residual norm 4.5784 >>>>> < 1 KSP Residual norm 0.264677 >>>>> < 2 KSP Residual norm 0.067428 >>>>> < 3 KSP Residual norm 0.0188871 >>>>> < 4 KSP Residual norm 0.00960792 >>>>> < 5 KSP Residual norm 0.00522938 >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> >
