barry/2019-10-15/bug-gamg-complexity/maint https://gitlab.com/petsc/petsc/merge_requests/2179
> On Oct 16, 2019, at 5:29 AM, Mark Lohry <[email protected]> wrote: > > Well that was a quick late night bug fix. Thanks Barry, I'll try it out. > > Just to confirm: You are running with with default double precision numbers > and have used the configure option --with-64-bit-indices ? > > Double precision floats, but 32 bit indices. I realize I'm playing with fire > here, but I'm bumping very close to available memory limits at this scale and > 64 bit indices tips me over. I figure integer index overflows would probably > show a catastrophic failure, but all output looks sane. > > I see you are using MATMFFD as the operator and MPIAIJ as the matrix from > which to build the preconditioner? This is not suppose to cause any > difficulties since the complexity computation code uses the second matrix, > that is the MPAIJ matrix to get the complexity information. > > Right, I'm using MATMFFD for the operator, and using a snes_lag_jacobian with > SNESComputeJacobianDefaultColor for the matrix used to build to > preconditioner. The actual behavior is exactly what I'd expect from smaller > runs and the results look good, so it sounds like what you describe. > > On Wed, Oct 16, 2019 at 12:17 AM Smith, Barry F. <[email protected]> wrote: > > I think I now see the bug: the code uses PetscInt lev, nnz0 = -1; > which will overflow. It should be using PetscLogDouble for nnz0 > > You can try changing that one place in the code and see that it now prints > a reasonable value for complexity. > > I will prepare a MR for maint to fix the bug permanently. > > Barry > > > static PetscErrorCode PCMGGetGridComplexity(PC pc, PetscReal *gc) > { > PetscErrorCode ierr; > PC_MG *mg = (PC_MG*)pc->data; > PC_MG_Levels **mglevels = mg->levels; > PetscInt lev, nnz0 = -1; > MatInfo info; > PetscFunctionBegin; > if (!mg->nlevels) SETERRQ(PETSC_COMM_SELF,PETSC_ERR_PLIB,"MG has no > levels"); > for (lev=0, *gc=0; lev<mg->nlevels; lev++) { > Mat dB; > ierr = KSPGetOperators(mglevels[lev]->smoothd,NULL,&dB);CHKERRQ(ierr); > ierr = MatGetInfo(dB,MAT_GLOBAL_SUM,&info);CHKERRQ(ierr); /* global > reduction */ > *gc += (PetscReal)info.nz_used; > if (lev==mg->nlevels-1) nnz0 = info.nz_used; > } > if (nnz0) *gc /= (PetscReal)nnz0; > else *gc = 0; > PetscFunctionReturn(0); > } > > > > > On Oct 15, 2019, at 11:11 PM, Smith, Barry F. <[email protected]> wrote: > > > > > > Mark, > > > > It may be caused by some overflow in the calculations somewhere due to > > your very large sizes and nonzeros but I could not see anything based on a > > quick inspection of the code. We seem to use double to store the counts > > which normally would be more than sufficient to hold the results without > > overflow. Unless somewhere there is a mistaken use of int that causes a > > problem. > > > > Just to confirm: You are running with with default double precision > > numbers and have used the configure option --with-64-bit-indices ? > > > > I see you are using MATMFFD as the operator and MPIAIJ as the matrix from > > which to build the preconditioner? This is not suppose to cause any > > difficulties since the complexity computation code uses the second matrix, > > that is the MPAIJ matrix to get the complexity information. > > > > There is definitely a bug but I am hard pressed to suggest how to find it > > since it seems only to be expressed in your giant runs. > > > > Barry > > > > > > > > > > > >> On Oct 15, 2019, at 9:16 PM, Mark Lohry via petsc-users > >> <[email protected]> wrote: > >> > >> I'm running some larger unsteady problems and trying to eek out some > >> better GAMG performance. As is, at very small time steps, ASM > >> preconditioner with ILU(0) is maybe 20% more efficient than my naive GAMG > >> setup, which gives me hope that some tuning of GAMG can give some > >> advantage. Convergence overall seems quite good, and light years better > >> than ASM/ILU at larger time steps. > >> > >> So looking through the manual and see a note that "grid complexity should > >> be well under 2.0 and preferably around 1.3 or lower". I check ksp_view > >> and see: > >> Complexity: grid = -40.5483 > >> > >> Is something funny happening here? > >> > >> Pasting whole -ksp_view below: > >> > >> KSP Object: 1920 MPI processes > >> type: fgmres > >> restart=100, using Classical (unmodified) Gram-Schmidt > >> Orthogonalization with no iterative refinement > >> happy breakdown tolerance 1e-30 > >> maximum iterations=30, initial guess is zero > >> tolerances: relative=0.0001, absolute=1e-06, divergence=10. > >> right preconditioning > >> using UNPRECONDITIONED norm type for convergence test > >> PC Object: 1920 MPI processes > >> type: gamg > >> type is MULTIPLICATIVE, levels=20 cycles=v > >> Cycles per PCApply=1 > >> Using externally compute Galerkin coarse grid matrices > >> GAMG specific options > >> Threshold for dropping small values in graph on each level = 0. > >> 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. > >> 0. 0. > >> Threshold scaling factor for each level not specified = 1. > >> AGG specific options > >> Symmetric graph false > >> Number of levels to square graph 1 > >> Number smoothing steps 0 > >> Complexity: grid = -40.5483 > >> Coarse grid solver -- level ------------------------------- > >> KSP Object: (mg_coarse_) 1920 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_coarse_) 1920 MPI processes > >> type: bjacobi > >> number of blocks = 1920 > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_coarse_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=1, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_coarse_sub_) 1 MPI processes > >> type: lu > >> out-of-place factorization > >> tolerance for zero pivot 2.22045e-14 > >> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] > >> matrix ordering: nd > >> factor fill ratio given 5., needed 1. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=15, cols=15, bs=5 > >> package used to perform factorization: petsc > >> total: nonzeros=175, allocated nonzeros=175 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 3 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=15, cols=15, bs=5 > >> total: nonzeros=175, allocated nonzeros=175 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 3 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=15, cols=15, bs=5 > >> total: nonzeros=175, allocated nonzeros=175 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 3 nodes, limit used > >> is 5 > >> Down solver (pre-smoother) on level 1 ------------------------------- > >> KSP Object: (mg_levels_1_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_1_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_1_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_1_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=4240, cols=4240 > >> package used to perform factorization: petsc > >> total: nonzeros=64800, allocated nonzeros=64800 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 848 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=4240, cols=4240 > >> package used to perform factorization: petsc > >> total: nonzeros=64800, allocated nonzeros=64800 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 848 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=4240, cols=4240, bs=5 > >> total: nonzeros=64800, allocated nonzeros=64800 > >> total number of mallocs used during MatSetValues calls =0 > >> using nonscalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 848 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 2 ------------------------------- > >> KSP Object: (mg_levels_2_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_2_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_2_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_2_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=4260, cols=4260 > >> package used to perform factorization: petsc > >> total: nonzeros=65200, allocated nonzeros=65200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 852 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=4260, cols=4260 > >> package used to perform factorization: petsc > >> total: nonzeros=65200, allocated nonzeros=65200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 852 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=4260, cols=4260, bs=5 > >> total: nonzeros=65200, allocated nonzeros=65200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 852 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 3 ------------------------------- > >> KSP Object: (mg_levels_3_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_3_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_3_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_3_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5440, cols=5440 > >> package used to perform factorization: petsc > >> total: nonzeros=90950, allocated nonzeros=90950 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1088 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5440, cols=5440 > >> package used to perform factorization: petsc > >> total: nonzeros=90950, allocated nonzeros=90950 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1088 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=5440, cols=5440, bs=5 > >> total: nonzeros=90950, allocated nonzeros=90950 > >> total number of mallocs used during MatSetValues calls =0 > >> using nonscalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 1088 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 4 ------------------------------- > >> KSP Object: (mg_levels_4_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_4_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_4_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_4_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5485, cols=5485 > >> package used to perform factorization: petsc > >> total: nonzeros=93075, allocated nonzeros=93075 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1097 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5485, cols=5485 > >> package used to perform factorization: petsc > >> total: nonzeros=93075, allocated nonzeros=93075 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1097 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=5485, cols=5485, bs=5 > >> total: nonzeros=93075, allocated nonzeros=93075 > >> total number of mallocs used during MatSetValues calls =0 > >> using nonscalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 1097 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 5 ------------------------------- > >> KSP Object: (mg_levels_5_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_5_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_5_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_5_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5685, cols=5685 > >> package used to perform factorization: petsc > >> total: nonzeros=98925, allocated nonzeros=98925 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1137 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5685, cols=5685 > >> package used to perform factorization: petsc > >> total: nonzeros=98925, allocated nonzeros=98925 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1137 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=5685, cols=5685, bs=5 > >> total: nonzeros=98925, allocated nonzeros=98925 > >> total number of mallocs used during MatSetValues calls =0 > >> using nonscalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 1137 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 6 ------------------------------- > >> KSP Object: (mg_levels_6_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_6_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_6_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_6_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5825, cols=5825 > >> package used to perform factorization: petsc > >> total: nonzeros=102325, allocated nonzeros=102325 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1165 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5825, cols=5825 > >> package used to perform factorization: petsc > >> total: nonzeros=102325, allocated nonzeros=102325 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1165 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=5825, cols=5825, bs=5 > >> total: nonzeros=102325, allocated nonzeros=102325 > >> total number of mallocs used during MatSetValues calls =0 > >> using nonscalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 1165 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 7 ------------------------------- > >> KSP Object: (mg_levels_7_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_7_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_7_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_7_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5925, cols=5925 > >> package used to perform factorization: petsc > >> total: nonzeros=104925, allocated nonzeros=104925 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1185 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=5925, cols=5925 > >> package used to perform factorization: petsc > >> total: nonzeros=104925, allocated nonzeros=104925 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1185 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=5925, cols=5925, bs=5 > >> total: nonzeros=104925, allocated nonzeros=104925 > >> total number of mallocs used during MatSetValues calls =0 > >> using nonscalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 1185 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 8 ------------------------------- > >> KSP Object: (mg_levels_8_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_8_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_8_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_8_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=6050, cols=6050 > >> package used to perform factorization: petsc > >> total: nonzeros=110200, allocated nonzeros=110200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1210 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=6050, cols=6050 > >> package used to perform factorization: petsc > >> total: nonzeros=110200, allocated nonzeros=110200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1210 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=6050, cols=6050, bs=5 > >> total: nonzeros=110200, allocated nonzeros=110200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 1210 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 9 ------------------------------- > >> KSP Object: (mg_levels_9_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_9_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_9_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_9_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=6890, cols=6890 > >> package used to perform factorization: petsc > >> total: nonzeros=153200, allocated nonzeros=153200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1378 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=6890, cols=6890 > >> package used to perform factorization: petsc > >> total: nonzeros=153200, allocated nonzeros=153200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1378 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=6890, cols=6890, bs=5 > >> total: nonzeros=153200, allocated nonzeros=153200 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 1378 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 10 ------------------------------- > >> KSP Object: (mg_levels_10_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_10_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_10_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_10_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=7395, cols=7395 > >> package used to perform factorization: petsc > >> total: nonzeros=180025, allocated nonzeros=180025 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1479 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=7395, cols=7395 > >> package used to perform factorization: petsc > >> total: nonzeros=180025, allocated nonzeros=180025 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1479 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=7395, cols=7395, bs=5 > >> total: nonzeros=180025, allocated nonzeros=180025 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 1479 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 11 ------------------------------- > >> KSP Object: (mg_levels_11_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_11_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_11_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_11_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=8960, cols=8960 > >> package used to perform factorization: petsc > >> total: nonzeros=259800, allocated nonzeros=259800 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1792 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=8960, cols=8960 > >> package used to perform factorization: petsc > >> total: nonzeros=259800, allocated nonzeros=259800 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 1792 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=8960, cols=8960, bs=5 > >> total: nonzeros=259800, allocated nonzeros=259800 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 1792 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 12 ------------------------------- > >> KSP Object: (mg_levels_12_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_12_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_12_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_12_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=1795, cols=1795 > >> package used to perform factorization: petsc > >> total: nonzeros=33275, allocated nonzeros=33275 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 359 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=1795, cols=1795 > >> package used to perform factorization: petsc > >> total: nonzeros=33275, allocated nonzeros=33275 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 359 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=11825, cols=11825, bs=5 > >> total: nonzeros=403125, allocated nonzeros=403125 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 359 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 13 ------------------------------- > >> KSP Object: (mg_levels_13_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_13_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_13_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_13_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=340, cols=340 > >> package used to perform factorization: petsc > >> total: nonzeros=3500, allocated nonzeros=3500 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 68 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=340, cols=340 > >> package used to perform factorization: petsc > >> total: nonzeros=3500, allocated nonzeros=3500 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 68 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=17210, cols=17210, bs=5 > >> total: nonzeros=696850, allocated nonzeros=696850 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 68 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 14 ------------------------------- > >> KSP Object: (mg_levels_14_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_14_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_14_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_14_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=125, cols=125 > >> package used to perform factorization: petsc > >> total: nonzeros=625, allocated nonzeros=625 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 25 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=125, cols=125 > >> package used to perform factorization: petsc > >> total: nonzeros=625, allocated nonzeros=625 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 25 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=29055, cols=29055, bs=5 > >> total: nonzeros=1475675, allocated nonzeros=1475675 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 25 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 15 ------------------------------- > >> KSP Object: (mg_levels_15_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_15_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_15_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_15_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=45, cols=45 > >> package used to perform factorization: petsc > >> total: nonzeros=225, allocated nonzeros=225 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 9 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=45, cols=45 > >> package used to perform factorization: petsc > >> total: nonzeros=225, allocated nonzeros=225 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 9 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=62935, cols=62935, bs=5 > >> total: nonzeros=3939025, allocated nonzeros=3939025 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 9 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 16 ------------------------------- > >> KSP Object: (mg_levels_16_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_16_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_16_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_16_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=55, cols=55 > >> package used to perform factorization: petsc > >> total: nonzeros=725, allocated nonzeros=725 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 11 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=55, cols=55 > >> package used to perform factorization: petsc > >> total: nonzeros=725, allocated nonzeros=725 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 11 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=205010, cols=205010, bs=5 > >> total: nonzeros=14780300, allocated nonzeros=14780300 > >> total number of mallocs used during MatSetValues calls =0 > >> using scalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 11 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 17 ------------------------------- > >> KSP Object: (mg_levels_17_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_17_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_17_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_17_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=360, cols=360 > >> package used to perform factorization: petsc > >> total: nonzeros=14350, allocated nonzeros=14350 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 72 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=360, cols=360 > >> package used to perform factorization: petsc > >> total: nonzeros=14350, allocated nonzeros=14350 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 72 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=921310, cols=921310, bs=5 > >> total: nonzeros=63203300, allocated nonzeros=63203300 > >> total number of mallocs used during MatSetValues calls =0 > >> using scalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 72 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 18 ------------------------------- > >> KSP Object: (mg_levels_18_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_18_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_18_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_18_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=2130, cols=2130 > >> package used to perform factorization: petsc > >> total: nonzeros=87950, allocated nonzeros=87950 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 426 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=2130, cols=2130 > >> package used to perform factorization: petsc > >> total: nonzeros=87950, allocated nonzeros=87950 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 426 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=4473930, cols=4473930, bs=5 > >> total: nonzeros=232427300, allocated nonzeros=232427300 > >> total number of mallocs used during MatSetValues calls =0 > >> using nonscalable MatPtAP() implementation > >> using I-node (on process 0) routines: found 426 nodes, limit used > >> is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> Down solver (pre-smoother) on level 19 ------------------------------- > >> KSP Object: (mg_levels_19_) 1920 MPI processes > >> type: richardson > >> damping factor=1. > >> maximum iterations=1, nonzero initial guess > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_19_) 1920 MPI processes > >> type: asm > >> total subdomain blocks = 1920, amount of overlap = 0 > >> restriction/interpolation type - RESTRICT > >> Local solve is same for all blocks, in the following KSP and PC > >> objects: > >> KSP Object: (mg_levels_19_sub_) 1 MPI processes > >> type: preonly > >> maximum iterations=10000, initial guess is zero > >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > >> left preconditioning > >> using NONE norm type for convergence test > >> PC Object: (mg_levels_19_sub_) 1 MPI processes > >> type: ilu > >> in-place factorization > >> 0 levels of fill > >> tolerance for zero pivot 2.22045e-14 > >> matrix ordering: natural > >> factor fill ratio given 0., needed 0. > >> Factored matrix follows: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=179050, cols=179050 > >> package used to perform factorization: petsc > >> total: nonzeros=42562500, allocated nonzeros=42562500 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 35810 nodes, limit used is 5 > >> linear system matrix = precond matrix: > >> Mat Object: 1 MPI processes > >> type: seqaij > >> rows=179050, cols=179050 > >> package used to perform factorization: petsc > >> total: nonzeros=42562500, allocated nonzeros=42562500 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node routines: found 35810 nodes, limit used is 5 > >> linear system matrix followed by preconditioner matrix: > >> Mat Object: 1920 MPI processes > >> type: mffd > >> rows=347149550, cols=347149550 > >> Matrix-free approximation: > >> err=1.49012e-08 (relative error in function evaluation) > >> Using wp compute h routine > >> Does not compute normU > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=347149550, cols=347149550, bs=5 > >> total: nonzeros=86758607500, allocated nonzeros=86758607500 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 35810 nodes, limit > >> used is 5 > >> Up solver (post-smoother) same as down solver (pre-smoother) > >> linear system matrix followed by preconditioner matrix: > >> Mat Object: 1920 MPI processes > >> type: mffd > >> rows=347149550, cols=347149550 > >> Matrix-free approximation: > >> err=1.49012e-08 (relative error in function evaluation) > >> Using wp compute h routine > >> Does not compute normU > >> Mat Object: 1920 MPI processes > >> type: mpiaij > >> rows=347149550, cols=347149550, bs=5 > >> total: nonzeros=86758607500, allocated nonzeros=86758607500 > >> total number of mallocs used during MatSetValues calls =0 > >> using I-node (on process 0) routines: found 35810 nodes, limit used > >> is 5 > >> Line search: Using full step: fnorm 2.025875581923e+03 gnorm > >> 2.801672254495e+00 > >> 1 SNES Function norm 2.801672254495e+00 > > >
