On 26 October 2016 at 09:38, Mark Adams <[email protected]> wrote:
> Please run with -info and grep on GAMG and send that. (-info is very
> noisy).
>
> I cat the grep at the end of the log file (see attachment
petsc-3.7.4-n2.log).
Also, increasing the local number of iterations in SOR, as suggested by
Barry, removed the indefinite preconditioner (file
petsc-3.7.4-n2-lits2.log).
> I'm not sure what is going on here. Divergence with parallelism. Here are
> some suggestions.
>
> Note, you do not need to set the null space for a scalar (Poisson) problem
> unless you have some special null space. And not getting it set (with the 6
> rigid body modes) for the velocity (elasticity) equation will only degrade
> convergence rates.
>
> There was a bug for a while (early 3.7 versions) where the coarse grid was
> not squeezed onto one processor, which could result in very bad
> convergence, but not divergence, on multiple processors (the -info output
> will report the number of 'active pes'). Perhaps this bug is causing
> divergence for you. We had another subtle bug where the eigen estimates
> used a bad seed vector, which gives a bad eigen estimate. This would cause
> divergence but it should not be a parallelism issue (these two bugs were
> both regressions in around 3.7)
>
> Divergence usually comes from a bad eigen estimate in a Chebyshev
> smoother, but this is not highly correlated with parallelism. The -info
> data will report the eigen estimates but that is not terribly useful but
> you can see if it changes (gets larger) with better parameters. Add these
> parameters, with the correct prefix, and use -options_left to make sure
> that "there are no unused options":
>
> -mg_levels_ksp_type chebyshev
> -mg_levels_esteig_ksp_type cg
> -mg_levels_esteig_ksp_max_it 10
>
> -mg_levels_ksp_chebyshev_esteig 0,.1,0,1.05
>
> petsc-3.7.4-n2-chebyshev.log contains the output when using the default
KSP Chebyshev.
When estimating the eigenvalues using cg with the translations [0, 0.1; 0,
1.05] (previously using default gmres with translations [0, 0.1; 0, 1.1]),
the max eigenvalue decreases from 1.0931 to 1.04366 and the indefinite
preconditioner appears ealier after 2 iterations (3 previously).
I attached the log (see petsc-3.7.4-chebyshev.log).
> chebyshev is the default, as Barry suggested, replace this with gmres or
> richardson (see below) and verify that this fixed the divergence problem.
>
>
Using gmres (-poisson_mg_levels_ksp_type gmres) fixes the divergence problem
(file petsc-3.7.4-n2-gmres.log)
.
Same observation with richardson (file petsc-3.7.4-n2-richardson.log).
> If your matrix is symmetric positive definite then use
> '-mg_levels_esteig_ksp_type cg', if not then use the default gmres.
>
I checked and I still get an indefinite preconditioner when using gmres to
estimate the eigenvalues.
>
> Increase/decrease '-mg_levels_esteig_ksp_max_it 10', you should see the
> estimates increase and converge with higher max_it. Setting this to a huge
> number, like 100, should fix the bad seed vector problem mentioned above.
>
> I played with the maximum number of iterations. Here are the min/max
eigenvalue estimates for the two levels:
- max_it 5: (min=0.0975079, max=1.02383) on level 1, (0.0975647, 1.02443)
on level 2
- max_it 10: (0.0991546, 1.04112), (0.0993962, 1.04366)
- max_it 20: (0.0995918, 1.04571), (0.115723, 1.21509)
- max_it 50: (0.0995651, 1.04543), (0.133744, 1.40431)
- max_it 100: (0.0995651, 1.04543), (0.133744, 1.40431)
Note that all those runs ended up with an indefinite preconditioner, except
when increasing the maximum number of iterations to 50 (and 100, which did
not improve the eigenvalue estimates).
> If eigen estimates are a pain, like with non SPD systems, then
> richardson is an option (instead of chebyshev):
>
> -mg_levels_ksp_type richardson
> -mg_levels_ksp_richardson_scale 0.6
>
> You then need to play with the scaling (that is what chebyshev does for
> you essentially).
>
>
> On Tue, Oct 25, 2016 at 10:22 PM, Matthew Knepley <[email protected]>
> wrote:
>
>> On Tue, Oct 25, 2016 at 9:20 PM, Barry Smith <[email protected]> wrote:
>>
>>>
>>> Olivier,
>>>
>>> Ok, so I've run the code in the debugger, but I don't not think the
>>> problem is with the null space. The code is correctly removing the null
>>> space on all the levels of multigrid.
>>>
>>> I think the error comes from changes in the behavior of GAMG. GAMG
>>> is relatively rapidly moving with different defaults and even different
>>> code with each release.
>>>
>>> To check this I added the option -poisson_mg_levels_pc_sor_lits 2
>>> and it stopped complaining about KSP_DIVERGED_INDEFINITE_PC. I've seen this
>>> before where the smoother is "too weak" and so the net result is that
>>> action of the preconditioner is indefinite. Mark Adams probably has better
>>> suggestions on how to make the preconditioner behave. Note you could also
>>> use a KSP of richardson or gmres instead of cg since they don't care about
>>> this indefinite business.
>>
>>
>> I think old GAMG squared the graph by default. You can see in the 3.7
>> output that it does not.
>>
>> Matt
>>
>>
>>>
>>> Barry
>>>
>>>
>>>
>>> > On Oct 25, 2016, at 5:39 PM, Olivier Mesnard <
>>> [email protected]> wrote:
>>> >
>>> > On 25 October 2016 at 17:51, Barry Smith <[email protected]> wrote:
>>> >
>>> > Olivier,
>>> >
>>> > In theory you do not need to change anything else. Are you using a
>>> different matrix object for the velocity_ksp object than the poisson_ksp
>>> object?
>>> >
>>> > The matrix is different for the velocity_ksp and the poisson_ksp.
>>> >
>>> > The code change in PETSc is very little but we have a report from
>>> another CFD user who also had problems with the change so there may be some
>>> subtle bug that we can't figure out causing things to not behave properly.
>>> >
>>> > First run the 3.7.4 code with -poisson_ksp_view and verify that
>>> when it prints the matrix information it prints something like has attached
>>> null space if it does not print that it means that somehow the matrix is
>>> not properly getting the matrix attached.
>>> >
>>> > When running with 3.7.4 and -poisson_ksp_view, the output shows that
>>> the nullspace is not attached to the KSP (as it was with 3.5.4); however
>>> the print statement is now under the Mat info (which is expected when
>>> moving from KSPSetNullSpace to MatSetNullSpace?).
>>> >
>>> > Though older versions had MatSetNullSpace() they didn't
>>> necessarily associate it with the KSP so it was not expected to work as a
>>> replacement for KSPSetNullSpace() with older versions.
>>> >
>>> > Because our other user had great difficulty trying to debug the
>>> issue feel free to send us at [email protected] your code with
>>> instructions on building and running and we can try to track down the
>>> problem. Better than hours and hours spent with fruitless email. We will,
>>> of course, not distribute the code and will delete in when we are finished
>>> with it.
>>> >
>>> > The code is open-source and hosted on GitHub (
>>> https://github.com/barbagroup/PetIBM).
>>> > I just pushed the branches `feature-compatible-petsc-3.7` and
>>> `revert-compatible-petsc-3.5` that I used to observe this problem.
>>> >
>>> > PETSc (both 3.5.4 and 3.7.4) was configured as follow:
>>> > export PETSC_ARCH="linux-gnu-dbg"
>>> > ./configure --PETSC_ARCH=$PETSC_ARCH \
>>> > --with-cc=gcc \
>>> > --with-cxx=g++ \
>>> > --with-fc=gfortran \
>>> > --COPTFLAGS="-O0" \
>>> > --CXXOPTFLAGS="-O0" \
>>> > --FOPTFLAGS="-O0" \
>>> > --with-debugging=1 \
>>> > --download-fblaslapack \
>>> > --download-mpich \
>>> > --download-hypre \
>>> > --download-yaml \
>>> > --with-x=1
>>> >
>>> > Our code was built using the following commands:
>>> > mkdir petibm-build
>>> > cd petibm-build
>>> > export PETSC_DIR=<directory of PETSc>
>>> > export PETSC_ARCH="linux-gnu-dbg"
>>> > export PETIBM_DIR=<directory of PetIBM git repo>
>>> > $PETIBM_DIR/configure --prefix=$PWD \
>>> > CXX=$PETSC_DIR/$PETSC_ARCH/bin/mpicxx \
>>> > CXXFLAGS="-g -O0 -std=c++11"
>>> > make all
>>> > make install
>>> >
>>> > Then
>>> > cd examples
>>> > make examples
>>> >
>>> > The example of the lid-driven cavity I was talking about can be found
>>> in the folder `examples/2d/convergence/lidDrivenCavity20/20/`
>>> >
>>> > To run it:
>>> > mpiexec -n N <path-to-petibm-build>/bin/petibm2d -directory
>>> <path-to-example>
>>> >
>>> > Let me know if you need more info. Thank you.
>>> >
>>> > Barry
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > > On Oct 25, 2016, at 4:38 PM, Olivier Mesnard <
>>> [email protected]> wrote:
>>> > >
>>> > > Hi all,
>>> > >
>>> > > We develop a CFD code using the PETSc library that solves the
>>> Navier-Stokes equations using the fractional-step method from Perot (1993).
>>> > > At each time-step, we solve two systems: one for the velocity field,
>>> the other, a Poisson system, for the pressure field.
>>> > > One of our test-cases is a 2D lid-driven cavity flow (Re=100) on a
>>> 20x20 grid using 1 or 2 procs.
>>> > > For the Poisson system, we usually use CG preconditioned with GAMG.
>>> > >
>>> > > So far, we have been using PETSc-3.5.4, and we would like to update
>>> the code with the latest release: 3.7.4.
>>> > >
>>> > > As suggested in the changelog of 3.6, we replaced the routine
>>> `KSPSetNullSpace()` with `MatSetNullSpace()`.
>>> > >
>>> > > Here is the list of options we use to configure the two solvers:
>>> > > * Velocity solver: prefix `-velocity_`
>>> > > -velocity_ksp_type bcgs
>>> > > -velocity_ksp_rtol 1.0E-08
>>> > > -velocity_ksp_atol 0.0
>>> > > -velocity_ksp_max_it 10000
>>> > > -velocity_pc_type jacobi
>>> > > -velocity_ksp_view
>>> > > -velocity_ksp_monitor_true_residual
>>> > > -velocity_ksp_converged_reason
>>> > > * Poisson solver: prefix `-poisson_`
>>> > > -poisson_ksp_type cg
>>> > > -poisson_ksp_rtol 1.0E-08
>>> > > -poisson_ksp_atol 0.0
>>> > > -poisson_ksp_max_it 20000
>>> > > -poisson_pc_type gamg
>>> > > -poisson_pc_gamg_type agg
>>> > > -poisson_pc_gamg_agg_nsmooths 1
>>> > > -poissonksp_view
>>> > > -poisson_ksp_monitor_true_residual
>>> > > -poisson_ksp_converged_reason
>>> > >
>>> > > With 3.5.4, the case runs normally on 1 or 2 procs.
>>> > > With 3.7.4, the case runs normally on 1 proc but not on 2.
>>> > > Why? The Poisson solver diverges because of an indefinite
>>> preconditioner (only with 2 procs).
>>> > >
>>> > > We also saw that the routine `MatSetNullSpace()` was already
>>> available in 3.5.4.
>>> > > With 3.5.4, replacing `KSPSetNullSpace()` with `MatSetNullSpace()`
>>> led to the Poisson solver diverging because of an indefinite matrix (on 1
>>> and 2 procs).
>>> > >
>>> > > Thus, we were wondering if we needed to update something else for
>>> the KSP, and not just modifying the name of the routine?
>>> > >
>>> > > I have attached the output files from the different cases:
>>> > > * `run-petsc-3.5.4-n1.log` (3.5.4, `KSPSetNullSpace()`, n=1)
>>> > > * `run-petsc-3.5.4-n2.log`
>>> > > * `run-petsc-3.5.4-nsp-n1.log` (3.5.4, `MatSetNullSpace()`, n=1)
>>> > > * `run-petsc-3.5.4-nsp-n2.log`
>>> > > * `run-petsc-3.7.4-n1.log` (3.7.4, `MatSetNullSpace()`, n=1)
>>> > > * `run-petsc-3.7.4-n2.log`
>>> > >
>>> > > Thank you for your help,
>>> > > Olivier
>>> > > <run-petsc-3.5.4-n1.log><run-petsc-3.5.4-n2.log><run-petsc-3
>>> .5.4-nsp-n1.log><run-petsc-3.5.4-nsp-n2.log><run-petsc-3.7.4
>>> -n1.log><run-petsc-3.7.4-n2.log>
>>> >
>>> >
>>>
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>
>
======================
*** PetIBM - Start ***
======================
directory: ./
Parsing file .//cartesianMesh.yaml... done.
Parsing file .//flowDescription.yaml... done.
Parsing file .//simulationParameters.yaml... done.
---------------------------------------
Cartesian grid
---------------------------------------
number of cells: 20 x 20
---------------------------------------
---------------------------------------
Flow
---------------------------------------
dimensions: 2
viscosity: 0.01
initial velocity field:
0
0
boundary conditions (component, type, value):
->location: xMinus (left)
0 DIRICHLET 0
1 DIRICHLET 0
->location: xPlus (right)
0 DIRICHLET 0
1 DIRICHLET 0
->location: yMinus (bottom)
0 DIRICHLET 0
1 DIRICHLET 0
->location: yPlus (top)
0 DIRICHLET 1
1 DIRICHLET 0
---------------------------------------
---------------------------------------
Time-stepping
---------------------------------------
formulation: Navier-Stokes solver (Perot, 1993)
convection: Euler-explicit
diffusion: Euler-implicit
time-increment: 0.0005
starting time-step: 0
number of time-steps: 1
saving-interval: 1
---------------------------------------
----------------------------------------
KSP info: Velocity system
----------------------------------------
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0, divergence=10000
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
PC has not been set up so information may be incomplete
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
----------------------------------------
KSP info: Poisson system
----------------------------------------
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0, divergence=10000
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
PC has not been set up so information may be incomplete
MG: type is MULTIPLICATIVE, levels=0 cycles=unknown
Cycles per PCApply=0
Using Galerkin computed coarse grid matrices
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
[time-step 0] Writing grid into file... done.
Residual norms for velocity_ solve.
0 KSP preconditioned resid norm 1.148589971947e-03 true resid norm 2.324746103222e+00 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 8.875846864181e-09 true resid norm 1.793849713607e-05 ||r(i)||/||b|| 7.716325284387e-06
2 KSP preconditioned resid norm 1.418061760018e-13 true resid norm 2.860743141913e-10 ||r(i)||/||b|| 1.230561538720e-10
Linear solve converged due to CONVERGED_RTOL iterations 2
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0, divergence=10000
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Residual norms for poisson_ solve.
0 KSP preconditioned resid norm 3.540871046457e+00 true resid norm 3.733946078026e-04 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 1.257128124270e-01 true resid norm 7.334555515521e-05 ||r(i)||/||b|| 1.964290689329e-01
2 KSP preconditioned resid norm 1.250635679151e-02 true resid norm 5.229463758195e-06 ||r(i)||/||b|| 1.400519356445e-02
3 KSP preconditioned resid norm 7.044787396120e-04 true resid norm 4.496085949179e-07 ||r(i)||/||b|| 1.204111107988e-03
4 KSP preconditioned resid norm 5.752276071869e-05 true resid norm 3.304114740417e-08 ||r(i)||/||b|| 8.848854995153e-05
5 KSP preconditioned resid norm 4.501641492631e-06 true resid norm 2.475433259121e-09 ||r(i)||/||b|| 6.629536708334e-06
6 KSP preconditioned resid norm 2.447627881266e-07 true resid norm 1.929318717776e-10 ||r(i)||/||b|| 5.166969949379e-07
7 KSP preconditioned resid norm 1.594610003820e-08 true resid norm 1.215791508998e-11 ||r(i)||/||b|| 3.256049989990e-08
Linear solve converged due to CONVERGED_RTOL iterations 7
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0, divergence=10000
left preconditioning
has attached null space
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
MG: type is MULTIPLICATIVE, levels=2 cycles=v
Cycles per PCApply=1
Using Galerkin computed coarse grid matrices
Coarse grid solver -- level -------------------------------
KSP Object: (poisson_mg_coarse_) 2 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_) 2 MPI processes
type: bjacobi
block Jacobi: number of blocks = 2
Local solve is same for all blocks, in the following KSP and PC objects:
KSP Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: lu
LU: out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5, needed 2.45338
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=66, cols=66
package used to perform factorization: petsc
total: nonzeros=1526, allocated nonzeros=1526
total number of mallocs used during MatSetValues calls =0
not using I-node routines
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=66, cols=66
total: nonzeros=622, allocated nonzeros=622
total number of mallocs used during MatSetValues calls =0
not using I-node routines
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=66, cols=66
total: nonzeros=622, allocated nonzeros=622
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (poisson_mg_levels_1_) 2 MPI processes
type: chebyshev
Chebyshev: eigenvalue estimates: min = 0.0990463, max = 2.07997
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_1_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
[time-step 1] Writing fluxes into file... done.
[time-step 1] Writing pressure into file... done.
=====================
*** PetIBM - Done ***
=====================
#PETSc Option Table entries:
-options_left
-poisson_ksp_atol 0.0
-poisson_ksp_converged_reason
-poisson_ksp_max_it 20000
-poisson_ksp_monitor_true_residual
-poisson_ksp_rtol 1.0E-08
-poisson_ksp_type cg
-poisson_ksp_view
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_gamg_type agg
-poisson_pc_type gamg
-velocity_ksp_atol 0.0
-velocity_ksp_converged_reason
-velocity_ksp_max_it 10000
-velocity_ksp_monitor_true_residual
-velocity_ksp_rtol 1.0E-08
-velocity_ksp_type bcgs
-velocity_ksp_view
-velocity_pc_type jacobi
#End of PETSc Option Table entries
There are no unused options.
======================
*** PetIBM - Start ***
======================
directory: .
Parsing file ./cartesianMesh.yaml... done.
Parsing file ./flowDescription.yaml... done.
Parsing file ./simulationParameters.yaml... done.
---------------------------------------
Cartesian grid
---------------------------------------
number of cells: 20 x 20
---------------------------------------
---------------------------------------
Flow
---------------------------------------
dimensions: 2
viscosity: 0.01
initial velocity field:
0.
0.
boundary conditions (component, type, value):
->location: xMinus (left)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: xPlus (right)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yMinus (bottom)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yPlus (top)
0 DIRICHLET 1.
1 DIRICHLET 0.
---------------------------------------
---------------------------------------
Time-stepping
---------------------------------------
formulation: Navier-Stokes solver (Perot, 1993)
convection: Euler-explicit
diffusion: Euler-implicit
time-increment: 0.0005
starting time-step: 0
number of time-steps: 1
saving-interval: 1
---------------------------------------
----------------------------------------
KSP info: Velocity system
----------------------------------------
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
PC has not been set up so information may be incomplete
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
----------------------------------------
KSP info: Poisson system
----------------------------------------
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
PC has not been set up so information may be incomplete
MG: type is MULTIPLICATIVE, levels=0 cycles=unknown
Cycles per PCApply=0
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
[time-step 0] Writing grid into file... done.
Residual norms for velocity_ solve.
0 KSP preconditioned resid norm 1.148589971947e-03 true resid norm 2.324746103222e+00 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 8.875846864181e-09 true resid norm 1.793849713607e-05 ||r(i)||/||b|| 7.716325284387e-06
2 KSP preconditioned resid norm 1.418061760018e-13 true resid norm 2.860743141913e-10 ||r(i)||/||b|| 1.230561538720e-10
Linear velocity_ solve converged due to CONVERGED_RTOL iterations 2
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Residual norms for poisson_ solve.
0 KSP preconditioned resid norm 3.663481636744e+00 true resid norm 3.733946078026e-04 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 1.682146562271e-01 true resid norm 2.719558922539e-05 ||r(i)||/||b|| 7.283337428313e-02
2 KSP preconditioned resid norm 7.119755653883e-03 true resid norm 1.420925878483e-06 ||r(i)||/||b|| 3.805426882957e-03
3 KSP preconditioned resid norm 2.543557070242e-04 true resid norm 5.182168464649e-08 ||r(i)||/||b|| 1.387853053140e-04
4 KSP preconditioned resid norm 1.098456886904e-05 true resid norm 2.119096316615e-09 ||r(i)||/||b|| 5.675219385428e-06
5 KSP preconditioned resid norm 2.051269834107e-07 true resid norm 7.232535590070e-11 ||r(i)||/||b|| 1.936968407935e-07
6 KSP preconditioned resid norm 2.591096621283e-09 true resid norm 2.471788276719e-12 ||r(i)||/||b|| 6.619774964789e-09
Linear poisson_ solve converged due to CONVERGED_RTOL iterations 6
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
MG: type is MULTIPLICATIVE, levels=3 cycles=v
Cycles per PCApply=1
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
Coarse grid solver -- level -------------------------------
KSP Object: (poisson_mg_coarse_) 2 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_) 2 MPI processes
type: bjacobi
block Jacobi: number of blocks = 2
Local solve is same for all blocks, in the following KSP and PC objects:
KSP Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: lu
LU: out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5., needed 1.09231
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
package used to perform factorization: petsc
total: nonzeros=142, allocated nonzeros=142
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 4 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 7 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 7 nodes, limit used is 5
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (poisson_mg_levels_1_) 2 MPI processes
type: chebyshev
Chebyshev: eigenvalue estimates: min = 0.0998467, max = 1.09831
Chebyshev: eigenvalues estimated using gmres with translations [0. 0.1; 0. 1.1]
KSP Object: (poisson_mg_levels_1_esteig_) 2 MPI processes
type: gmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-30
maximum iterations=10, initial guess is zero
tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_1_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 2, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=62, cols=62
total: nonzeros=556, allocated nonzeros=556
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (poisson_mg_levels_2_) 2 MPI processes
type: chebyshev
Chebyshev: eigenvalue estimates: min = 0.14153, max = 1.55683
Chebyshev: eigenvalues estimated using gmres with translations [0. 0.1; 0. 1.1]
KSP Object: (poisson_mg_levels_2_esteig_) 2 MPI processes
type: gmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-30
maximum iterations=10, initial guess is zero
tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_2_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 2, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
[time-step 1] Writing fluxes into file... done.
[time-step 1] Writing pressure into file... done.
=====================
*** PetIBM - Done ***
=====================
#PETSc Option Table entries:
-options_left
-poisson_ksp_atol 0.0
-poisson_ksp_converged_reason
-poisson_ksp_max_it 20000
-poisson_ksp_monitor_true_residual
-poisson_ksp_rtol 1.0E-08
-poisson_ksp_type cg
-poisson_ksp_view
-poisson_mg_levels_pc_sor_lits 2
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_gamg_type agg
-poisson_pc_type gamg
-velocity_ksp_atol 0.0
-velocity_ksp_converged_reason
-velocity_ksp_max_it 10000
-velocity_ksp_monitor_true_residual
-velocity_ksp_rtol 1.0E-08
-velocity_ksp_type bcgs
-velocity_ksp_view
-velocity_pc_type jacobi
#End of PETSc Option Table entries
There are no unused options.
GAMG specific options
[0] PCSetUp_GAMG(): level 0) N=400, n data rows=1, n data cols=1, nnz/row (ave)=5, np=2
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 4.8 nnz ave. (N=400)
[0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
[0] PCGAMGProlongator_AGG(): New grid 62 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.977326e+00 min=3.920450e-02 PC=jacobi
[0] PCGAMGCreateLevel_GAMG(): Number of equations (loc) 30 with simple aggregation
[0] PCSetUp_GAMG(): 1) N=62, n data cols=1, nnz/row (ave)=8, 1 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 8.96774 nnz ave. (N=62)
[0] PCGAMGProlongator_AGG(): New grid 12 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.556629e+00 min=1.967905e-03 PC=jacobi
[0] PCSetUp_GAMG(): 2) N=12, n data cols=1, nnz/row (ave)=10, 1 active pes
[0] PCSetUp_GAMG(): 3 levels, grid complexity = 1.35729
GAMG specific options
======================
*** PetIBM - Start ***
======================
directory: .
Parsing file ./cartesianMesh.yaml... done.
Parsing file ./flowDescription.yaml... done.
Parsing file ./simulationParameters.yaml... done.
---------------------------------------
Cartesian grid
---------------------------------------
number of cells: 20 x 20
---------------------------------------
---------------------------------------
Flow
---------------------------------------
dimensions: 2
viscosity: 0.01
initial velocity field:
0.
0.
boundary conditions (component, type, value):
->location: xMinus (left)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: xPlus (right)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yMinus (bottom)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yPlus (top)
0 DIRICHLET 1.
1 DIRICHLET 0.
---------------------------------------
---------------------------------------
Time-stepping
---------------------------------------
formulation: Navier-Stokes solver (Perot, 1993)
convection: Euler-explicit
diffusion: Euler-implicit
time-increment: 0.0005
starting time-step: 0
number of time-steps: 1
saving-interval: 1
---------------------------------------
----------------------------------------
KSP info: Velocity system
----------------------------------------
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
PC has not been set up so information may be incomplete
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
----------------------------------------
KSP info: Poisson system
----------------------------------------
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
PC has not been set up so information may be incomplete
MG: type is MULTIPLICATIVE, levels=0 cycles=unknown
Cycles per PCApply=0
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
[time-step 0] Writing grid into file... done.
Residual norms for velocity_ solve.
0 KSP preconditioned resid norm 1.148589971947e-03 true resid norm 2.324746103222e+00 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 8.875846864181e-09 true resid norm 1.793849713607e-05 ||r(i)||/||b|| 7.716325284387e-06
2 KSP preconditioned resid norm 1.418061760018e-13 true resid norm 2.860743141913e-10 ||r(i)||/||b|| 1.230561538720e-10
Linear velocity_ solve converged due to CONVERGED_RTOL iterations 2
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Residual norms for poisson_ solve.
0 KSP preconditioned resid norm 3.594808901578e+00 true resid norm 3.733946078026e-04 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.501868872941e-01 true resid norm 5.302866028514e-05 ||r(i)||/||b|| 1.420177452407e-01
Linear poisson_ solve did not converge due to DIVERGED_INDEFINITE_PC iterations 2
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
MG: type is MULTIPLICATIVE, levels=3 cycles=v
Cycles per PCApply=1
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
Coarse grid solver -- level -------------------------------
KSP Object: (poisson_mg_coarse_) 2 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_) 2 MPI processes
type: bjacobi
block Jacobi: number of blocks = 2
Local solve is same for all blocks, in the following KSP and PC objects:
KSP Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: lu
LU: out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5., needed 1.09231
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
package used to perform factorization: petsc
total: nonzeros=142, allocated nonzeros=142
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 4 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 7 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 7 nodes, limit used is 5
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (poisson_mg_levels_1_) 2 MPI processes
type: chebyshev
Chebyshev: eigenvalue estimates: min = 0.0991546, max = 1.04112
Chebyshev: eigenvalues estimated using cg with translations [0. 0.1; 0. 1.05]
KSP Object: (poisson_mg_levels_1_esteig_) 2 MPI processes
type: cg
maximum iterations=10, initial guess is zero
tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_1_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=62, cols=62
total: nonzeros=556, allocated nonzeros=556
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (poisson_mg_levels_2_) 2 MPI processes
type: chebyshev
Chebyshev: eigenvalue estimates: min = 0.0993962, max = 1.04366
Chebyshev: eigenvalues estimated using cg with translations [0. 0.1; 0. 1.05]
KSP Object: (poisson_mg_levels_2_esteig_) 2 MPI processes
type: cg
maximum iterations=10, initial guess is zero
tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_2_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
[time-step 0] ERROR: Poisson solver diverged due to reason: -8
#PETSc Option Table entries:
-options_left
-poisson_ksp_atol 0.0
-poisson_ksp_converged_reason
-poisson_ksp_max_it 20000
-poisson_ksp_monitor_true_residual
-poisson_ksp_rtol 1.0E-08
-poisson_ksp_type cg
-poisson_ksp_view
-poisson_mg_levels_esteig_ksp_max_it 10
-poisson_mg_levels_esteig_ksp_type cg
-poisson_mg_levels_ksp_chebyshev_esteig 0,0.1,0,1.05
-poisson_mg_levels_ksp_type chebyshev
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_gamg_type agg
-poisson_pc_type gamg
-velocity_ksp_atol 0.0
-velocity_ksp_converged_reason
-velocity_ksp_max_it 10000
-velocity_ksp_monitor_true_residual
-velocity_ksp_rtol 1.0E-08
-velocity_ksp_type bcgs
-velocity_ksp_view
-velocity_pc_type jacobi
#End of PETSc Option Table entries
There are no unused options.
GAMG specific options
[0] PCSetUp_GAMG(): level 0) N=400, n data rows=1, n data cols=1, nnz/row (ave)=5, np=2
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 4.8 nnz ave. (N=400)
[0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
[0] PCGAMGProlongator_AGG(): New grid 62 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.977326e+00 min=3.920450e-02 PC=jacobi
[0] PCGAMGCreateLevel_GAMG(): Number of equations (loc) 30 with simple aggregation
[0] PCSetUp_GAMG(): 1) N=62, n data cols=1, nnz/row (ave)=8, 1 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 8.96774 nnz ave. (N=62)
[0] PCGAMGProlongator_AGG(): New grid 12 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.556629e+00 min=1.967905e-03 PC=jacobi
[0] PCSetUp_GAMG(): 2) N=12, n data cols=1, nnz/row (ave)=10, 1 active pes
[0] PCSetUp_GAMG(): 3 levels, grid complexity = 1.35729
GAMG specific options
======================
*** PetIBM - Start ***
======================
directory: .
Parsing file ./cartesianMesh.yaml... done.
Parsing file ./flowDescription.yaml... done.
Parsing file ./simulationParameters.yaml... done.
---------------------------------------
Cartesian grid
---------------------------------------
number of cells: 20 x 20
---------------------------------------
---------------------------------------
Flow
---------------------------------------
dimensions: 2
viscosity: 0.01
initial velocity field:
0.
0.
boundary conditions (component, type, value):
->location: xMinus (left)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: xPlus (right)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yMinus (bottom)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yPlus (top)
0 DIRICHLET 1.
1 DIRICHLET 0.
---------------------------------------
---------------------------------------
Time-stepping
---------------------------------------
formulation: Navier-Stokes solver (Perot, 1993)
convection: Euler-explicit
diffusion: Euler-implicit
time-increment: 0.0005
starting time-step: 0
number of time-steps: 1
saving-interval: 1
---------------------------------------
----------------------------------------
KSP info: Velocity system
----------------------------------------
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
PC has not been set up so information may be incomplete
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
----------------------------------------
KSP info: Poisson system
----------------------------------------
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
PC has not been set up so information may be incomplete
MG: type is MULTIPLICATIVE, levels=0 cycles=unknown
Cycles per PCApply=0
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
[time-step 0] Writing grid into file... done.
Residual norms for velocity_ solve.
0 KSP preconditioned resid norm 1.148589971947e-03 true resid norm 2.324746103222e+00 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 8.875846864181e-09 true resid norm 1.793849713607e-05 ||r(i)||/||b|| 7.716325284387e-06
2 KSP preconditioned resid norm 1.418061760018e-13 true resid norm 2.860743141913e-10 ||r(i)||/||b|| 1.230561538720e-10
Linear velocity_ solve converged due to CONVERGED_RTOL iterations 2
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Residual norms for poisson_ solve.
0 KSP preconditioned resid norm 3.424335725400e+00 true resid norm 3.733946078026e-04 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.659151327207e-01 true resid norm 2.770573855571e-05 ||r(i)||/||b|| 7.419962146415e-02
2 KSP preconditioned resid norm 9.652628613282e-03 true resid norm 1.575860242402e-06 ||r(i)||/||b|| 4.220361541041e-03
3 KSP preconditioned resid norm 6.766395999128e-04 true resid norm 8.952331318965e-08 ||r(i)||/||b|| 2.397552383429e-04
4 KSP preconditioned resid norm 4.812796694023e-05 true resid norm 5.719272397815e-09 ||r(i)||/||b|| 1.531696569341e-05
5 KSP preconditioned resid norm 2.911550268135e-06 true resid norm 4.528520225057e-10 ||r(i)||/||b|| 1.212797434786e-06
6 KSP preconditioned resid norm 2.676930489338e-07 true resid norm 2.976304875449e-11 ||r(i)||/||b|| 7.970936947817e-08
7 KSP preconditioned resid norm 1.556661791263e-08 true resid norm 2.106196187333e-12 ||r(i)||/||b|| 5.640671138042e-09
Linear poisson_ solve converged due to CONVERGED_RTOL iterations 7
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
MG: type is MULTIPLICATIVE, levels=3 cycles=v
Cycles per PCApply=1
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
Coarse grid solver -- level -------------------------------
KSP Object: (poisson_mg_coarse_) 2 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_) 2 MPI processes
type: bjacobi
block Jacobi: number of blocks = 2
Local solve is same for all blocks, in the following KSP and PC objects:
KSP Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: lu
LU: out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5., needed 1.09231
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
package used to perform factorization: petsc
total: nonzeros=142, allocated nonzeros=142
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 4 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 7 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 7 nodes, limit used is 5
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (poisson_mg_levels_1_) 2 MPI processes
type: gmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-30
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_1_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=62, cols=62
total: nonzeros=556, allocated nonzeros=556
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (poisson_mg_levels_2_) 2 MPI processes
type: gmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-30
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_2_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
[time-step 1] Writing fluxes into file... done.
[time-step 1] Writing pressure into file... done.
=====================
*** PetIBM - Done ***
=====================
#PETSc Option Table entries:
-options_left
-poisson_ksp_atol 0.0
-poisson_ksp_converged_reason
-poisson_ksp_max_it 20000
-poisson_ksp_monitor_true_residual
-poisson_ksp_rtol 1.0E-08
-poisson_ksp_type cg
-poisson_ksp_view
-poisson_mg_levels_esteig_ksp_max_it 10
-poisson_mg_levels_esteig_ksp_type cg
-poisson_mg_levels_ksp_chebyshev_esteig 0,0.1,0,1.05
-poisson_mg_levels_ksp_type gmres
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_gamg_type agg
-poisson_pc_type gamg
-velocity_ksp_atol 0.0
-velocity_ksp_converged_reason
-velocity_ksp_max_it 10000
-velocity_ksp_monitor_true_residual
-velocity_ksp_rtol 1.0E-08
-velocity_ksp_type bcgs
-velocity_ksp_view
-velocity_pc_type jacobi
#End of PETSc Option Table entries
There are 3 unused database options. They are:
Option left: name:-poisson_mg_levels_esteig_ksp_max_it value: 10
Option left: name:-poisson_mg_levels_esteig_ksp_type value: cg
Option left: name:-poisson_mg_levels_ksp_chebyshev_esteig value: 0,0.1,0,1.05
GAMG specific options
[0] PCSetUp_GAMG(): level 0) N=400, n data rows=1, n data cols=1, nnz/row (ave)=5, np=2
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 4.8 nnz ave. (N=400)
[0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
[0] PCGAMGProlongator_AGG(): New grid 62 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.977326e+00 min=3.920450e-02 PC=jacobi
[0] PCGAMGCreateLevel_GAMG(): Number of equations (loc) 30 with simple aggregation
[0] PCSetUp_GAMG(): 1) N=62, n data cols=1, nnz/row (ave)=8, 1 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 8.96774 nnz ave. (N=62)
[0] PCGAMGProlongator_AGG(): New grid 12 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.556629e+00 min=1.967905e-03 PC=jacobi
[0] PCSetUp_GAMG(): 2) N=12, n data cols=1, nnz/row (ave)=10, 1 active pes
[0] PCSetUp_GAMG(): 3 levels, grid complexity = 1.35729
GAMG specific options
======================
*** PetIBM - Start ***
======================
directory: .
Parsing file ./cartesianMesh.yaml... done.
Parsing file ./flowDescription.yaml... done.
Parsing file ./simulationParameters.yaml... done.
---------------------------------------
Cartesian grid
---------------------------------------
number of cells: 20 x 20
---------------------------------------
---------------------------------------
Flow
---------------------------------------
dimensions: 2
viscosity: 0.01
initial velocity field:
0.
0.
boundary conditions (component, type, value):
->location: xMinus (left)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: xPlus (right)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yMinus (bottom)
0 DIRICHLET 0.
1 DIRICHLET 0.
->location: yPlus (top)
0 DIRICHLET 1.
1 DIRICHLET 0.
---------------------------------------
---------------------------------------
Time-stepping
---------------------------------------
formulation: Navier-Stokes solver (Perot, 1993)
convection: Euler-explicit
diffusion: Euler-implicit
time-increment: 0.0005
starting time-step: 0
number of time-steps: 1
saving-interval: 1
---------------------------------------
----------------------------------------
KSP info: Velocity system
----------------------------------------
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
PC has not been set up so information may be incomplete
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
----------------------------------------
KSP info: Poisson system
----------------------------------------
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using DEFAULT norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
PC has not been set up so information may be incomplete
MG: type is MULTIPLICATIVE, levels=0 cycles=unknown
Cycles per PCApply=0
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
[time-step 0] Writing grid into file... done.
Residual norms for velocity_ solve.
0 KSP preconditioned resid norm 1.148589971947e-03 true resid norm 2.324746103222e+00 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 8.875846864181e-09 true resid norm 1.793849713607e-05 ||r(i)||/||b|| 7.716325284387e-06
2 KSP preconditioned resid norm 1.418061760018e-13 true resid norm 2.860743141913e-10 ||r(i)||/||b|| 1.230561538720e-10
Linear velocity_ solve converged due to CONVERGED_RTOL iterations 2
KSP Object:(velocity_) 2 MPI processes
type: bcgs
maximum iterations=10000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(velocity_) 2 MPI processes
type: jacobi
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=760, cols=760
total: nonzeros=3644, allocated nonzeros=3800
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Residual norms for poisson_ solve.
0 KSP preconditioned resid norm 3.207494894887e+00 true resid norm 3.733946078026e-04 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.665754589097e-01 true resid norm 4.745469052195e-05 ||r(i)||/||b|| 1.270899191641e-01
2 KSP preconditioned resid norm 1.340279285368e-02 true resid norm 2.487245167918e-06 ||r(i)||/||b|| 6.661170557751e-03
3 KSP preconditioned resid norm 9.327267911746e-04 true resid norm 1.104982045593e-07 ||r(i)||/||b|| 2.959287634324e-04
4 KSP preconditioned resid norm 6.260935918573e-05 true resid norm 7.047641929772e-09 ||r(i)||/||b|| 1.887451447477e-05
5 KSP preconditioned resid norm 4.524061727977e-06 true resid norm 4.788798715978e-10 ||r(i)||/||b|| 1.282503446999e-06
6 KSP preconditioned resid norm 3.220602857052e-07 true resid norm 4.345347247124e-11 ||r(i)||/||b|| 1.163741295756e-07
7 KSP preconditioned resid norm 1.968010362397e-08 true resid norm 3.380926909350e-12 ||r(i)||/||b|| 9.054568112932e-09
Linear poisson_ solve converged due to CONVERGED_RTOL iterations 7
KSP Object:(poisson_) 2 MPI processes
type: cg
maximum iterations=20000
tolerances: relative=1e-08, absolute=0., divergence=10000.
left preconditioning
using nonzero initial guess
using PRECONDITIONED norm type for convergence test
PC Object:(poisson_) 2 MPI processes
type: gamg
MG: type is MULTIPLICATIVE, levels=3 cycles=v
Cycles per PCApply=1
Using Galerkin computed coarse grid matrices
GAMG specific options
Threshold for dropping small values from graph 0.
AGG specific options
Symmetric graph false
Coarse grid solver -- level -------------------------------
KSP Object: (poisson_mg_coarse_) 2 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_) 2 MPI processes
type: bjacobi
block Jacobi: number of blocks = 2
Local solve is same for all blocks, in the following KSP and PC objects:
KSP Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (poisson_mg_coarse_sub_) 1 MPI processes
type: lu
LU: out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5., needed 1.09231
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
package used to perform factorization: petsc
total: nonzeros=142, allocated nonzeros=142
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 4 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 7 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=12, cols=12
total: nonzeros=130, allocated nonzeros=130
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 7 nodes, limit used is 5
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (poisson_mg_levels_1_) 2 MPI processes
type: richardson
Richardson: damping factor=1.
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_1_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=62, cols=62
total: nonzeros=556, allocated nonzeros=556
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (poisson_mg_levels_2_) 2 MPI processes
type: richardson
Richardson: damping factor=1.
maximum iterations=2
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (poisson_mg_levels_2_) 2 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=400, cols=400
total: nonzeros=1920, allocated nonzeros=1920
total number of mallocs used during MatSetValues calls =0
has attached null space
not using I-node (on process 0) routines
[time-step 1] Writing fluxes into file... done.
[time-step 1] Writing pressure into file... done.
=====================
*** PetIBM - Done ***
=====================
#PETSc Option Table entries:
-options_left
-poisson_ksp_atol 0.0
-poisson_ksp_converged_reason
-poisson_ksp_max_it 20000
-poisson_ksp_monitor_true_residual
-poisson_ksp_rtol 1.0E-08
-poisson_ksp_type cg
-poisson_ksp_view
-poisson_mg_levels_esteig_ksp_max_it 10
-poisson_mg_levels_esteig_ksp_type cg
-poisson_mg_levels_ksp_chebyshev_esteig 0,0.1,0,1.05
-poisson_mg_levels_ksp_type richardson
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_gamg_type agg
-poisson_pc_type gamg
-velocity_ksp_atol 0.0
-velocity_ksp_converged_reason
-velocity_ksp_max_it 10000
-velocity_ksp_monitor_true_residual
-velocity_ksp_rtol 1.0E-08
-velocity_ksp_type bcgs
-velocity_ksp_view
-velocity_pc_type jacobi
#End of PETSc Option Table entries
There are 3 unused database options. They are:
Option left: name:-poisson_mg_levels_esteig_ksp_max_it value: 10
Option left: name:-poisson_mg_levels_esteig_ksp_type value: cg
Option left: name:-poisson_mg_levels_ksp_chebyshev_esteig value: 0,0.1,0,1.05
GAMG specific options
[0] PCSetUp_GAMG(): level 0) N=400, n data rows=1, n data cols=1, nnz/row (ave)=5, np=2
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 4.8 nnz ave. (N=400)
[0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
[0] PCGAMGProlongator_AGG(): New grid 62 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.977326e+00 min=3.920450e-02 PC=jacobi
[0] PCGAMGCreateLevel_GAMG(): Number of equations (loc) 30 with simple aggregation
[0] PCSetUp_GAMG(): 1) N=62, n data cols=1, nnz/row (ave)=8, 1 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 8.96774 nnz ave. (N=62)
[0] PCGAMGProlongator_AGG(): New grid 12 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.556629e+00 min=1.967905e-03 PC=jacobi
[0] PCSetUp_GAMG(): 2) N=12, n data cols=1, nnz/row (ave)=10, 1 active pes
[0] PCSetUp_GAMG(): 3 levels, grid complexity = 1.35729
GAMG specific options
