We are running with the -ksp_norm_type unpreconditioned so the convergence is done with the true residual for all the previous tests. I have a case with 800 000 nodes that I have run for 200 iterations on 36 CPU cor es (Intel Xeon Gold 6140 - Skylake) and the Poisson solver gives me
| Krylov Solver | Poisson running time [s] | | `cg` | 3.9150E+00 | | `gmres` | 4.6527E+00 | | `bcgs` | 5.4416E+00 | Only the ksp_type has been changed in the following line: mpirun -np $nb_cpu $exec -ksp_initial_guess_nonzero true \ -ksp_type bcgs -ksp_norm_type unpreconditioned \ -ksp_rtol 1e-10 \ -pc_type gamg -mg_levels_pc_type sor -mg_levels_ksp_type richardson \ So CG is better than gmres (I have included the BiCGStab runs as well as I have talked about them earlier). I find it really weird that it behaves well with the preconditioner gamg I can't really find an explanation why, it is really against my intuition. Apart from that I have also played around with the number of multi-grid levels (-pc_mg_levels): | Number of MG levels | Poisson running time [s] | | ------------------------------- | ------------------------ | | 2 | 1.0385E+01 | | 3 | 5.0015E+00 | | 4 | 3.9150E+00 | | 5 | 4.5015E+00 | | 6 (default petsc for this case) | 4.5510E+00 | So that I find an optimum for 4 and not 6 as in the default PETSc configuration and not specifying anything. How should I choose the number of multi grid level depending on my problem? How does GAMG evaluate the number of grid levels required? Lionel De: "Barry Smith" <bsm...@petsc.dev> À: "cheng" <ch...@cerfacs.fr> Cc: "petsc-users" <petsc-users@mcs.anl.gov> Envoyé: Lundi 25 Octobre 2021 15:33:50 Objet: Re: [petsc-users] Convergence on Axisymmetric Poisson matrix Are you running with -ksp_monitor_true_residual to track the b - A*x residual instead of just the preconditioned residual? GAMG definitely does not symmetrize the system but it is possible the preconditioner results in the solve "not seeing" the unsymmetry during the solution process and hence CG still converging; it would be dangerous to rely on this in general I think. You could also run this case with GMRES to see if that is better than the CG iterations. Barry On Oct 24, 2021, at 7:00 PM, Lionel CHENG < [ mailto:ch...@cerfacs.fr | ch...@cerfacs.fr ] > wrote: Hello everyone, I have some questions regarding a linear system that I am solving in my plasma simulations. We have in this case a strongly non-symmetric matrix due to the cylindrical coordinates for which the Laplacian cell is given by Fig. 2 for two kinds of triangles. The different unstructured grids have from 300 000 nodes to 7 000 000 nodes. To my understanding, CG should not work properly on this matrix but BiCGStab(1) should. When using SOR preconditioner it is indeed the case: -ksp_type cg -pc_type sor yields solutions in 10 to 20 times more iterations than -ksp_type bcgs -pc_type sor. However, when switching to -ksp_type cg -pc_type gamg the convergence is great and even slightly better than -ksp_type bcgs. I do not understand how CG is able to make the system converge when using GAMG although the matrix is non-symmetric ? Is GAMG able to somehow symmetrize the system? I have the impression that when using -pc_type gamg the Krylov solver is actually the Pre-relaxation and post-relaxation of the initial grid, is that right? For GAMG since the matrix is non-symmetric -mg_levels_pc_type sor for and -mg_levels_ksp_type richardson have been used and yields better results than the original chebychev solver. Sincerely yours, Lionel Cheng <main.pdf>
