Hello Mark,
Thanks for your answer. Indeed, I didn't see the information that classical AMG
was not really supported:
-solver2_pc_gamg_type <now classical : formerly agg>: Type of AMG method (only
'agg' supported and useful) (one of) classical geo agg (PCGAMGSetType)
We switched very recently from GAMG("agg") to GAMG("classical") for a weak
scaling test up to 32000 cores, where we saw very good scalability with
GAMG("classical") compared to GAMG("agg"). But it was with PETSc 3.14...
So today, we are going to upgrade to 3.20 and focus on GAMG("agg") or Hypre
Classical AMG. We will see how it compares.
May I ask you what is your point of view of the current state of the GPU
versions of GAMG("agg") versus Hypre AMG Classical ?
In fact, the reason of our move from 3.14 to 3.20 is to take advantage of all
the progress in PETSc and Hypre on accelerated solvers/preconditioners during
the last 2 years.
Greatly appreciate your help,
Pierre LEDAC
Commissariat à l’énergie atomique et aux énergies alternatives
Centre de SACLAY
DES/ISAS/DM2S/SGLS/LCAN
Bâtiment 451 – point courrier n°43
F-91191 Gif-sur-Yvette
+33 1 69 08 04 03
+33 6 83 42 05 79
________________________________
De : Mark Adams <[email protected]>
Envoyé : mercredi 13 décembre 2023 20:54:17
À : LEDAC Pierre
Cc : [email protected]; BRUNETON Adrien
Objet : Re: [petsc-users] PETSc 3.14 to PETSc 3.20: Different (slower)
convergence for classical AMG (sequential and especially in parallel)
Hi Pierre,
Sorry I missed this post and your issues were brought to my attention today.
First, the classic version is not supported well. The postdoc that wrote the
code is long gone and I don't know the code at all.
It is really a reference implementation that someone could build on and is not
meant for production.
In 10 years you are the first user that has connected us.
The hypre package is a very good AMG solver and it uses classical AMG as the
main solver.
I wrote GAMG ("agg") which is a smoothed aggregation AMG solver and is very
different from classical.
I would suggest you move to hypre or '-pc_gamg_type agg'.
The coarsening was developed in this time frame and there was a lot of churn as
a new strategy for aggressive coarsening did not work well for some users and I
had to add the old method in and then made it the default (again).
This change missed v3.20, but you can get the old aggressive strategy with
'-pc_gamg_aggressive_square_graph'.
Check with -options_left to check that it is being used.
As far as your output (nice formatting, thank you), the coarse grid is smaller
in the new code.
rows=41, cols=41 | rows=30, cols=30
"square graph" should fix this.
You can also try not using aggressive coarsening with:
You could try '-pc_gamg_aggressive_coarsening 0'
Let me know how it goes and let's try to get you into a more sustainable state
... I really try not to change this code but sometimes need to.
Thanks,
Mark
On Mon, Oct 9, 2023 at 10:43 AM LEDAC Pierre
<[email protected]<mailto:[email protected]>> wrote:
Hello all,
I am struggling to find the same convergence in iterations when using classical
algebric multigrid in my code with PETSc 3.20 compared to PETSc 3.14.
I am using in order to solve a Poisson system:
-ksp_type cg -pc_type gamg -pc_gamg_type classical
I read the different releases notes between 3.15 and 3.20:
https://petsc.org/release/changes/317
https://petsc.org/main/manualpages/PC/PCGAMGSetThreshold/
And have a look at the archive mailing list (especially this one:
https://www.mail-archive.com/[email protected]/msg46688.html)
so I added some other options to try to have the same behaviour than PETSc 3.14:
-ksp_type cg -pc_type gamg -pc_gamg_type classical -mg_levels_pc_type sor
-pc_gamg_threshold 0.
It improves the convergence but there still a different convergence though (26
vs 18 iterations).
On another of my test case, the number of levels is different (e.g. 6 vs 4)
also, and here it is the same, but with a different coarsening according to the
output from the -ksp_view option
The main point is that the convergence dramatically degrades in parallel on a
third test case, so I can't upgrade to PETSc 3.20 for now unhappily.
I send you the partial report (petsc_314_vs_petsc_320.ksp_view) with -ksp_view
(left PETSc 3.14, right PETSc 3.20) and the configure/command line options used
(in petsc_XXX_petsc.TU files).
Could my issue related to the following 3.18 change ? I have not tried the
first one.
* Remove PCGAMGSetSymGraph() and -pc_gamg_sym_graph. The user should now
indicate symmetry and structural symmetry using
MatSetOption<https://petsc.org/release/manualpages/Mat/MatSetOption/>() and
GAMG will symmetrize the graph if a symmetric options is not set
* Change -pc_gamg_reuse_interpolation default from false to true.
Any advice would be greatly appreciated,
Pierre LEDAC
Commissariat à l’énergie atomique et aux énergies alternatives
Centre de SACLAY
DES/ISAS/DM2S/SGLS/LCAN
Bâtiment 451 – point courrier n°43
F-91191 Gif-sur-Yvette
+33 1 69 08 04 03
+33 6 83 42 05 79
