Increasing the threshold should increase the size of the coarse grids, but yours are decreasing. I'm puzzled by that.
On Tue, Mar 5, 2019 at 11:53 AM Myriam Peyrounette < myriam.peyroune...@idris.fr> wrote: > I used PCView to display the size of the linear system in each level of > the MG. You'll find the outputs attached to this mail (zip file) for both > the default threshold value and a value of 0.1, and for both 3.6 and 3.10 > PETSc versions. > > For convenience, I summarized the information in a graph, also attached > (png file). > > As you can see, there are slight differences between the two versions but > none is critical, in my opinion. Do you see anything suspicious in the > outputs? > > + I can't find the default threshold value. Do you know where I can find > it? > > Thanks for the follow-up > > Myriam > > Le 03/05/19 à 14:06, Matthew Knepley a écrit : > > On Tue, Mar 5, 2019 at 7:14 AM Myriam Peyrounette < > myriam.peyroune...@idris.fr> wrote: > >> Hi Matt, >> >> I plotted the memory scalings using different threshold values. The two >> scalings are slightly translated (from -22 to -88 mB) but this gain is >> neglectable. The 3.6-scaling keeps being robust while the 3.10-scaling >> deteriorates. >> >> Do you have any other suggestion? >> > Mark, what is the option she can give to output all the GAMG data? > > Also, run using -ksp_view. GAMG will report all the sizes of its grids, so > it should be easy to see > if the coarse grid sizes are increasing, and also what the effect of the > threshold value is. > > Thanks, > > Matt > >> Thanks >> Myriam >> >> Le 03/02/19 à 02:27, Matthew Knepley a écrit : >> >> On Fri, Mar 1, 2019 at 10:53 AM Myriam Peyrounette via petsc-users < >> petsc-users@mcs.anl.gov> wrote: >> >>> Hi, >>> >>> I used to run my code with PETSc 3.6. Since I upgraded the PETSc version >>> to 3.10, this code has a bad memory scaling. >>> >>> To report this issue, I took the PETSc script ex42.c and slightly >>> modified it so that the KSP and PC configurations are the same as in my >>> code. In particular, I use a "personnalised" multi-grid method. The >>> modifications are indicated by the keyword "TopBridge" in the attached >>> scripts. >>> >>> To plot the memory (weak) scaling, I ran four calculations for each >>> script with increasing problem sizes and computations cores: >>> >>> 1. 100,000 elts on 4 cores >>> 2. 1 million elts on 40 cores >>> 3. 10 millions elts on 400 cores >>> 4. 100 millions elts on 4,000 cores >>> >>> The resulting graph is also attached. The scaling using PETSc 3.10 >>> clearly deteriorates for large cases, while the one using PETSc 3.6 is >>> robust. >>> >>> After a few tests, I found that the scaling is mostly sensitive to the >>> use of the AMG method for the coarse grid (line 1780 in >>> main_ex42_petsc36.cc). In particular, the performance strongly >>> deteriorates when commenting lines 1777 to 1790 (in >>> main_ex42_petsc36.cc). >>> >>> Do you have any idea of what changed between version 3.6 and version >>> 3.10 that may imply such degradation? >>> >> >> I believe the default values for PCGAMG changed between versions. It >> sounds like the coarsening rate >> is not great enough, so that these grids are too large. This can be set >> using: >> >> >> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGAMGSetThreshold.html >> >> There is some explanation of this effect on that page. Let us know if >> setting this does not correct the situation. >> >> Thanks, >> >> Matt >> >> >>> Let me know if you need further information. >>> >>> Best, >>> >>> Myriam Peyrounette >>> >>> >>> -- >>> Myriam Peyrounette >>> CNRS/IDRIS - HLST >>> -- >>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/%7Eknepley/> >> >> >> -- >> Myriam Peyrounette >> CNRS/IDRIS - HLST >> -- >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/%7Eknepley/> > > > -- > Myriam Peyrounette > CNRS/IDRIS - HLST > -- > >