More precisely: something happens when upgrading the functions MatPtAPNumeric_MPIAIJ_MPIAIJ and/or MatPtAPSymbolic_MPIAIJ_MPIAIJ.
Unfortunately, there are a lot of differences between the old and new versions of these functions. I keep investigating but if you have any idea, please let me know. Best, Myriam Le 03/20/19 à 13:48, Myriam Peyrounette a écrit : > > Hi all, > > I used git bisect to determine when the memory need increased. I found > that the first "bad" commit is aa690a28a7284adb519c28cb44eae20a2c131c85. > > Barry was right, this commit seems to be about an evolution of > MatPtAPSymbolic_MPIAIJ_MPIAIJ. You mentioned the option "-matptap_via > scalable" but I can't find any information about it. Can you tell me more? > > Thanks > > Myriam > > > Le 03/11/19 à 14:40, Mark Adams a écrit : >> Is there a difference in memory usage on your tiny problem? I assume no. >> >> I don't see anything that could come from GAMG other than the RAP >> stuff that you have discussed already. >> >> On Mon, Mar 11, 2019 at 9:32 AM Myriam Peyrounette >> <[email protected] <mailto:[email protected]>> wrote: >> >> The code I am using here is the example 42 of PETSc >> >> (https://www.mcs.anl.gov/petsc/petsc-3.9/src/ksp/ksp/examples/tutorials/ex42.c.html). >> Indeed it solves the Stokes equation. I thought it was a good >> idea to use an example you might know (and didn't find any that >> uses GAMG functions). I just changed the PCMG setup so that the >> memory problem appears. And it appears when adding PCGAMG. >> >> I don't care about the performance or even the result rightness >> here, but only about the difference in memory use between 3.6 and >> 3.10. Do you think finding a more adapted script would help? >> >> I used the threshold of 0.1 only once, at the beginning, to test >> its influence. I used the default threshold (of 0, I guess) for >> all the other runs. >> >> Myriam >> >> >> Le 03/11/19 à 13:52, Mark Adams a écrit : >>> In looking at this larger scale run ... >>> >>> * Your eigen estimates are much lower than your tiny test >>> problem. But this is Stokes apparently and it should not work >>> anyway. Maybe you have a small time step that adds a lot of mass >>> that brings the eigen estimates down. And your min eigenvalue >>> (not used) is positive. I would expect negative for Stokes ... >>> >>> * You seem to be setting a threshold value of 0.1 -- that is >>> very high >>> >>> * v3.6 says "using nonzero initial guess" but this is not in >>> v3.10. Maybe we just stopped printing that. >>> >>> * There were some changes to coasening parameters in going from >>> v3.6 but it does not look like your problem was effected. (The >>> coarsening algo is non-deterministic by default and you can see >>> small difference on different runs) >>> >>> * We may have also added a "noisy" RHS for eigen estimates by >>> default from v3.6. >>> >>> * And for non-symetric problems you can try >>> -pc_gamg_agg_nsmooths 0, but again GAMG is not built for Stokes >>> anyway. >>> >>> >>> On Tue, Mar 5, 2019 at 11:53 AM Myriam Peyrounette >>> <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> I used PCView to display the size of the linear system in >>> each level of the MG. You'll find the outputs attached to >>> this mail (zip file) for both the default threshold value >>> and a value of 0.1, and for both 3.6 and 3.10 PETSc versions. >>> >>> For convenience, I summarized the information in a graph, >>> also attached (png file). >>> >>> As you can see, there are slight differences between the two >>> versions but none is critical, in my opinion. Do you see >>> anything suspicious in the outputs? >>> >>> + I can't find the default threshold value. Do you know >>> where I can find it? >>> >>> Thanks for the follow-up >>> >>> Myriam >>> >>> >>> Le 03/05/19 à 14:06, Matthew Knepley a écrit : >>>> On Tue, Mar 5, 2019 at 7:14 AM Myriam Peyrounette >>>> <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hi Matt, >>>> >>>> I plotted the memory scalings using different threshold >>>> values. The two scalings are slightly translated (from >>>> -22 to -88 mB) but this gain is neglectable. The >>>> 3.6-scaling keeps being robust while the 3.10-scaling >>>> deteriorates. >>>> >>>> Do you have any other suggestion? >>>> >>>> Mark, what is the option she can give to output all the >>>> GAMG data? >>>> >>>> Also, run using -ksp_view. GAMG will report all the sizes >>>> of its grids, so it should be easy to see >>>> if the coarse grid sizes are increasing, and also what the >>>> effect of the threshold value is. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> Thanks >>>> >>>> Myriam >>>> >>>> Le 03/02/19 à 02:27, Matthew Knepley a écrit : >>>>> On Fri, Mar 1, 2019 at 10:53 AM Myriam Peyrounette via >>>>> petsc-users <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I used to run my code with PETSc 3.6. Since I >>>>> upgraded the PETSc version >>>>> to 3.10, this code has a bad memory scaling. >>>>> >>>>> To report this issue, I took the PETSc script >>>>> ex42.c and slightly >>>>> modified it so that the KSP and PC configurations >>>>> are the same as in my >>>>> code. In particular, I use a "personnalised" >>>>> multi-grid method. The >>>>> modifications are indicated by the keyword >>>>> "TopBridge" in the attached >>>>> scripts. >>>>> >>>>> To plot the memory (weak) scaling, I ran four >>>>> calculations for each >>>>> script with increasing problem sizes and >>>>> computations cores: >>>>> >>>>> 1. 100,000 elts on 4 cores >>>>> 2. 1 million elts on 40 cores >>>>> 3. 10 millions elts on 400 cores >>>>> 4. 100 millions elts on 4,000 cores >>>>> >>>>> The resulting graph is also attached. The scaling >>>>> using PETSc 3.10 >>>>> clearly deteriorates for large cases, while the >>>>> one using PETSc 3.6 is >>>>> robust. >>>>> >>>>> After a few tests, I found that the scaling is >>>>> mostly sensitive to the >>>>> use of the AMG method for the coarse grid (line >>>>> 1780 in >>>>> main_ex42_petsc36.cc). In particular, the >>>>> performance strongly >>>>> deteriorates when commenting lines 1777 to 1790 >>>>> (in main_ex42_petsc36.cc). >>>>> >>>>> Do you have any idea of what changed between >>>>> version 3.6 and version >>>>> 3.10 that may imply such degradation? >>>>> >>>>> >>>>> I believe the default values for PCGAMG changed >>>>> between versions. It sounds like the coarsening rate >>>>> is not great enough, so that these grids are too >>>>> large. This can be set using: >>>>> >>>>> >>>>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGAMGSetThreshold.html >>>>> >>>>> There is some explanation of this effect on that page. >>>>> Let us know if setting this does not correct the >>>>> situation. >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>> Let me know if you need further information. >>>>> >>>>> Best, >>>>> >>>>> Myriam Peyrounette >>>>> >>>>> >>>>> -- >>>>> Myriam Peyrounette >>>>> CNRS/IDRIS - HLST >>>>> -- >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they >>>>> begin their experiments is infinitely more interesting >>>>> than any results to which their experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> https://www.cse.buffalo.edu/~knepley/ >>>>> <http://www.cse.buffalo.edu/%7Eknepley/> >>>> >>>> -- >>>> Myriam Peyrounette >>>> CNRS/IDRIS - HLST >>>> -- >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin >>>> their experiments is infinitely more interesting than any >>>> results to which their experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> <http://www.cse.buffalo.edu/%7Eknepley/> >>> >>> -- >>> Myriam Peyrounette >>> CNRS/IDRIS - HLST >>> -- >>> >> >> -- >> Myriam Peyrounette >> CNRS/IDRIS - HLST >> -- >> > > -- > Myriam Peyrounette > CNRS/IDRIS - HLST > -- -- Myriam Peyrounette CNRS/IDRIS - HLST --
smime.p7s
Description: Signature cryptographique S/MIME
