Re: [petsc-users] PETSc testing recipes

[Alexander Lindsay]

Thank you all for the info!

> On Apr 14, 2023, at 3:48 PM, Jacob Faibussowitsch  wrote:
> 
> More specifically for configure handling of C++ dialect. It has 3 modes:
> 
> 1. You don’t specify any C++ dialect (i.e. configure without passing 
> `--with-cxx-dialect`):
> 
> Configure will determine the dialect range for your given setup. It takes 
> into account all of the compilers (including “extra” ones like nvcc) it 
> finds, as well as all of your requested packages. It will also enable GNU 
> extensions if they are available, i.e. passing `-std=gnu++17` over 
> `-std=c++17`. From this range, it will select the highest possible dialect 
> and apply it to all builds.
> 
> 2. You specify `--with-cxx-dialect=14`:
> 
> Configure will do the above but will impose an arbitrary cap of C++14. If 
> something in your environment conflicts with this (i.e. maybe your compiler 
> doesn’t support it, or maybe a package requires C++17) it will produce a hard 
> error telling you why this won’t work. It still attempts to enable GNU 
> extensions as above.
> 
> 3. You specify `--with-cxx-dialect=c++14` OR you have `-std=c++14` in 
> `[--]CXXFLAGS` and/or `[--]CXXOPTFLAGS`:
> 
> configure assumes you know something it doesn’t. It will blindly apply 
> exactly `-std=c++14` everywhere. This includes building PETSc itself and all 
> packages that PETSc builds. It will still check your compiler supports 
> `-std=c++14` though (and error if it does not).
> 
> So if your goal is to have the highest available dialect chosen for you, do 
> nothing. Do not pass any dialect flags, and do not set them in your 
> `CXXFLAGS` or `CXXOPTFLAGS`. Configure will do The Right Thing.
> 
> If your goal is to explicitly set the dialect, we recommend using option 2 
> since that might still allow you to enable GNU extensions. 
> 
> If something is just not working then 1. send us your configure.log — maybe 
> we have a bug — and 2. use option #3, it will do exactly as you ask.
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> 
>> On Apr 14, 2023, at 18:18, Satish Balay via petsc-users 
>>  wrote:
>> 
>> Hm depending on what you are seeking - you'll have to look at multiple 
>> places [and co-related the info from them]
>> 
>> - you can check one of the pipelines to see exactly what jobs are run (and 
>> the logs of the jobs for details). for ex:
>> https://gitlab.com/petsc/petsc/-/pipelines/836609909
>> 
>> - most of the build scripts are at config/examples/arch-ci-*.py - so thats 
>> one way to get a view of what pkgs are tested by what jobs..
>> 
>> balay@p1 /home/balay/petsc (main =)
>> $ ls config/examples/arch-ci-*.py |wc -l
>> 56
>> 
>> - And wrt specific examples corresponding to external pkg - say - 
>> superlu_dist - you can do something like:
>> 
>> balay@p1 /home/balay/petsc (main =)
>> $ git grep 'requires:' src |grep superlu_dist |head -5
>> src/ksp/ksp/tests/ex17.c:  requires: superlu_dist complex
>> src/ksp/ksp/tests/ex17.c:  requires: superlu_dist complex
>> src/ksp/ksp/tests/ex33.c:  requires: superlu_dist !complex
>> src/ksp/ksp/tests/ex33.c:  requires: superlu_dist !complex
>> src/ksp/ksp/tests/ex49.c:  requires: superlu_dist
>> 
>> etc.
>> 
>> - Wrt C++ dialect:
>> 
>> balay@p1 /home/balay/petsc (main =)
>> $ git grep self.minCxxVersion config/BuildSystem/config/packages
>> config/BuildSystem/config/packages/AMReX.py:self.minCxxVersion = 
>> 'c++14'
>> config/BuildSystem/config/packages/h2opus.py:self.minCxxVersion  
>> = 'c++14'
>> config/BuildSystem/config/packages/kokkos.py:self.minCxxVersion= 
>> 'c++17'
>> config/BuildSystem/config/packages/raja.py:self.minCxxVersion  = 
>> 'c++14'
>> config/BuildSystem/config/packages/sycl.py:self.minCxxVersion= 
>> 'c++17'
>> balay@p1 /home/balay/petsc (main =)
>> $ git grep self.maxCxxVersion config/BuildSystem/config/packages
>> config/BuildSystem/config/packages/MOAB.py:self.maxCxxVersion = 
>> 'c++14'
>> config/BuildSystem/config/packages/amgx.py:self.maxCxxVersion= 
>> 'c++17' # https://github.com/NVIDIA/AMGX/issues/231
>> config/BuildSystem/config/packages/elemental.py:self.maxCxxVersion= 
>> 'c++14'
>> config/BuildSystem/config/packages/grid.py:self.maxCxxVersion  = 'c++17'
>> config/BuildSystem/config/packages/kokkos.py:self.maxCxxVersion= 
>> 'c++17'
>> 
>> However configure determines the max that the compiler supports and
>> attempts to use that [when only min is set]. And this info is usually
>> in configure.log [or make.log] for the corresponding build.
>> 
>> Satish
>> 
>> 
>>> On Fri, 14 Apr 2023, Alexander Lindsay wrote:
>>> 
>>> Hi, is there a place I can look to understand the testing recipes used in
>>> PETSc CI, e.g. what external packages are included (if any), what C++
>>> dialect is used for any external packages built with C++, etc.?
>>> 
>>> Alex
>>> 
>> 
> 

Re: [petsc-users] PETSc testing recipes

[Jacob Faibussowitsch]

More specifically for configure handling of C++ dialect. It has 3 modes:

1. You don’t specify any C++ dialect (i.e. configure without passing 
`--with-cxx-dialect`):

Configure will determine the dialect range for your given setup. It takes into 
account all of the compilers (including “extra” ones like nvcc) it finds, as 
well as all of your requested packages. It will also enable GNU extensions if 
they are available, i.e. passing `-std=gnu++17` over `-std=c++17`. From this 
range, it will select the highest possible dialect and apply it to all builds.

2. You specify `--with-cxx-dialect=14`:

Configure will do the above but will impose an arbitrary cap of C++14. If 
something in your environment conflicts with this (i.e. maybe your compiler 
doesn’t support it, or maybe a package requires C++17) it will produce a hard 
error telling you why this won’t work. It still attempts to enable GNU 
extensions as above.

3. You specify `--with-cxx-dialect=c++14` OR you have `-std=c++14` in 
`[--]CXXFLAGS` and/or `[--]CXXOPTFLAGS`:

configure assumes you know something it doesn’t. It will blindly apply exactly 
`-std=c++14` everywhere. This includes building PETSc itself and all packages 
that PETSc builds. It will still check your compiler supports `-std=c++14` 
though (and error if it does not).

So if your goal is to have the highest available dialect chosen for you, do 
nothing. Do not pass any dialect flags, and do not set them in your `CXXFLAGS` 
or `CXXOPTFLAGS`. Configure will do The Right Thing.

If your goal is to explicitly set the dialect, we recommend using option 2 
since that might still allow you to enable GNU extensions. 

If something is just not working then 1. send us your configure.log — maybe we 
have a bug — and 2. use option #3, it will do exactly as you ask.

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)

> On Apr 14, 2023, at 18:18, Satish Balay via petsc-users 
>  wrote:
> 
> Hm depending on what you are seeking - you'll have to look at multiple places 
> [and co-related the info from them]
> 
> - you can check one of the pipelines to see exactly what jobs are run (and 
> the logs of the jobs for details). for ex:
> https://gitlab.com/petsc/petsc/-/pipelines/836609909
> 
> - most of the build scripts are at config/examples/arch-ci-*.py - so thats 
> one way to get a view of what pkgs are tested by what jobs..
> 
> balay@p1 /home/balay/petsc (main =)
> $ ls config/examples/arch-ci-*.py |wc -l
> 56
> 
> - And wrt specific examples corresponding to external pkg - say - 
> superlu_dist - you can do something like:
> 
> balay@p1 /home/balay/petsc (main =)
> $ git grep 'requires:' src |grep superlu_dist |head -5
> src/ksp/ksp/tests/ex17.c:  requires: superlu_dist complex
> src/ksp/ksp/tests/ex17.c:  requires: superlu_dist complex
> src/ksp/ksp/tests/ex33.c:  requires: superlu_dist !complex
> src/ksp/ksp/tests/ex33.c:  requires: superlu_dist !complex
> src/ksp/ksp/tests/ex49.c:  requires: superlu_dist
> 
> etc.
> 
> - Wrt C++ dialect:
> 
> balay@p1 /home/balay/petsc (main =)
> $ git grep self.minCxxVersion config/BuildSystem/config/packages
> config/BuildSystem/config/packages/AMReX.py:self.minCxxVersion = 
> 'c++14'
> config/BuildSystem/config/packages/h2opus.py:self.minCxxVersion  
> = 'c++14'
> config/BuildSystem/config/packages/kokkos.py:self.minCxxVersion= 
> 'c++17'
> config/BuildSystem/config/packages/raja.py:self.minCxxVersion  = 
> 'c++14'
> config/BuildSystem/config/packages/sycl.py:self.minCxxVersion= 'c++17'
> balay@p1 /home/balay/petsc (main =)
> $ git grep self.maxCxxVersion config/BuildSystem/config/packages
> config/BuildSystem/config/packages/MOAB.py:self.maxCxxVersion = 
> 'c++14'
> config/BuildSystem/config/packages/amgx.py:self.maxCxxVersion= 
> 'c++17' # https://github.com/NVIDIA/AMGX/issues/231
> config/BuildSystem/config/packages/elemental.py:self.maxCxxVersion= 
> 'c++14'
> config/BuildSystem/config/packages/grid.py:self.maxCxxVersion  = 'c++17'
> config/BuildSystem/config/packages/kokkos.py:self.maxCxxVersion= 
> 'c++17'
> 
> However configure determines the max that the compiler supports and
> attempts to use that [when only min is set]. And this info is usually
> in configure.log [or make.log] for the corresponding build.
> 
> Satish
> 
> 
> On Fri, 14 Apr 2023, Alexander Lindsay wrote:
> 
>> Hi, is there a place I can look to understand the testing recipes used in
>> PETSc CI, e.g. what external packages are included (if any), what C++
>> dialect is used for any external packages built with C++, etc.?
>> 
>> Alex
>> 
> 

Re: [petsc-users] PETSc testing recipes

[Satish Balay via petsc-users]

Hm depending on what you are seeking - you'll have to look at multiple places 
[and co-related the info from them]

 - you can check one of the pipelines to see exactly what jobs are run (and the 
logs of the jobs for details). for ex:
https://gitlab.com/petsc/petsc/-/pipelines/836609909

 - most of the build scripts are at config/examples/arch-ci-*.py - so thats one 
way to get a view of what pkgs are tested by what jobs..

balay@p1 /home/balay/petsc (main =)
$ ls config/examples/arch-ci-*.py |wc -l
56

- And wrt specific examples corresponding to external pkg - say - superlu_dist 
- you can do something like:

balay@p1 /home/balay/petsc (main =)
$ git grep 'requires:' src |grep superlu_dist |head -5
src/ksp/ksp/tests/ex17.c:  requires: superlu_dist complex
src/ksp/ksp/tests/ex17.c:  requires: superlu_dist complex
src/ksp/ksp/tests/ex33.c:  requires: superlu_dist !complex
src/ksp/ksp/tests/ex33.c:  requires: superlu_dist !complex
src/ksp/ksp/tests/ex49.c:  requires: superlu_dist

etc.

- Wrt C++ dialect:

balay@p1 /home/balay/petsc (main =)
$ git grep self.minCxxVersion config/BuildSystem/config/packages
config/BuildSystem/config/packages/AMReX.py:self.minCxxVersion = 'c++14'
config/BuildSystem/config/packages/h2opus.py:self.minCxxVersion  = 
'c++14'
config/BuildSystem/config/packages/kokkos.py:self.minCxxVersion= 'c++17'
config/BuildSystem/config/packages/raja.py:self.minCxxVersion  = 'c++14'
config/BuildSystem/config/packages/sycl.py:self.minCxxVersion= 'c++17'
balay@p1 /home/balay/petsc (main =)
$ git grep self.maxCxxVersion config/BuildSystem/config/packages
config/BuildSystem/config/packages/MOAB.py:self.maxCxxVersion = 'c++14'
config/BuildSystem/config/packages/amgx.py:self.maxCxxVersion= 'c++17' 
# https://github.com/NVIDIA/AMGX/issues/231
config/BuildSystem/config/packages/elemental.py:self.maxCxxVersion= 
'c++14'
config/BuildSystem/config/packages/grid.py:self.maxCxxVersion  = 'c++17'
config/BuildSystem/config/packages/kokkos.py:self.maxCxxVersion= 'c++17'

However configure determines the max that the compiler supports and
attempts to use that [when only min is set]. And this info is usually
in configure.log [or make.log] for the corresponding build.

Satish


On Fri, 14 Apr 2023, Alexander Lindsay wrote:

> Hi, is there a place I can look to understand the testing recipes used in
> PETSc CI, e.g. what external packages are included (if any), what C++
> dialect is used for any external packages built with C++, etc.?
> 
> Alex
> 

Re: [petsc-users] PETSc testing recipes

[Jed Brown]

Look at config/examples/arch-ci-*.py for the configurations. They're driven 
from .gitlab-ci.yml

Alexander Lindsay  writes:

> Hi, is there a place I can look to understand the testing recipes used in
> PETSc CI, e.g. what external packages are included (if any), what C++
> dialect is used for any external packages built with C++, etc.?
>
> Alex

[petsc-users] PETSc testing recipes

[Alexander Lindsay]

Hi, is there a place I can look to understand the testing recipes used in
PETSc CI, e.g. what external packages are included (if any), what C++
dialect is used for any external packages built with C++, etc.?

Alex

Re: [petsc-users] Issue with SNES Solver in Fortran - Converging with Zero Residual

[Barry Smith]

   Likely there is an issue with the FormFunction you are providing. For a 
small problem you can call VecView() on the input vector at the start of your 
routine
and then VecView() on the output vector at the end of your routine. This might 
provide some insight. Since the output vector is presumably all zero you can 
try 
running in the debugger and trace through the second call to your FormFunction 
to see why it generating exactly zero output when it presumably should not be.

  Barry


> On Apr 14, 2023, at 12:39 PM, Khaled Nabil Shar Abdelaziz 
>  wrote:
> 
> Hello,
> I hope you are well. I am currently working with the SNES solver in Fortran, 
> utilizing Newton's method. I have provided both the residual function and the 
> Jacobian. However, I have encountered an issue that I hope you can help me 
> with.
>  
> In the first non-linear iteration, the solver calculates a non-zero residual 
> value and starts the KSP solver to minimize the error. However, in the second 
> non-linear iteration, it returns an exact zero residual, considering it 
> converged.
>  
> As a result, the first few steps provide a somewhat accurate solution, but 
> after around 15 steps, the solution starts diverging rapidly. I assume this 
> is due to accumulating errors, as the residual in subsequent steps becomes 
> increasingly larger.
>  
> Here are some outputs from the solver:
>  
> start_SNES_petsc_solver: var_nd%name=u
>   0 SNES Function norm 3.412918650183e+01 
>  Attempt!
> 0 KSP Residual norm 5.861642176595e-01 
> 1 KSP Residual norm 1.411858179645e-01 
> 2 KSP Residual norm 1.388288156571e-01 
> 3 KSP Residual norm 4.900215159087e-02 
> 4 KSP Residual norm 2.559630070894e-02 
> 5 KSP Residual norm 1.007110997387e-02 
> 6 KSP Residual norm 6.371175598940e-03 
>   1 SNES Function norm 0.e+00 
>  iteration=   1
> SNESConvergedReason=   2
>  
>  
> istep=   1
> **
>  
> start_SNES_petsc_solver: var_nd%name=u
>   0 SNES Function norm 6.831896456736e+01 
>  Attempt!
> 0 KSP Residual norm 5.850729730568e+00 
> 1 KSP Residual norm 5.176135972454e+00 
> 2 KSP Residual norm 5.106059774079e-01 
> 3 KSP Residual norm 2.058573608172e-01 
> 4 KSP Residual norm 8.430267458444e-02 
> 5 KSP Residual norm 2.421049820170e-02 
> 6 KSP Residual norm 1.387479046692e-02 
> 7 KSP Residual norm 6.556624109622e-03 
>   1 SNES Function norm 0.e+00 
>  iteration=   1
> SNESConvergedReason=   2
>  
> istep=   2
> **
>  
> start_SNES_petsc_solver: var_nd%name=u
>   0 SNES Function norm 1.024330722398e+02 
>  Attempt!
> 0 KSP Residual norm 1.324263482159e+01 
> 1 KSP Residual norm 8.772774639367e+00 
> 2 KSP Residual norm 8.717824082000e-01 
> 3 KSP Residual norm 3.965086318719e-01 
> 4 KSP Residual norm 1.380063519887e-01 
> 5 KSP Residual norm 3.983781619335e-02 
> 6 KSP Residual norm 1.690524902818e-02 
> 7 KSP Residual norm 1.371656480592e-02 
> 8 KSP Residual norm 4.598826535286e-03 
>   1 SNES Function norm 0.e+00 
>  iteration=   1
> SNESConvergedReason=   2
>  
> istep=   3
> **
>  
> start_SNES_petsc_solver: var_nd%name=u
>   0 SNES Function norm 1.371337889713e+02 
>  Attempt!
> 0 KSP Residual norm 1.823603533770e+01 
> 1 KSP Residual norm 1.823546028484e+01 
> 2 KSP Residual norm 8.167385988622e-01 
> 3 KSP Residual norm 1.930526020067e-01 
> 4 KSP Residual norm 1.768873013055e-01 
> 5 KSP Residual norm 2.692456250466e-02 
> 6 KSP Residual norm 1.130530545617e-02 
> 7 KSP Residual norm 6.934825615412e-03 
>   1 SNES Function norm 0.e+00 
>  iteration=   1
> SNESConvergedReason=   2
>  
> istep=   4
> **
>  
> start_SNES_petsc_solver: var_nd%name=u
>   0 SNES Function norm 1.717647834464e+02 
>  Attempt!
> 0 KSP Residual norm 1.015846744437e+02 
> 1 KSP Residual norm 7.622793741160e+00 
> 2 KSP Residual norm 1.381803723895e+00 
> 3 KSP Residual norm 1.305423467184e-01 
> 4 KSP Residual norm 3.606398975875e-02 
> 5 KSP Residual norm 3.463999556864e-02 
> 6 KSP Residual norm 3.268229989659e-02 
> 7 KSP Residual norm 8.847393497789e-03 
>   1 SNES Function norm 0.e+00 
>  iteration=   1
> SNESConvergedReason=   2
>  
> Do you have any idea what might be causing this behavior? I appreciate any 
> insights you might have.
>  
> Best,
> Khaled

[petsc-users] Issue with SNES Solver in Fortran - Converging with Zero Residual

[Khaled Nabil Shar Abdelaziz]

Hello,
I hope you are well. I am currently working with the SNES solver in Fortran, 
utilizing Newton's method. I have provided both the residual function and the 
Jacobian. However, I have encountered an issue that I hope you can help me with.

In the first non-linear iteration, the solver calculates a non-zero residual 
value and starts the KSP solver to minimize the error. However, in the second 
non-linear iteration, it returns an exact zero residual, considering it 
converged.

As a result, the first few steps provide a somewhat accurate solution, but 
after around 15 steps, the solution starts diverging rapidly. I assume this is 
due to accumulating errors, as the residual in subsequent steps becomes 
increasingly larger.

Here are some outputs from the solver:

start_SNES_petsc_solver: var_nd%name=u
  0 SNES Function norm 3.412918650183e+01
 Attempt!
0 KSP Residual norm 5.861642176595e-01
1 KSP Residual norm 1.411858179645e-01
2 KSP Residual norm 1.388288156571e-01
3 KSP Residual norm 4.900215159087e-02
4 KSP Residual norm 2.559630070894e-02
5 KSP Residual norm 1.007110997387e-02
6 KSP Residual norm 6.371175598940e-03
  1 SNES Function norm 0.e+00
 iteration=   1
SNESConvergedReason=   2


istep=   1
**

start_SNES_petsc_solver: var_nd%name=u
  0 SNES Function norm 6.831896456736e+01
 Attempt!
0 KSP Residual norm 5.850729730568e+00
1 KSP Residual norm 5.176135972454e+00
2 KSP Residual norm 5.106059774079e-01
3 KSP Residual norm 2.058573608172e-01
4 KSP Residual norm 8.430267458444e-02
5 KSP Residual norm 2.421049820170e-02
6 KSP Residual norm 1.387479046692e-02
7 KSP Residual norm 6.556624109622e-03
  1 SNES Function norm 0.e+00
 iteration=   1
SNESConvergedReason=   2

istep=   2
**

start_SNES_petsc_solver: var_nd%name=u
  0 SNES Function norm 1.024330722398e+02
 Attempt!
0 KSP Residual norm 1.324263482159e+01
1 KSP Residual norm 8.772774639367e+00
2 KSP Residual norm 8.717824082000e-01
3 KSP Residual norm 3.965086318719e-01
4 KSP Residual norm 1.380063519887e-01
5 KSP Residual norm 3.983781619335e-02
6 KSP Residual norm 1.690524902818e-02
7 KSP Residual norm 1.371656480592e-02
8 KSP Residual norm 4.598826535286e-03
  1 SNES Function norm 0.e+00
 iteration=   1
SNESConvergedReason=   2

istep=   3
**

start_SNES_petsc_solver: var_nd%name=u
  0 SNES Function norm 1.371337889713e+02
 Attempt!
0 KSP Residual norm 1.823603533770e+01
1 KSP Residual norm 1.823546028484e+01
2 KSP Residual norm 8.167385988622e-01
3 KSP Residual norm 1.930526020067e-01
4 KSP Residual norm 1.768873013055e-01
5 KSP Residual norm 2.692456250466e-02
6 KSP Residual norm 1.130530545617e-02
7 KSP Residual norm 6.934825615412e-03
  1 SNES Function norm 0.e+00
 iteration=   1
SNESConvergedReason=   2

istep=   4
**

start_SNES_petsc_solver: var_nd%name=u
  0 SNES Function norm 1.717647834464e+02
 Attempt!
0 KSP Residual norm 1.015846744437e+02
1 KSP Residual norm 7.622793741160e+00
2 KSP Residual norm 1.381803723895e+00
3 KSP Residual norm 1.305423467184e-01
4 KSP Residual norm 3.606398975875e-02
5 KSP Residual norm 3.463999556864e-02
6 KSP Residual norm 3.268229989659e-02
7 KSP Residual norm 8.847393497789e-03
  1 SNES Function norm 0.e+00
 iteration=   1
SNESConvergedReason=   2

Do you have any idea what might be causing this behavior? I appreciate any 
insights you might have.

Best,
Khaled

Re: [petsc-users] Effect of -pc_gamg_threshold vs PETSc version

[Matthew Knepley]

On Fri, Apr 14, 2023 at 8:54 AM Jeremy Theler  wrote:

> Hi Mark. So glad you answered.
>
> > 0) what is your test problem? eg, 3D Lapacian with Q1 finite
> > elements.
>
> I said in my first email it was linear elasticty (and I gave a link
> where you can see the geometry, BCs, etc.) but I did not specifty
> further details.
> It is linear elasticity with displacement-based FEM formulation using
> unstructured curved 10-noded tetrahedra.
>

I believe our jargon for this would be "P_2 Lagrange element".


> The matrix is marked as SPD with MatSetOption() and the solver is
> indeed CG and not the default GMRES.
>
> > First, you can get GAMG diagnostics by running with '-info :pc' and
> > grep on GAMG.
>
> Great advice. Now I have a lot more of information but I'm not sure how
> to analyze it. Find attached for each combination of threshold and
> PETSc version the output of -info :pc -ksp_monitor -ksp_view
>
> In general it looks like 3.18 and 3.19 have less KSP iterations than
> 3.17 but the overall time is larger.
>

I will also look and see if I can figure out the change. This kind of
behavior usually means that
we somehow made the coarse problem larger. This can make it more accurate
(fewer iterations)
but more costly. This also makes sense that it is sensitive to the
threshold parameter, but that is
not the only thing that controls the sparsity. There is also squaring the
graph.

Mark, do you know if we change the default for squaring?

  Thanks,

Matt


> > Anyway, sorry for the changes.
> > I hate changing GAMG for this reason and I hate AMG for this reason!
>
> No need to apologize, I just want to better understand how to better
> exploit your code!
>
> Thanks
> --
> jeremy
>
> >
> > Thanks,
> > Mark
> >
> >
> >
> > On Thu, Apr 13, 2023 at 8:17 AM Jeremy Theler 
> > wrote:
> > > When using GAMG+cg for linear elasticity and providing the near
> > > nullspace computed by MatNullSpaceCreateRigidBody(), I used to find
> > > "experimentally" that a small value of -pc_gamg_threshold in the
> > > order
> > > of 0.0001 would slightly decrease the solve time.
> > >
> > > Starting with 3.18, I started seeing that any positive value for
> > > the
> > > treshold would increase the solve time. I did a quick parametric
> > > (serial) run solving an elastic problem with a matrix size of
> > > approx
> > > 570k x 570k for different values of GAMG threshold and different
> > > PETSc
> > > versions (compiled with the same compiler, options and flags).
> > >
> > > I noted that
> > >
> > >  1. starting from 3.18, a threshold of 0.0001 that used to improve
> > > the
> > > speed now worsens it.
> > >  2. PETSc 3.17 looks like a "sweet spot" of speed
> > >
> > > I would like to hear any comments you might have.
> > >
> > > The wall time shown includes the time needed to read the mesh and
> > > assemble the stiffness matrix. It is a refined version of the
> > > NAFEMS
> > > LE10 benchmark described here:
> > > https://seamplex.com/feenox/examples/mechanical.html#nafems-le10-
> > > thick-plate-pressure-benchmark
> > >
> > > If you want, I could dump the matrix, rhs and near nullspace
> > > vectors
> > > and share them.
> > >
> > > --
> > > jeremy theler
> > >
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] Effect of -pc_gamg_threshold vs PETSc version

[Jeremy Theler]

Hi Mark. So glad you answered.

> 0) what is your test problem? eg, 3D Lapacian with Q1 finite
> elements.

I said in my first email it was linear elasticty (and I gave a link
where you can see the geometry, BCs, etc.) but I did not specifty
further details.
It is linear elasticity with displacement-based FEM formulation using
unstructured curved 10-noded tetrahedra.

The matrix is marked as SPD with MatSetOption() and the solver is
indeed CG and not the default GMRES.

> First, you can get GAMG diagnostics by running with '-info :pc' and
> grep on GAMG.

Great advice. Now I have a lot more of information but I'm not sure how
to analyze it. Find attached for each combination of threshold and
PETSc version the output of -info :pc -ksp_monitor -ksp_view

In general it looks like 3.18 and 3.19 have less KSP iterations than
3.17 but the overall time is larger.

> Anyway, sorry for the changes. 
> I hate changing GAMG for this reason and I hate AMG for this reason!

No need to apologize, I just want to better understand how to better
exploit your code!

Thanks
--
jeremy

> 
> Thanks,
> Mark
> 
> 
> 
> On Thu, Apr 13, 2023 at 8:17 AM Jeremy Theler 
> wrote:
> > When using GAMG+cg for linear elasticity and providing the near
> > nullspace computed by MatNullSpaceCreateRigidBody(), I used to find
> > "experimentally" that a small value of -pc_gamg_threshold in the
> > order
> > of 0.0001 would slightly decrease the solve time.
> > 
> > Starting with 3.18, I started seeing that any positive value for
> > the
> > treshold would increase the solve time. I did a quick parametric
> > (serial) run solving an elastic problem with a matrix size of
> > approx
> > 570k x 570k for different values of GAMG threshold and different
> > PETSc
> > versions (compiled with the same compiler, options and flags).
> > 
> > I noted that
> > 
> >  1. starting from 3.18, a threshold of 0.0001 that used to improve
> > the
> > speed now worsens it. 
> >  2. PETSc 3.17 looks like a "sweet spot" of speed
> > 
> > I would like to hear any comments you might have.
> > 
> > The wall time shown includes the time needed to read the mesh and
> > assemble the stiffness matrix. It is a refined version of the
> > NAFEMS
> > LE10 benchmark described here:
> > https://seamplex.com/feenox/examples/mechanical.html#nafems-le10-
> > thick-plate-pressure-benchmark
> > 
> > If you want, I could dump the matrix, rhs and near nullspace
> > vectors
> > and share them.
> > 
> > --
> > jeremy theler
> > 



log.tar.gz
Description: application/compressed-tar

Re: [petsc-users] Drawing a line plot with two lines

[Matthew Knepley]

On Fri, Apr 14, 2023 at 2:58 AM Smit Thijs  wrote:

> Hi All,
>
>
> I am trying to plot a loglog plot with two lines, one for my error and one
> with a slope of 1. Plotting only the error or only the slope of 1 works
> fine, but I like both lines in the same plot (that doesn’t work till now).
> Does somebody know how to solve this? Find a code snipet below:
>
> PetscDraw draw;
>
> PetscDrawLG   lg;
>
> PetscDrawAxis axis;
>
> PetscReal xc, yc;
>
> PetscDrawCreate(PETSC_COMM_SELF, NULL, "Log(Error) vs Log(dx)",
> PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, PETSC_DRAW_HALF_SIZE,
> );
>
> PetscDrawSetFromOptions(draw);
>
> PetscDrawLGCreate(draw, 2, );
>
> PetscDrawLGSetUseMarkers(lg, PETSC_TRUE);
>
> PetscDrawLGGetAxis(lg, );
>
> PetscDrawAxisSetLabels(axis, NULL, "Log(dx)", "Log(Error)");
>
> for loop {
>
> xc = PetscLog10Real(dx);
>
> yc = PetscLog10Real(error);
>
> PetscDrawLGAddPoint(lg, , ); // to plot the error
>

Here is want an array of points, one for each curve:

  https://petsc.org/main/manualpages/Draw/PetscDrawLGAddPoint/

  Thanks,

 Matt


> PetscDrawLGAddPoint(lg, , ); // to plot line with slope 1
>
> PetscDrawLGDraw(lg);
> }
>
> PetscDrawSetPause(draw, -2);
>
> PetscDrawLGDestroy();
>
> PetscDrawDestroy();
>
>
>
> Best regards,
>
>
>
> Thijs Smit
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

[petsc-users] Drawing a line plot with two lines

[Smit Thijs]

Hi All,

I am trying to plot a loglog plot with two lines, one for my error and one with 
a slope of 1. Plotting only the error or only the slope of 1 works fine, but I 
like both lines in the same plot (that doesn't work till now). Does somebody 
know how to solve this? Find a code snipet below:

PetscDraw draw;
PetscDrawLG   lg;
PetscDrawAxis axis;
PetscReal xc, yc;
PetscDrawCreate(PETSC_COMM_SELF, NULL, "Log(Error) vs Log(dx)", PETSC_DECIDE, 
PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, PETSC_DRAW_HALF_SIZE, );
PetscDrawSetFromOptions(draw);
PetscDrawLGCreate(draw, 2, );
PetscDrawLGSetUseMarkers(lg, PETSC_TRUE);
PetscDrawLGGetAxis(lg, );
PetscDrawAxisSetLabels(axis, NULL, "Log(dx)", "Log(Error)");

for loop {
xc = PetscLog10Real(dx);
yc = PetscLog10Real(error);
PetscDrawLGAddPoint(lg, , ); // to plot the error
PetscDrawLGAddPoint(lg, , ); // to plot line with slope 1
PetscDrawLGDraw(lg);
}

PetscDrawSetPause(draw, -2);
PetscDrawLGDestroy();
PetscDrawDestroy();

Best regards,

Thijs Smit