Re: [petsc-users] Regarding Adding to ex11.c

[Mark Adams]

You want to make an MR (check to delete branch after merge and squash
commits), you can mark as WIP, run the pipeline and after you get it to run
clean and merge any comments, umark WIP. If it does not get merged in a few
days you could ask Satish.
Mark

On Tue, Jun 30, 2020 at 5:12 PM MUKKUND SUNJII 
wrote:

> Greetings,
>
> Thank you for your message.
>
> Excellent! Does it suffice to ask for the merge request once I made the
> necessary additions to ex11.c?
>
> Regards,
>
> Mukkund
>
> On 30 Jun 2020, at 19:16, Matthew Knepley  wrote:
>
> On Tue, Jun 30, 2020 at 12:56 PM MUKKUND SUNJII 
> wrote:
>
>> Greetings,
>>
>> I am a master’s student in TU Delft, The Netherlands, and I am carrying
>> out my thesis using the adaptive mesh refinement solver provided in
>> ./ts/tutorials/ex11.c.
>>
>> In the process of my work, I have added a HLL Riemann Solver for the
>> shallow water equation. I had already tested and validated the Riemann
>> Solver using several cases (I am happy to provide a copy of my thesis
>> report). It would indeed be nice to see my work reflected in the current
>> version of ex11.c as an addition to its capability.
>>
>> Furthermore, I had also added a Riemann Solver that contains topography
>> terms and is well-balanced*. The caveat is that I see that there some
>> variations in the well-balancedness when choosing between p4est and Plex
>> DM’s. Perhaps, this could be looked at and fixed in the future.
>>
>> The purpose of this email is to propose the addition of the HLL Riemann
>> Solver for the shallow water equations to ex11.c and perhaps later explore
>> the possibility of adding the Riemann Solver with the bathymetry terms.
>>
>> I have already gone through the CI process instituted by PETSc
>> (previously integrated ./ts/tutorials/ex54.c into the repository with the
>> valuable assistance of Mr. Satish Balay). Hence, I would be more or less
>> familiar with the process.
>>
>> I kindly ask you provide any feedback on this proposition.
>>
>
> Yes, this sounds great to me.
>
>   Thanks,
>
>  Matt
>
>
>> Regards,
>>
>> Mukkund Sunjii
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> 
>
>
>

Re: [petsc-users] Regarding Adding to ex11.c

[MUKKUND SUNJII]

Greetings, 

Thank you for your message. 

Excellent! Does it suffice to ask for the merge request once I made the 
necessary additions to ex11.c?

Regards, 

Mukkund 

> On 30 Jun 2020, at 19:16, Matthew Knepley  wrote:
> 
> On Tue, Jun 30, 2020 at 12:56 PM MUKKUND SUNJII  > wrote:
> Greetings, 
> 
> I am a master’s student in TU Delft, The Netherlands, and I am carrying out 
> my thesis using the adaptive mesh refinement solver provided in 
> ./ts/tutorials/ex11.c. 
> 
> In the process of my work, I have added a HLL Riemann Solver for the shallow 
> water equation. I had already tested and validated the Riemann Solver using 
> several cases (I am happy to provide a copy of my thesis report). It would 
> indeed be nice to see my work reflected in the current version of ex11.c as 
> an addition to its capability. 
> 
> Furthermore, I had also added a Riemann Solver that contains topography terms 
> and is well-balanced*. The caveat is that I see that there some variations in 
> the well-balancedness when choosing between p4est and Plex DM’s. Perhaps, 
> this could be looked at and fixed in the future. 
> 
> The purpose of this email is to propose the addition of the HLL Riemann 
> Solver for the shallow water equations to ex11.c and perhaps later explore 
> the possibility of adding the Riemann Solver with the bathymetry terms. 
> 
> I have already gone through the CI process instituted by PETSc (previously 
> integrated ./ts/tutorials/ex54.c into the repository with the valuable 
> assistance of Mr. Satish Balay). Hence, I would be more or less familiar with 
> the process. 
> 
> I kindly ask you provide any feedback on this proposition. 
> 
> Yes, this sounds great to me.
> 
>   Thanks,
> 
>  Matt
>  
> Regards, 
> 
> Mukkund Sunjii 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments 
> is infinitely more interesting than any results to which their experiments 
> lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] Regarding Adding to ex11.c

[Matthew Knepley]

On Tue, Jun 30, 2020 at 12:56 PM MUKKUND SUNJII 
wrote:

> Greetings,
>
> I am a master’s student in TU Delft, The Netherlands, and I am carrying
> out my thesis using the adaptive mesh refinement solver provided in
> ./ts/tutorials/ex11.c.
>
> In the process of my work, I have added a HLL Riemann Solver for the
> shallow water equation. I had already tested and validated the Riemann
> Solver using several cases (I am happy to provide a copy of my thesis
> report). It would indeed be nice to see my work reflected in the current
> version of ex11.c as an addition to its capability.
>
> Furthermore, I had also added a Riemann Solver that contains topography
> terms and is well-balanced*. The caveat is that I see that there some
> variations in the well-balancedness when choosing between p4est and Plex
> DM’s. Perhaps, this could be looked at and fixed in the future.
>
> The purpose of this email is to propose the addition of the HLL Riemann
> Solver for the shallow water equations to ex11.c and perhaps later explore
> the possibility of adding the Riemann Solver with the bathymetry terms.
>
> I have already gone through the CI process instituted by PETSc (previously
> integrated ./ts/tutorials/ex54.c into the repository with the valuable
> assistance of Mr. Satish Balay). Hence, I would be more or less familiar
> with the process.
>
> I kindly ask you provide any feedback on this proposition.
>

Yes, this sounds great to me.

  Thanks,

 Matt


> Regards,
>
> Mukkund Sunjii



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

[petsc-users] Regarding Adding to ex11.c

[MUKKUND SUNJII]

Greetings, 

I am a master’s student in TU Delft, The Netherlands, and I am carrying out my 
thesis using the adaptive mesh refinement solver provided in 
./ts/tutorials/ex11.c. 

In the process of my work, I have added a HLL Riemann Solver for the shallow 
water equation. I had already tested and validated the Riemann Solver using 
several cases (I am happy to provide a copy of my thesis report). It would 
indeed be nice to see my work reflected in the current version of ex11.c as an 
addition to its capability. 

Furthermore, I had also added a Riemann Solver that contains topography terms 
and is well-balanced*. The caveat is that I see that there some variations in 
the well-balancedness when choosing between p4est and Plex DM’s. Perhaps, this 
could be looked at and fixed in the future. 

The purpose of this email is to propose the addition of the HLL Riemann Solver 
for the shallow water equations to ex11.c and perhaps later explore the 
possibility of adding the Riemann Solver with the bathymetry terms. 

I have already gone through the CI process instituted by PETSc (previously 
integrated ./ts/tutorials/ex54.c into the repository with the valuable 
assistance of Mr. Satish Balay). Hence, I would be more or less familiar with 
the process. 

I kindly ask you provide any feedback on this proposition. 

Regards, 

Mukkund Sunjii 

Re: [petsc-users] PETSc and Windows 10

[Pierre Jolivet]

Please use the 3.13.2 tarball, this was fixed by Satish in the previous commit 
I already linked 
(https://gitlab.com/petsc/petsc/-/commit/2cd8068296b34e127f055bb32f556e3599f17523
 
).
(If you want FreeFEM to do the dirty work for you, just switch to the develop 
branch, and redo “make petsc-slepc”)
But I think you’ve got everything you need now for a smooth compilation :)

Thanks,
Pierre

> On 30 Jun 2020, at 3:09 PM, Paolo Lampitella  
> wrote:
> 
> Dear Pierre,
>  
> thanks for the fast response. Unfortunately it still fails, but now in the 
> configure of ScaLAPACK
> (which means that it went ok for slepc, tetgen, metis, parmetis, ptscotch, 
> superlu and suitesparse).
>  
> The way I applied the modification is by manually editing the Makefile in the 
> 3rdparty/ff-petsc folder, adding -fallow-invalid-boz to both CFLAGS and 
> FFLAGS (this entry added by me). Then executed make petsc-slepc.
>  
> As my project is much less ambitious, I have a good feeling that I will be 
> able to use your Makefile successfully, but as I am kind of slow I tought 
> that it would have been useful for you to know. The configure.log is 
> attached. This time the error is:
>  
> Rank mismatch between actual argument at (1) and actual argument at (2) 
> (scalar and rank-1)
>  
> in subroutine pclarf.f of ScaLAPACK.
>  
> However, before attampting with my project, I have few questions about your 
> Makefile, in particular this piece:
>  
> --with-mpi-lib=/c/Windows/System32/msmpi.dll 
> --with-mpi-include=/home/paolo/FreeFem-sources/3rdparty/include/msmpi 
> --with-mpiexec="/C/Program\ Files/Microsoft\ MPI/Bin/mpiexec"
>  
> I see from MPI.py that I should not use ‘--with-mpi-lib/include’ if I want to 
> use my now working mpi wrappers. Is this correct?
>  
> Paolo 
>  
> Inviato da Posta  per Windows 
> 10
>  
> Da: Pierre Jolivet 
> Inviato: lunedì 29 giugno 2020 21:37
> A: Paolo Lampitella 
> Cc: Matthew Knepley ; petsc-users 
> 
> Oggetto: Re: [petsc-users] PETSc and Windows 10
>  
> I do not give up easily on Windows problems:
> 1) that’s around 50% of our (FreeFEM) user-base (and I want them to use PETSc 
> and SLEPc, ofc…)
> 2) most people I work with from corporations just have Windows 
> laptops/desktops and I always recommend MSYS because it’s very lightweight 
> and you can pass .exe around
> 3) I’ve bothered enough Satish, Jed, and Matt on GitLab to take (at least 
> partially) the blame now when it doesn’t work on MSYS
>  
> That being said, the magic keyword is the added flag 
> FFLAGS="-fallow-invalid-boz" (see, I told you ./configure issues were easier 
> to deal with than the others).
> Here you’ll see that everything goes through just fine (sorry, it took me a 
> long time to post this because everything is slow on my VM):
> 1) http://jolivet.perso.enseeiht.fr/win10/configure.log 
> 
> 2) http://jolivet.perso.enseeiht.fr/win10/make.log 
>  (both steps #1 and #2 in 
> MSYS terminal, gcc/gfortran 10, MS-MPI see screenshot)
> 3) http://jolivet.perso.enseeiht.fr/win10/ex2.txt 
>  (Command Prompt, 4 processes 
> + MUMPS, I can send you the .exe if you want to try on your machine)
> I just realize that I didn’t generate the Fortran bindings, but you can see I 
> compiled MUMPS and ScaLAPACK, so that shouldn’t be a problem.
> Or if there is a problem, we will need to fix this in PETSc.
>  
> I’ll push this added flag to the FreeFEM repo, thanks for reminding me of the 
> brokenness of gcc/gfortran 10 + MS-MPI.
> Here is to hoping this won’t affect PETSc ./configure with previous 
> gcc/gfortran version (unlikely, this option is apparently 13-year old 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=29471 
> )
>  
> Let me know of the next hiccup, if any.
> Thanks,
> Pierre
> 
> 
> On 29 Jun 2020, at 8:09 PM, Paolo Lampitella  > wrote:
>  
> Dear Pierre,
>  
> thanks again for your time
>  
> I guess there is no way for me to use the toolchain you are using (I don’t 
> remember having any choice on which version of MSYS or GCC I could install)
>  
> Paolo
>  
> Inviato da Posta  per Windows 
> 10
>  
> Da: Pierre Jolivet 
> Inviato: lunedì 29 giugno 2020 20:01
> A: Matthew Knepley 
> Cc: Paolo Lampitella ; petsc-users 
> 
> Oggetto: Re: [petsc-users] PETSc and Windows 10
>  
>  
> 
> 
> 
> On 29 Jun 2020, at 7:47 PM, Matthew Knepley  

Re: [petsc-users] DIVERGED_NANORINF when using HYPRE/BoomerAMG

[Matthew Knepley]

On Tue, Jun 30, 2020 at 7:59 AM Andrea Iob  wrote:

> Hi,
>
> I'm trying to solve a linear system using HYPRE/BoomerAMG as
> preconditioner. The system comes from a two-dimensional adjoint
> Navier-Stokes problem.
>

Do you expect the system you are solving to be elliptic? BoomerAMG is
designed for elliptic systems, and can easily fail if applied
to a more general system, say one with zeros on the diagonal.

  Thanks,

 Matt


> The mesh is structured (6400 cells) and there are 24 DoFs for each cell
> (the matrix has a total of 153600 rows). The condition number of the matrix
> should be in the order of 1e9. Using ILU preconditioner and FGMRES, the
> system is correctly solved (it takes 800 iterations to reach a residual of
> 1e-12). However, replacing ILU with HYPRE/BoomerAMG, the solver stops right
> after the first iteration (DIVERGED_NANORINF). I've tried different
> BoomerAMG options and different solvers (including Richardson to use
> BoomerAMG without a Krylov method), but I always get the same
> DIVERGED_NANORINF error.
>
> Using the same HYPRE/BoomerAMG + FGMERS setup on another problem
> (Laplacian on a highly stretched grid), the solver reaches convergence very
> quickly without any problems.
>
> At the bottom of this mail, you'll find the log of a run that stops with
> DIVERGED_NANORINF. Do you spot something in my setup that may explain why
> the solver is not converging?
>
> Thanks. Best regards,
> Andrea
>
> -
> Assembly started...
>  Reading matrix file...
>- Number of rows (N) = 153600
>- Number of non-zero entries (NNZ) = 18247680
>  Assembly completed.
>  Time elapsed 83.3231s
>  Set up started...
>Set up completed.
>Time elapsed 6.62441s
>  Solution started...
>   0 KSP unpreconditioned resid norm 7.736650641501e-01 true resid norm
> 7.736650641501e-01 ||r(i)||/||b|| 1.e+00
>   0 KSP Residual norm 7.736650641501e-01 % max 1.e+00 min
> 1.e+00 max/min 1.e+00
> [0]PETSC ERROR: - Error Message
> --
> [0]PETSC ERROR:
> [0]PETSC ERROR: KSPSolve has not converged due to Nan or Inf inner product
> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.10.3, unknown
> [0]PETSC ERROR: bitpit_system_solver on a  named xxx by xxx Tue Jun 30
> 09:42:09 2020
> [0]PETSC ERROR: Configure options PETSC_ARCH=arch-linux2-c-opt-gcc7-hypre
> --with-blaslapack-dir=/opt/lapack/3.8.0-gcc.7.4.0/lib64 --with-debugging=0
> COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 --with-valgrind=1
> --with-valgrind-dir=/opt/valgrind/3.14.0/
> --prefix=/opt/petsc/3.10.3_gcc-7.4.0 --with-mpi=1 --download-hypre
> [0]PETSC ERROR: #1 KSPGMRESClassicalGramSchmidtOrthogonalization() line 67
> in /root/InstallSources/petsc/src/ksp/ksp/impls/gmres/borthog2.c
> [0]PETSC ERROR: #2 KSPFGMRESCycle() line 175 in
> /root/InstallSources/petsc/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
> [0]PETSC ERROR: #3 KSPSolve_FGMRES() line 291 in
> /root/InstallSources/petsc/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
> [0]PETSC ERROR: #4 KSPSolve() line 780 in
> /root/InstallSources/petsc/src/ksp/ksp/interface/itfunc.c
> KSP Object: 1 MPI processes
>   type: fgmres
> restart=45, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
> happy breakdown tolerance 1e-30
>   maximum iterations=1, nonzero initial guess
>   tolerances:  relative=1e-12, absolute=1e-50, divergence=1.
>   right preconditioning
>   using UNPRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
>   type: hypre
> HYPRE BoomerAMG preconditioning
>   Cycle type V
>   Maximum number of levels 25
>   Maximum number of iterations PER hypre call 1
>   Convergence tolerance PER hypre call 0.
>   Threshold for strong coupling 0.7
>   Interpolation truncation factor 0.3
>   Interpolation: max elements per row 2
>   Number of levels of aggressive coarsening 4
>   Number of paths for aggressive coarsening 5
>   Maximum row sums 0.9
>   Sweeps down 1
>   Sweeps up   1
>   Sweeps on coarse1
>   Relax down  sequential-Gauss-Seidel
>   Relax upsequential-Gauss-Seidel
>   Relax on coarse Gaussian-elimination
>   Relax weight  (all)  1.
>   Outer relax weight (all) 1.
>   Using CF-relaxation
>   Smooth type  Euclid
>   Smooth num levels25
>   Euclid ILU(k) levels 0
>   Euclid ILU(k) drop tolerance 0.
>   Euclid ILU use Block-Jacobi? 1
>   Measure typelocal
>   Coarsen typeHMIS
>   Interpolation type  ext+i
>   linear system matrix = precond matrix:
>   Mat Object: Mat_0x1e88040_0 1 MPI processes
> type: seqaij
> rows=153600, cols=153600, bs=24
> total: nonzeros=18247680, 

[petsc-users] DIVERGED_NANORINF when using HYPRE/BoomerAMG

[Andrea Iob]

Hi,

I'm trying to solve a linear system using HYPRE/BoomerAMG as preconditioner. 
The system comes from a two-dimensional adjoint Navier-Stokes problem. The mesh 
is structured (6400 cells) and there are 24 DoFs for each cell (the matrix has 
a total of 153600 rows). The condition number of the matrix should be in the 
order of 1e9. Using ILU preconditioner and FGMRES, the system is correctly 
solved (it takes 800 iterations to reach a residual of 1e-12). However, 
replacing ILU with HYPRE/BoomerAMG, the solver stops right after the first 
iteration (DIVERGED_NANORINF). I've tried different BoomerAMG options and 
different solvers (including Richardson to use BoomerAMG without a Krylov 
method), but I always get the same DIVERGED_NANORINF error.

Using the same HYPRE/BoomerAMG + FGMERS setup on another problem (Laplacian on 
a highly stretched grid), the solver reaches convergence very quickly without 
any problems.

At the bottom of this mail, you'll find the log of a run that stops with 
DIVERGED_NANORINF. Do you spot something in my setup that may explain why the 
solver is not converging?

Thanks. Best regards,
Andrea

-
Assembly started...
 Reading matrix file...
   - Number of rows (N) = 153600
   - Number of non-zero entries (NNZ) = 18247680
 Assembly completed.
 Time elapsed 83.3231s
 Set up started...
   Set up completed.
   Time elapsed 6.62441s
 Solution started...
  0 KSP unpreconditioned resid norm 7.736650641501e-01 true resid norm 
7.736650641501e-01 ||r(i)||/||b|| 1.e+00
  0 KSP Residual norm 7.736650641501e-01 % max 1.e+00 min 
1.e+00 max/min 1.e+00
[0]PETSC ERROR: - Error Message 
--
[0]PETSC ERROR:
[0]PETSC ERROR: KSPSolve has not converged due to Nan or Inf inner product
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for 
trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.10.3, unknown
[0]PETSC ERROR: bitpit_system_solver on a  named xxx by xxx Tue Jun 30 09:42:09 
2020
[0]PETSC ERROR: Configure options PETSC_ARCH=arch-linux2-c-opt-gcc7-hypre 
--with-blaslapack-dir=/opt/lapack/3.8.0-gcc.7.4.0/lib64 --with-debugging=0 
COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 --with-valgrind=1 
--with-valgrind-dir=/opt/valgrind/3.14.0/ --prefix=/opt/petsc/3.10.3_gcc-7.4.0 
--with-mpi=1 --download-hypre
[0]PETSC ERROR: #1 KSPGMRESClassicalGramSchmidtOrthogonalization() line 67 in 
/root/InstallSources/petsc/src/ksp/ksp/impls/gmres/borthog2.c
[0]PETSC ERROR: #2 KSPFGMRESCycle() line 175 in 
/root/InstallSources/petsc/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
[0]PETSC ERROR: #3 KSPSolve_FGMRES() line 291 in 
/root/InstallSources/petsc/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
[0]PETSC ERROR: #4 KSPSolve() line 780 in 
/root/InstallSources/petsc/src/ksp/ksp/interface/itfunc.c
KSP Object: 1 MPI processes
  type: fgmres
restart=45, using Classical (unmodified) Gram-Schmidt Orthogonalization 
with no iterative refinement
happy breakdown tolerance 1e-30
  maximum iterations=1, nonzero initial guess
  tolerances:  relative=1e-12, absolute=1e-50, divergence=1.
  right preconditioning
  using UNPRECONDITIONED norm type for convergence test
PC Object: 1 MPI processes
  type: hypre
HYPRE BoomerAMG preconditioning
  Cycle type V
  Maximum number of levels 25
  Maximum number of iterations PER hypre call 1
  Convergence tolerance PER hypre call 0.
  Threshold for strong coupling 0.7
  Interpolation truncation factor 0.3
  Interpolation: max elements per row 2
  Number of levels of aggressive coarsening 4
  Number of paths for aggressive coarsening 5
  Maximum row sums 0.9
  Sweeps down 1
  Sweeps up   1
  Sweeps on coarse1
  Relax down  sequential-Gauss-Seidel
  Relax upsequential-Gauss-Seidel
  Relax on coarse Gaussian-elimination
  Relax weight  (all)  1.
  Outer relax weight (all) 1.
  Using CF-relaxation
  Smooth type  Euclid
  Smooth num levels25
  Euclid ILU(k) levels 0
  Euclid ILU(k) drop tolerance 0.
  Euclid ILU use Block-Jacobi? 1
  Measure typelocal
  Coarsen typeHMIS
  Interpolation type  ext+i
  linear system matrix = precond matrix:
  Mat Object: Mat_0x1e88040_0 1 MPI processes
type: seqaij
rows=153600, cols=153600, bs=24
total: nonzeros=18247680, allocated nonzeros=18247680
total number of mallocs used during MatSetValues calls =0
  using I-node routines: found 32000 nodes, limit used is 5
   Solution completed.
   Time elapsed 6.93336s
 Solution information...
Error (L2)   : 32.8359
Error (Linf) : 1.93278

*** WIDEN YOUR WINDOW TO 120 CHARACTERS.