[petsc-users] DMPlex Halo Communication or Graph Partitioner Issue

[Mike Michell]

Dear PETSc team,

I am using PETSc for Fortran with DMPlex. I have been using this version of
PETSc:
>>git rev-parse origin
>>995ec06f924a86c4d28df68d1fdd6572768b0de1
>>git rev-parse FETCH_HEAD
>>9a04a86bf40bf893fb82f466a1bc8943d9bc2a6b

There has been no issue, before the one with VTK viewer, which Jed fixed
today (
https://gitlab.com/petsc/petsc/-/merge_requests/6081/diffs?commit_id=27ba695b8b62ee2bef0e5776c33883276a2a1735
).

Since that MR has been merged into the main repo, I pulled the latest
version of PETSc (basically I cloned it from scratch). But if I use the
same configure option with before, and run my code, then there is an issue
with halo exchange. The code runs without error message, but it gives wrong
solution field. I guess the issue I have is related to graph partitioner or
halo exchange part. This is because if I run the code with 1-proc, the
solution is correct. I only updated the version of PETSc and there was no
change in my own code. Could I get any comments on the issue? I was
wondering if there have been many changes in halo exchange or graph
partitioning & distributing part related to DMPlex.

Thanks,
Mike

Re: [petsc-users] TS failed due to diverged_step_rejected

[Tu, Jiannan]

I need to find out what causes negative temperature first. Following is the 
message with adaptivity turned off. The G(u) gives right-hand equation for 
electron temperature at bottom boundary. The F(u, u’) function is F(u, u’) = X 
= G(u) and the jacobian element is d F(u, u’) / dX =1. The solution from TSStep 
is checked for positivity of densities and temperatures.

>From the message below, it is seen that G(u) > 0 (I added output of right-hand 
>equation for electron temperature). The solution for electron temperature X 
>should be X * jacobian element = G(u) > 0 since jacobian element = 1. I don’t 
>understand why it becomes negative. Is my understanding of TS formula 
>incorrect?

Thank you,
Jiannan

--
G(u) = 1.86534e-07
0 SNES Function norm 2.274473072183e+03
1 SNES Function norm 8.641749325070e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
G(u) = 1.86534e-07
0 SNES Function norm 8.716501970511e-02
1 SNES Function norm 2.213263548813e-04
2 SNES Function norm 2.779985176426e-08
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
G(u) = 1.86534e-07
0 SNES Function norm 3.177195995186e-01
1 SNES Function norm 3.607702491344e-04
2 SNES Function norm 4.345809629121e-08
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
G(u) = 1.86534e-07
  TSAdapt none arkimex 0:3 step   0 accepted t=42960  + 2.189e-02 
dt=2.189e-02
electron temperature = -3.6757e-15 at (i, j, k) = (0, 1, 0)



From: Barry Smith
Sent: Friday, February 17, 2023 3:45 PM
To: Tu, Jiannan; Hong 
Zhang; Emil 
Constantinescu
Cc: petsc-users
Subject: Re: [petsc-users] TS failed due to diverged_step_rejected

CAUTION: This email was sent from outside the UMass Lowell network.




On Feb 17, 2023, at 3:32 PM, Tu, Jiannan  wrote:

The ts_type arkimex is used. There is right hand-side function RHSFunction set 
by TSSetRHSFunction() and also stiff function set by TSSetIFunction().

With adaptivity shut off, TS can finish its first time step after the 3rd 
“Nonlinear solve converged due to …”. The solution gives negative electron and 
neutral temperatures at the bottom boundary. I need to fix the negative 
temperatures and see how the code works.

BTW, what is this ts_adapt? Is it by default on?

   It is default for some of the TSTypes (in particular, the better ones). It 
adapts the timestep to ensure some local error estimate is below a certain 
tolerance. As Matt notes normally as it tries smaller and smaller time steps 
the local error estimate would get smaller and smaller; this is not happening 
here, hence the error.

   Have you tried with the argument -ts_arkimex_fully_implicit ?

   I am not an expert but my guess is something is "odd" about your functions, 
either the RHSFunction or the Function or both. Do you have a hierarchy of 
models for your problem? Could you try runs with fewer terms in your functions, 
that may be producing the difficulties? If you can determine what triggers the 
problem with the local error estimators, that might help the experts in ODE 
solution (not me) determine what could be going wrong.

  Barry





Thank you,
Jiannan

From: Matthew Knepley
Sent: Friday, February 17, 2023 3:15 PM
To: Tu, Jiannan
Cc: Barry Smith; 
petsc-users
Subject: Re: [petsc-users] TS failed due to diverged_step_rejected

CAUTION: This email was sent from outside the UMass Lowell network.

I am not sure what TS you are using, but the estimate of the local truncation 
error is 91.4, and does not seem
to change when you make the step smaller, so something is off. You can shut off 
the adaptivity using

 -ts_adapt_type none

  Thanks,

  Matt

On Fri, Feb 17, 2023 at 3:01 PM Tu, Jiannan 
mailto:jiannan...@uml.edu>> wrote:
These are what I got with the options you suggested.

Thank you,
Jiannan

---
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.673091274668e-03
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 8.715428433630e-02
1 SNES Function norm 4.995727626692e-04
2 SNES Function norm 5.498018152230e-08
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 3.237461568254e-01
1 SNES Function norm 7.988531005091e-04
2 SNES Function norm 1.280948196292e-07
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 2.189e-02 
dt=4.374e-03  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.881903203545e-04
  Nonlinear solve 

Re: [petsc-users] TS failed due to diverged_step_rejected

[Tu, Jiannan]

It’s possible. I’m checking the rhs and stiff equations.

Thank you,
Jiannan

From: Matthew Knepley
Sent: Friday, February 17, 2023 3:39 PM
To: Tu, Jiannan
Cc: Barry Smith; 
petsc-users
Subject: Re: [petsc-users] TS failed due to diverged_step_rejected

CAUTION: This email was sent from outside the UMass Lowell network.

On Fri, Feb 17, 2023 at 3:32 PM Tu, Jiannan 
mailto:jiannan...@uml.edu>> wrote:
The ts_type arkimex is used. There is right hand-side function RHSFunction set 
by TSSetRHSFunction() and also stiff function set by TSSetIFunction().

With adaptivity shut off, TS can finish its first time step after the 3rd 
“Nonlinear solve converged due to …”. The solution gives negative electron and 
neutral temperatures at the bottom boundary. I need to fix the negative 
temperatures and see how the code works.

BTW, what is this ts_adapt? Is it by default on?

It is a controller for adaptive timestepping. It is on by default, but I 
suspect that you step into an unphysical regime, which makes
its estimates unreliable.

  Thanks,

 Matt

Thank you,
Jiannan

From: Matthew Knepley
Sent: Friday, February 17, 2023 3:15 PM
To: Tu, Jiannan
Cc: Barry Smith; 
petsc-users
Subject: Re: [petsc-users] TS failed due to diverged_step_rejected

CAUTION: This email was sent from outside the UMass Lowell network.

I am not sure what TS you are using, but the estimate of the local truncation 
error is 91.4, and does not seem
to change when you make the step smaller, so something is off. You can shut off 
the adaptivity using

 -ts_adapt_type none

  Thanks,

  Matt

On Fri, Feb 17, 2023 at 3:01 PM Tu, Jiannan 
mailto:jiannan...@uml.edu>> wrote:
These are what I got with the options you suggested.

Thank you,
Jiannan

---
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.673091274668e-03
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 8.715428433630e-02
1 SNES Function norm 4.995727626692e-04
2 SNES Function norm 5.498018152230e-08
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 3.237461568254e-01
1 SNES Function norm 7.988531005091e-04
2 SNES Function norm 1.280948196292e-07
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 2.189e-02 
dt=4.374e-03  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.881903203545e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
   0 SNES Function norm 7.562592690785e-02
1 SNES Function norm 1.143078818923e-04
2 SNES Function norm 9.834547907735e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 2.683968949758e-01
1 SNES Function norm 1.838028436639e-04
2 SNES Function norm 9.470813523140e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-03 
dt=4.374e-04  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.821562431175e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 1.005443458812e-01
1 SNES Function norm 3.66946661e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.515368382715e-01
1 SNES Function norm 3.389298316830e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-04 
dt=4.374e-05  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.541003359206e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.713800906043e-01
1 SNES Function norm 1.179958172167e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.020265094117e-01
1 SNES Function norm 1.513971290464e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-05 
dt=4.374e-06  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 6.090269704320e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.136603895703e-01
1 SNES Function norm 1.877474016012e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.127812462507e-01
1 SNES Function norm 

Re: [petsc-users] TS failed due to diverged_step_rejected

[Barry Smith]

> On Feb 17, 2023, at 3:32 PM, Tu, Jiannan  wrote:
> 
> The ts_type arkimex is used. There is right hand-side function RHSFunction 
> set by TSSetRHSFunction() and also stiff function set by TSSetIFunction().
>  
> With adaptivity shut off, TS can finish its first time step after the 3rd 
> “Nonlinear solve converged due to …”. The solution gives negative electron 
> and neutral temperatures at the bottom boundary. I need to fix the negative 
> temperatures and see how the code works.
>  
> BTW, what is this ts_adapt? Is it by default on?

   It is default for some of the TSTypes (in particular, the better ones). It 
adapts the timestep to ensure some local error estimate is below a certain 
tolerance. As Matt notes normally as it tries smaller and smaller time steps 
the local error estimate would get smaller and smaller; this is not happening 
here, hence the error.

   Have you tried with the argument -ts_arkimex_fully_implicit ?

   I am not an expert but my guess is something is "odd" about your functions, 
either the RHSFunction or the Function or both. Do you have a hierarchy of 
models for your problem? Could you try runs with fewer terms in your functions, 
that may be producing the difficulties? If you can determine what triggers the 
problem with the local error estimators, that might help the experts in ODE 
solution (not me) determine what could be going wrong.

  Barry



>  
> Thank you,
> Jiannan
>  
> From: Matthew Knepley 
> Sent: Friday, February 17, 2023 3:15 PM
> To: Tu, Jiannan 
> Cc: Barry Smith ; petsc-users 
> 
> Subject: Re: [petsc-users] TS failed due to diverged_step_rejected
>  
> CAUTION: This email was sent from outside the UMass Lowell network.
>  
> I am not sure what TS you are using, but the estimate of the local truncation 
> error is 91.4, and does not seem
> to change when you make the step smaller, so something is off. You can shut 
> off the adaptivity using
>  
>  -ts_adapt_type none
>  
>   Thanks,
>  
>   Matt
>  
> On Fri, Feb 17, 2023 at 3:01 PM Tu, Jiannan  > wrote:
> These are what I got with the options you suggested.
>  
> Thank you,
> Jiannan
>  
> ---
> 0 SNES Function norm 2.274473072186e+03
> 1 SNES Function norm 1.673091274668e-03
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
> 0 SNES Function norm 8.715428433630e-02
> 1 SNES Function norm 4.995727626692e-04
> 2 SNES Function norm 5.498018152230e-08
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
> 0 SNES Function norm 3.237461568254e-01
> 1 SNES Function norm 7.988531005091e-04
> 2 SNES Function norm 1.280948196292e-07
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 2.189e-02 
> dt=4.374e-03  wlte= 91.4  wltea=   -1 wlter=   -1
> 0 SNES Function norm 2.274473072186e+03
> 1 SNES Function norm 4.881903203545e-04
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
>0 SNES Function norm 7.562592690785e-02
> 1 SNES Function norm 1.143078818923e-04
> 2 SNES Function norm 9.834547907735e-09
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
> 0 SNES Function norm 2.683968949758e-01
> 1 SNES Function norm 1.838028436639e-04
> 2 SNES Function norm 9.470813523140e-09
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-03 
> dt=4.374e-04  wlte= 91.4  wltea=   -1 wlter=   -1
> 0 SNES Function norm 2.274473072186e+03
> 1 SNES Function norm 1.821562431175e-04
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
> 0 SNES Function norm 1.005443458812e-01
> 1 SNES Function norm 3.66946661e-05
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 1.515368382715e-01
> 1 SNES Function norm 3.389298316830e-05
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-04 
> dt=4.374e-05  wlte= 91.4  wltea=   -1 wlter=   -1
> 0 SNES Function norm 2.274473072186e+03
> 1 SNES Function norm 4.541003359206e-05
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 1.713800906043e-01
> 1 SNES Function norm 1.179958172167e-05
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.020265094117e-01
> 1 SNES Function norm 1.513971290464e-05
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-05 
> dt=4.374e-06  wlte= 91.4  

Re: [petsc-users] TS failed due to diverged_step_rejected

[Matthew Knepley]

On Fri, Feb 17, 2023 at 3:32 PM Tu, Jiannan  wrote:

> The ts_type arkimex is used. There is right hand-side function RHSFunction
> set by TSSetRHSFunction() and also stiff function set by TSSetIFunction().
>
>
>
> With adaptivity shut off, TS can finish its first time step after the 3rd
> “Nonlinear solve converged due to …”. The solution gives negative electron
> and neutral temperatures at the bottom boundary. I need to fix the negative
> temperatures and see how the code works.
>
>
>
> BTW, what is this ts_adapt? Is it by default on?
>

It is a controller for adaptive timestepping. It is on by default, but I
suspect that you step into an unphysical regime, which makes
its estimates unreliable.

  Thanks,

 Matt


> Thank you,
>
> Jiannan
>
>
>
> *From: *Matthew Knepley 
> *Sent: *Friday, February 17, 2023 3:15 PM
> *To: *Tu, Jiannan 
> *Cc: *Barry Smith ; petsc-users
> 
> *Subject: *Re: [petsc-users] TS failed due to diverged_step_rejected
>
>
>
> *CAUTION:* This email was sent from outside the UMass Lowell network.
>
>
>
> I am not sure what TS you are using, but the estimate of the local
> truncation error is 91.4, and does not seem
>
> to change when you make the step smaller, so something is off. You can
> shut off the adaptivity using
>
>
>
>  -ts_adapt_type none
>
>
>
>   Thanks,
>
>
>
>   Matt
>
>
>
> On Fri, Feb 17, 2023 at 3:01 PM Tu, Jiannan  wrote:
>
> These are what I got with the options you suggested.
>
>
>
> Thank you,
>
> Jiannan
>
>
>
>
> ---
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 1.673091274668e-03
>
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
>
> 0 SNES Function norm 8.715428433630e-02
>
> 1 SNES Function norm 4.995727626692e-04
>
> 2 SNES Function norm 5.498018152230e-08
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
> 0 SNES Function norm 3.237461568254e-01
>
> 1 SNES Function norm 7.988531005091e-04
>
> 2 SNES Function norm 1.280948196292e-07
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 2.189e-02
> dt=4.374e-03  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 4.881903203545e-04
>
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
>
>0 SNES Function norm 7.562592690785e-02
>
> 1 SNES Function norm 1.143078818923e-04
>
> 2 SNES Function norm 9.834547907735e-09
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
> 0 SNES Function norm 2.683968949758e-01
>
> 1 SNES Function norm 1.838028436639e-04
>
> 2 SNES Function norm 9.470813523140e-09
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-03
> dt=4.374e-04  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 1.821562431175e-04
>
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
>
> 0 SNES Function norm 1.005443458812e-01
>
> 1 SNES Function norm 3.66946661e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 1.515368382715e-01
>
> 1 SNES Function norm 3.389298316830e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-04
> dt=4.374e-05  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 4.541003359206e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 1.713800906043e-01
>
> 1 SNES Function norm 1.179958172167e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 2.020265094117e-01
>
> 1 SNES Function norm 1.513971290464e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-05
> dt=4.374e-06  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 6.090269704320e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 2.136603895703e-01
>
> 1 SNES Function norm 1.877474016012e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 3.127812462507e-01
>
> 1 SNES Function norm 2.713146825704e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-06
> dt=4.374e-07  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 

Re: [petsc-users] TS failed due to diverged_step_rejected

[Tu, Jiannan]

The ts_type arkimex is used. There is right hand-side function RHSFunction set 
by TSSetRHSFunction() and also stiff function set by TSSetIFunction().

With adaptivity shut off, TS can finish its first time step after the 3rd 
“Nonlinear solve converged due to …”. The solution gives negative electron and 
neutral temperatures at the bottom boundary. I need to fix the negative 
temperatures and see how the code works.

BTW, what is this ts_adapt? Is it by default on?

Thank you,
Jiannan

From: Matthew Knepley
Sent: Friday, February 17, 2023 3:15 PM
To: Tu, Jiannan
Cc: Barry Smith; 
petsc-users
Subject: Re: [petsc-users] TS failed due to diverged_step_rejected

CAUTION: This email was sent from outside the UMass Lowell network.

I am not sure what TS you are using, but the estimate of the local truncation 
error is 91.4, and does not seem
to change when you make the step smaller, so something is off. You can shut off 
the adaptivity using

 -ts_adapt_type none

  Thanks,

  Matt

On Fri, Feb 17, 2023 at 3:01 PM Tu, Jiannan 
mailto:jiannan...@uml.edu>> wrote:
These are what I got with the options you suggested.

Thank you,
Jiannan

---
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.673091274668e-03
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 8.715428433630e-02
1 SNES Function norm 4.995727626692e-04
2 SNES Function norm 5.498018152230e-08
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 3.237461568254e-01
1 SNES Function norm 7.988531005091e-04
2 SNES Function norm 1.280948196292e-07
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 2.189e-02 
dt=4.374e-03  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.881903203545e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
   0 SNES Function norm 7.562592690785e-02
1 SNES Function norm 1.143078818923e-04
2 SNES Function norm 9.834547907735e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 2.683968949758e-01
1 SNES Function norm 1.838028436639e-04
2 SNES Function norm 9.470813523140e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-03 
dt=4.374e-04  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.821562431175e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 1.005443458812e-01
1 SNES Function norm 3.66946661e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.515368382715e-01
1 SNES Function norm 3.389298316830e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-04 
dt=4.374e-05  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.541003359206e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.713800906043e-01
1 SNES Function norm 1.179958172167e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.020265094117e-01
1 SNES Function norm 1.513971290464e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-05 
dt=4.374e-06  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 6.090269704320e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.136603895703e-01
1 SNES Function norm 1.877474016012e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.127812462507e-01
1 SNES Function norm 2.713146825704e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-06 
dt=4.374e-07  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 2.793512213059e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.205196267430e-01
1 SNES Function norm 2.572653773308e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.260057361977e-01
1 SNES Function norm 2.705816087598e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 

Re: [petsc-users] TS failed due to diverged_step_rejected

[Matthew Knepley]

I am not sure what TS you are using, but the estimate of the local
truncation error is 91.4, and does not seem
to change when you make the step smaller, so something is off. You can shut
off the adaptivity using

 -ts_adapt_type none

  Thanks,

  Matt

On Fri, Feb 17, 2023 at 3:01 PM Tu, Jiannan  wrote:

> These are what I got with the options you suggested.
>
>
>
> Thank you,
>
> Jiannan
>
>
>
>
> ---
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 1.673091274668e-03
>
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
>
> 0 SNES Function norm 8.715428433630e-02
>
> 1 SNES Function norm 4.995727626692e-04
>
> 2 SNES Function norm 5.498018152230e-08
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
> 0 SNES Function norm 3.237461568254e-01
>
> 1 SNES Function norm 7.988531005091e-04
>
> 2 SNES Function norm 1.280948196292e-07
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 2.189e-02
> dt=4.374e-03  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 4.881903203545e-04
>
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
>
>0 SNES Function norm 7.562592690785e-02
>
> 1 SNES Function norm 1.143078818923e-04
>
> 2 SNES Function norm 9.834547907735e-09
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
> 0 SNES Function norm 2.683968949758e-01
>
> 1 SNES Function norm 1.838028436639e-04
>
> 2 SNES Function norm 9.470813523140e-09
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-03
> dt=4.374e-04  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 1.821562431175e-04
>
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
>
> 0 SNES Function norm 1.005443458812e-01
>
> 1 SNES Function norm 3.66946661e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 1.515368382715e-01
>
> 1 SNES Function norm 3.389298316830e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-04
> dt=4.374e-05  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 4.541003359206e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 1.713800906043e-01
>
> 1 SNES Function norm 1.179958172167e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 2.020265094117e-01
>
> 1 SNES Function norm 1.513971290464e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-05
> dt=4.374e-06  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 6.090269704320e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 2.136603895703e-01
>
> 1 SNES Function norm 1.877474016012e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 3.127812462507e-01
>
> 1 SNES Function norm 2.713146825704e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-06
> dt=4.374e-07  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 2.793512213059e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 2.205196267430e-01
>
> 1 SNES Function norm 2.572653773308e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 3.260057361977e-01
>
> 1 SNES Function norm 2.705816087598e-06
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
>   TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-07
> dt=4.374e-08  wlte= 91.4  wltea=   -1 wlter=   -1
>
> 0 SNES Function norm 2.274473072186e+03
>
> 1 SNES Function norm 2.764855860446e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 2.212505522844e-01
>
> 1 SNES Function norm 2.958996472386e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
>
> 0 SNES Function norm 3.273222034162e-01
>
> 1 SNES Function norm 2.994512887620e-05
>
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS 

Re: [petsc-users] TS failed due to diverged_step_rejected

[Tu, Jiannan]

These are what I got with the options you suggested.

Thank you,
Jiannan

---
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.673091274668e-03
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 8.715428433630e-02
1 SNES Function norm 4.995727626692e-04
2 SNES Function norm 5.498018152230e-08
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 3.237461568254e-01
1 SNES Function norm 7.988531005091e-04
2 SNES Function norm 1.280948196292e-07
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 2.189e-02 
dt=4.374e-03  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.881903203545e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
   0 SNES Function norm 7.562592690785e-02
1 SNES Function norm 1.143078818923e-04
2 SNES Function norm 9.834547907735e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 2.683968949758e-01
1 SNES Function norm 1.838028436639e-04
2 SNES Function norm 9.470813523140e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-03 
dt=4.374e-04  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.821562431175e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 1.005443458812e-01
1 SNES Function norm 3.66946661e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.515368382715e-01
1 SNES Function norm 3.389298316830e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-04 
dt=4.374e-05  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.541003359206e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.713800906043e-01
1 SNES Function norm 1.179958172167e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.020265094117e-01
1 SNES Function norm 1.513971290464e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-05 
dt=4.374e-06  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 6.090269704320e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.136603895703e-01
1 SNES Function norm 1.877474016012e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.127812462507e-01
1 SNES Function norm 2.713146825704e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-06 
dt=4.374e-07  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 2.793512213059e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.205196267430e-01
1 SNES Function norm 2.572653773308e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.260057361977e-01
1 SNES Function norm 2.705816087598e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-07 
dt=4.374e-08  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 2.764855860446e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.212505522844e-01
1 SNES Function norm 2.958996472386e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.273222034162e-01
1 SNES Function norm 2.994512887620e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
  TSAdapt basic arkimex 0:3 step   0 rejected t=42960  + 4.374e-08 
dt=4.374e-09  wlte= 91.4  wltea=   -1 wlter=   -1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 3.317240589134e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 2.213246532918e-01
1 SNES Function norm 2.799468604767e-04
  Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1
0 SNES Function norm 3.274570888397e-01
1 SNES Function norm 3.066048050994e-04
  Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1
  TSAdapt 

Re: [petsc-users] PetscViewer with 64bit

[Mike Michell]

It works as I wanted. Thank you for the change.

Mike


> Okay, this works now. I'm pretty sure I tested this long ago with
> connectivity using Int64 and found that didn't work. That may have been
> ancient history, but I'm hesitant to revamp to match PetscInt. If doing
> that, it would require changing the signature of DMPlexGetVTKConnectivity
> to use PetscInt instead of PetscVTKInt. I'm already underwater and don't
> have the stamina to test it, but this MR will get you going for problems in
> which individual parts don't have more than 2B dofs.
>
>
> https://gitlab.com/petsc/petsc/-/merge_requests/6081/diffs?commit_id=27ba695b8b62ee2bef0e5776c33883276a2a1735
>
> Mike Michell  writes:
>
> > Jed,
> >
> > It does not work for me even with the reproducer case with the small 2D
> > square mesh. Can you run the case that I sent and open the created
> > "sol.vtu" file with paraview?
> >
> > Thanks,
> >
> >
> >> Thanks, Dave.
> >>
> >> Mike, can you test that this branch works with your large problems? I
> >> tested that .vtu works in parallel for small problems, where works =
> loads
> >> correctly in Paraview and VisIt.
> >>
> >> https://gitlab.com/petsc/petsc/-/merge_requests/6081
> >>
> >> Dave May  writes:
> >>
> >> > On Tue 14. Feb 2023 at 21:27, Jed Brown  wrote:
> >> >
> >> >> Dave May  writes:
> >> >>
> >> >> > On Tue 14. Feb 2023 at 17:17, Jed Brown  wrote:
> >> >> >
> >> >> >> Can you share a reproducer? I think I recall the format requiring
> >> >> certain
> >> >> >> things to be Int32.
> >> >> >
> >> >> >
> >> >> > By default, the byte offset used with the appended data format is
> >> >> UInt32. I
> >> >> > believe that’s where the sizeof(int) is coming from. This default
> is
> >> >> > annoying as it limits the total size of your appended data to be <
> 3
> >> GB.
> >> >> > That said, in the opening of the paraview file you can add this
> >> attribute
> >> >> >
> >> >> > header_type="UInt64"
> >> >>
> >> >> You mean in the header of the .vtu?
> >> >
> >> >
> >> > Yeah, within the open VTKFile tag.
> >> > Like this
> >> > < VTKFile type=“xxx”,  byte_order="LittleEndian" header_type="UInt64"
> >
> >> >
> >> > Do you happen to have an example or pointers to docs describing this
> >> >> feature?
> >> >
> >> >
> >> > Example yes - will send it to you tomorrow. Docs… not really. Only
> stuff
> >> > like this
> >> >
> >> >
> >>
> https://kitware.github.io/paraview-docs/latest/python/paraview.simple.XMLPStructuredGridWriter.html
> >> >
> >> >
> >> >
> >>
> https://kitware.github.io/paraview-docs/v5.8.0/python/paraview.simple.XMLMultiBlockDataWriter.html
> >> >
> >> > All the writers seem to support it.
> >> >
> >> >
> >> > Can we always do this?
> >> >
> >> >
> >> > Yep!
> >> >
> >> >
> >> > It isn't mentioned in these:
> >> >>
> >> >> https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf   (PDF
> was
> >> >> created in 2003)
> >> >>
> >> >>
> >>
> https://kitware.github.io/vtk-examples/site/VTKFileFormats/#xml-file-formats
> >> >>
> >> >
> >> > Yes I know. I’ve tied myself in knots for years because the of the
> >> > assumption that the offset had to be an int.
> >> >
> >> > Credit for the discovery goes to Carsten Uphoff. He showed me this.
> >> >
> >> > Cheers,
> >> > Dave
> >> >
> >> >
> >> >
> >> >> > then the size of the offset is now UInt64 and now large files can
> be
> >> >> > finally written.
> >> >> >
> >> >> >
> >> >> > Cheers,
> >> >> > Dave
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >>
> >> >> >> Mike Michell  writes:
> >> >> >>
> >> >> >> > Thanks for the note.
> >> >> >> > I understood that PETSc calculates the offsets for me through
> >> >> "boffset"
> >> >> >> > variable in plexvtu.c file. Please correct me if it is wrong.
> >> >> >> >
> >> >> >> > If plexvtu.c has a bug, it could be around "write file header"
> >> part in
> >> >> >> > which the boffset is also computed. Is this correct? I am not
> using
> >> >> >> complex
> >> >> >> > number.
> >> >> >> > There are several mixed parts among "Int32, UInt8, PetscInt_FMT,
> >> >> >> > PetscInt64_FMT" in writing the header.
> >> >> >> >
> >> >> >> > Which combination of those flags is correct for 64bit indices?
> I am
> >> >> gonna
> >> >> >> > modify plexvtu.c file with "#if
> defined(PETSC_USE_64BIT_INDICES)"
> >> >> >> > statement, but I do not know what is the correct form of the
> header
> >> >> flag
> >> >> >> > for 64bit indices.
> >> >> >> >
> >> >> >> > It is also confusing to me:
> >> >> >> > boffset += gpiece[r].ncells * sizeof(PetscInt) + sizeof(int);
> >> >> >> > How is sizeof(PetscInt) different from sizeof(int)?
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Mike
> >> >> >> >
> >> >> >> >
> >> >> >> >> On Tue, Feb 14, 2023 at 11:45 AM Mike Michell <
> >> mi.mike1...@gmail.com
> >> >> >
> >> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >>> I was trying to modify the header flags from "Int32" to
> "Int64",
> >> but
> >> >> >> the
> >> >> >> >>> problem was not resolved. Could I get any additional comments?
> >> >> >> 

Re: [petsc-users] TS failed due to diverged_step_rejected

[Barry Smith]

  Can you please run with also the options -ts_monitor -ts_adapt_monitor ?

  The output is confusing because it prints that the Nonlinear solve has 
converged but then TSStep has failed due to DIVERGED_STEP_REJECTED which seems 
contradictory



> On Feb 17, 2023, at 12:09 PM, Tu, Jiannan  wrote:
> 
> My code uses TS to solve a set of multi-fluid MHD equations. The jacobian is 
> provided with function F(t, u, u'). Both linear and nonlinear solvers 
> converge but snes repeats itself until gets "TSStep has failed due to 
> diverged_step_rejected." 
> 
> Is it because I used TSStep rather than TSSolve? I have checked the condition 
> number. The condition number with pc_type asm is about 1 (without 
> precondition it is about 4x10^4). The maximum ratio of off-diagonal jacobian 
> element over diagonal element is about 21.
> 
> Could you help me to identify what is going wrong?
> 
> Thank you very much!
> 
> Jiannan
> 
> ---
> Run command with options
> 
> mpiexec -n $1 ./iditm3d -ts_type arkimex -snes_tyep ngmres -ksp_type gmres 
> -pc_type asm \
> -ts_rtol 1.0e-4 -ts_atol 1.0e-4 -snes_monitor -snes_rtol 1.0e-4 -snes_atol 
> 1.0e-4 \
> -snes_converged_reason
> 
> The output message is 
> 
> Start time advancing ...
> 0 SNES Function norm 2.274473072186e+03 
> 1 SNES Function norm 1.673091274668e-03 
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
> 0 SNES Function norm 8.715428433630e-02 
> 1 SNES Function norm 4.995727626692e-04 
> 2 SNES Function norm 5.498018152230e-08 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
> 0 SNES Function norm 3.237461568254e-01 
> 1 SNES Function norm 7.988531005091e-04 
> 2 SNES Function norm 1.280948196292e-07 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
> 0 SNES Function norm 2.274473072186e+03 
> 1 SNES Function norm 4.881903203545e-04 
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
> 0 SNES Function norm 7.562592690785e-02 
> 1 SNES Function norm 1.143078818923e-04 
> 2 SNES Function norm 9.834547907735e-09 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
> 0 SNES Function norm 2.683968949758e-01 
> 1 SNES Function norm 1.838028436639e-04 
> 2 SNES Function norm 9.470813523140e-09 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
> 0 SNES Function norm 2.274473072186e+03 
> 1 SNES Function norm 1.821562431175e-04 
>   Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
> 0 SNES Function norm 1.005443458812e-01 
> 1 SNES Function norm 3.66946661e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 1.515368382715e-01 
> 1 SNES Function norm 3.389298316830e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.274473072186e+03 
> 1 SNES Function norm 4.541003359206e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 1.713800906043e-01 
> 1 SNES Function norm 1.179958172167e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.020265094117e-01 
> 1 SNES Function norm 1.513971290464e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.274473072186e+03 
> 1 SNES Function norm 6.090269704320e-06 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.136603895703e-01 
> 1 SNES Function norm 1.877474016012e-06 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 3.127812462507e-01 
> 1 SNES Function norm 2.713146825704e-06 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.274473072186e+03 
> 1 SNES Function norm 2.793512213059e-06 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.205196267430e-01 
> 1 SNES Function norm 2.572653773308e-06 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 3.260057361977e-01 
> 1 SNES Function norm 2.705816087598e-06 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.274473072186e+03 
> 1 SNES Function norm 2.764855860446e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 2.212505522844e-01 
> 1 SNES Function norm 2.958996472386e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES Function norm 3.273222034162e-01 
> 1 SNES Function norm 2.994512887620e-05 
>   Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
> 0 SNES 

Re: [petsc-users] Question about rank of matrix

[Barry Smith]

> On Feb 17, 2023, at 2:43 AM, user_gong Kim  wrote:
> 
> Hello,
> 
> I have a question about rank of matrix.
> At the problem 
> Au = b, 
> 
> In my case, sometimes global matrix A is not full rank.
> In this case, the global matrix A is more likely to be singular, and if it 
> becomes singular, the problem cannot be solved even in the case of the direct 
> solver.
> I haven't solved the problem with an iterative solver yet, but I would like 
> to ask someone who has experienced this kind of problem.
> 
> 1. If it is not full rank, is there a numerical technique to solve it by 
> catching rows and columns with empty ranks in advance?

   Matrices with completely empty (zero) rows and their corresponding columns 
are a simple type of singularity (so long as the corresponding right hand side 
values are zero) that are not difficult to solve.

   Essentially iterative solvers never "see" the empty rows and columns and 
successfully solve the linear systems. 

   You can also use PCREDISTRIBUTE (even on one MPI rank) in the main git 
branch of PETSc to automatically extract the nontrivial rows and columns from 
the matrix and solve that smaller system with any solver including direct 
solvers.



> 
> 2.If anyone has solved it in a different way than the above numerical 
> analysis method, please tell me your experience.
> 
> Thanks,
> Hyung Kim
> 
> 

[petsc-users] TS failed due to diverged_step_rejected

[Tu, Jiannan]

My code uses TS to solve a set of multi-fluid MHD equations. The jacobian is 
provided with function F(t, u, u'). Both linear and nonlinear solvers converge 
but snes repeats itself until gets "TSStep has failed due to 
diverged_step_rejected."

Is it because I used TSStep rather than TSSolve? I have checked the condition 
number. The condition number with pc_type asm is about 1 (without precondition 
it is about 4x10^4). The maximum ratio of off-diagonal jacobian element over 
diagonal element is about 21.

Could you help me to identify what is going wrong?

Thank you very much!

Jiannan

---
Run command with options

mpiexec -n $1 ./iditm3d -ts_type arkimex -snes_tyep ngmres -ksp_type gmres 
-pc_type asm \
-ts_rtol 1.0e-4 -ts_atol 1.0e-4 -snes_monitor -snes_rtol 1.0e-4 -snes_atol 
1.0e-4 \
-snes_converged_reason

The output message is

Start time advancing ...
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.673091274668e-03
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 8.715428433630e-02
1 SNES Function norm 4.995727626692e-04
2 SNES Function norm 5.498018152230e-08
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 3.237461568254e-01
1 SNES Function norm 7.988531005091e-04
2 SNES Function norm 1.280948196292e-07
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.881903203545e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 7.562592690785e-02
1 SNES Function norm 1.143078818923e-04
2 SNES Function norm 9.834547907735e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 2.683968949758e-01
1 SNES Function norm 1.838028436639e-04
2 SNES Function norm 9.470813523140e-09
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 2
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 1.821562431175e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 1.005443458812e-01
1 SNES Function norm 3.66946661e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.515368382715e-01
1 SNES Function norm 3.389298316830e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 4.541003359206e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 1.713800906043e-01
1 SNES Function norm 1.179958172167e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.020265094117e-01
1 SNES Function norm 1.513971290464e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 6.090269704320e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.136603895703e-01
1 SNES Function norm 1.877474016012e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.127812462507e-01
1 SNES Function norm 2.713146825704e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 2.793512213059e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.205196267430e-01
1 SNES Function norm 2.572653773308e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.260057361977e-01
1 SNES Function norm 2.705816087598e-06
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 2.764855860446e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.212505522844e-01
1 SNES Function norm 2.958996472386e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 3.273222034162e-01
1 SNES Function norm 2.994512887620e-05
  Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
0 SNES Function norm 2.274473072186e+03
1 SNES Function norm 3.317240589134e-04
  Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1
0 SNES Function norm 2.213246532918e-01
1 SNES Function norm 2.799468604767e-04
  Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1
0 SNES Function norm 3.274570888397e-01
1 SNES Function norm 3.066048050994e-04
  Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1
0 SNES Function norm 2.274473072189e+03
1 SNES Function norm 

Re: [petsc-users] Help with fieldsplit performance

[Matthew Knepley]

On Fri, Feb 17, 2023 at 7:09 AM Edoardo alinovi 
wrote:

> Thanks Matt, That was my thinking.
>
> So If I am going to build my own Schur approximation matrix, I guess I can
> forget about the numbering issue, correct?
>

Yes, you just match the numbering you are using.

  Thanks,

Matt

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] Help with fieldsplit performance

[Edoardo alinovi]

Thanks Matt, That was my thinking.

So If I am going to build my own Schur approximation matrix, I guess I can
forget about the numbering issue, correct?

Re: [petsc-users] Help with fieldsplit performance

[Matthew Knepley]

On Tue, Feb 14, 2023 at 1:43 PM Edoardo alinovi 
wrote:

> Hi Matt,
>
> So I have done some research these days and I have found out that I might
> try to assemble the SIMPLE for Schur approximation (myS = A11 - A10
> inv(DIAGFORM(A00)) A01).
>

I show this on some slides. You can do it with options since the inverse
diagonal is just Preonly/Jacobi. It is not a great preconditioner, at least
for Stokes.


> Reading papers around, I come up with a doubt, which I believe to be a
> very silly one but worth asking...
>
> Is the way the unknowns are packed in the matrix relevant for schur
> preconditioning?
>

No, we pull out the matrices before running the algorithm, so you can use
any numbering you want as long as you
use it when you call SetIS().

  Thanks,

 Matt


> I was studying a bit ex70.c, there the block matrix is defined like:
>
> A = [A00 A10
>A10  A11]
> Where A00 is the momentum equation matrix, A11 is the pressure equation
> matrix, while A01 and A10 are the matrices for the coupling terms (i.e.
> pressure gradient and continuity). The unknowns are x = [u1..uN v1...vN
> w1...wN p1...pN]^T
>
> In my case, I assemble the matrix cell by cell (FV method), and the result
> will be this one:
>
> [image: image.png]
>
> Then I split the fields giving index 0-1 for u and 2 for p. I guess Petsc
> is already doing the correct handling picking up the *a^33s* to assemble
> A11, but worth being 100% sure :)
>
> Thank you!
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] Question about rank of matrix

[Matthew Knepley]

On Fri, Feb 17, 2023 at 2:43 AM user_gong Kim  wrote:

> Hello,
>
> I have a question about rank of matrix.
> At the problem
> Au = b,
>
> In my case, sometimes global matrix A is not full rank.
> In this case, the global matrix A is more likely to be singular, and if it
> becomes singular, the problem cannot be solved even in the case of the
> direct solver.
> I haven't solved the problem with an iterative solver yet, but I would
> like to ask someone who has experienced this kind of problem.
>
> 1. If it is not full rank, is there a numerical technique to solve it by
> catching rows and columns with empty ranks in advance?
>
> 2.If anyone has solved it in a different way than the above numerical
> analysis method, please tell me your experience.
>

As Pierre points out,  MUMPS can solve singular systems.

If you have an explicit characterization of the null space, then many
iterative methods can also solve it by projecting out
the null space. You call MatSetNullSpace() on the system matrix.

  Thanks,

 Matt


> Thanks,
> Hyung Kim
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] Question about rank of matrix

[Pierre Jolivet]

> On 17 Feb 2023, at 8:56 AM, Stefano Zampini  wrote:
> 
> 
> On Fri, Feb 17, 2023, 10:43 user_gong Kim  > wrote:
>> Hello,
>> 
>> I have a question about rank of matrix.
>> At the problem 
>> Au = b, 
>> 
>> In my case, sometimes global matrix A is not full rank.
>> In this case, the global matrix A is more likely to be singular, and if it 
>> becomes singular, the problem cannot be solved even in the case of the 
>> direct solver.
>> I haven't solved the problem with an iterative solver yet, but I would like 
>> to ask someone who has experienced this kind of problem.
>> 
>> 1. If it is not full rank, is there a numerical technique to solve it by 
>> catching rows and columns with empty ranks in advance?
>> 
>> 2.If anyone has solved it in a different way than the above numerical 
>> analysis method, please tell me your experience.
>> 
>> Thanks,
>> Hyung Kim
> 
> 
> My experience with this is usually associated to reading a book and find the 
> solution I'm looking for. 

On top of that, some exact factorization packages can solve singular systems, 
unlike what you are stating.
E.g., MUMPS, together with the option -mat_mumps_icntl_24, see 
https://mumps-solver.org/doc/userguide_5.5.1.pdf

Thanks,
Pierre