Re: [petsc-dev] Recover from TS failure

2017-11-02 Thread Mark Adams
> > > >This is absolute nonsense. No way should you or do you need to do this. > TS can/is suppose to handle failures in the solver cleanly. We need a > reproducible result where it does not handle it correctly and then we fix > the problem > > No problem, Jed gave me the magic incantations :)

Re: [petsc-dev] Recover from TS failure

2017-11-02 Thread Smith, Barry F.
> On Nov 1, 2017, at 1:13 PM, Mark Adams wrote: > > Yea, I don't understand the linear solve error: > > -ts_monitor -ts_type beuler -pc_type lu -pc_factor_mat_solver_package mumps > -ksp_type preonly -snes_monitor -snes_rtol 1.e-10 -snes_stol 1.e-10 > -snes_converged_reason -snes_atol 1.e-18

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: > On Wed, Nov 1, 2017 at 8:30 PM, Jed Brown wrote: > >> Mark Adams writes: >> >> >> >> >> >> >> >> >> The last relevant output you've shown me is SNES failing a bunch of >> >> times as the adaptive controller attempts to shrink the step size and >> >> retry. If you fixed tha

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
On Wed, Nov 1, 2017 at 8:30 PM, Jed Brown wrote: > Mark Adams writes: > > >> > >> > >> > >> The last relevant output you've shown me is SNES failing a bunch of > >> times as the adaptive controller attempts to shrink the step size and > >> retry. If you fixed that problem, you need to tell me.

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: >> >> >> >> The last relevant output you've shown me is SNES failing a bunch of >> times as the adaptive controller attempts to shrink the step size and >> retry. If you fixed that problem, you need to tell me. If not, THAT is >> the problem you need to fix. >> > > That is th

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
> > > > The last relevant output you've shown me is SNES failing a bunch of > times as the adaptive controller attempts to shrink the step size and > retry. If you fixed that problem, you need to tell me. If not, THAT is > the problem you need to fix. > That is the problem I am trying to fix. No

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: > On Wed, Nov 1, 2017 at 7:21 PM, Jed Brown wrote: > >> Mark Adams writes: >> >> >> >> >> >> >> >> >> You're in PetscTraceBackErrorHandler which is way too late. Back up. >> >> What caused the error? >> >> >> >> >> > SNES problem. It can be linear solver error, max its, or

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
On Wed, Nov 1, 2017 at 7:21 PM, Jed Brown wrote: > Mark Adams writes: > > >> > >> > >> > >> You're in PetscTraceBackErrorHandler which is way too late. Back up. > >> What caused the error? > >> > >> > > SNES problem. It can be linear solver error, max its, or line search > > failure. See attac

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: >> >> >> >> You're in PetscTraceBackErrorHandler which is way too late. Back up. >> What caused the error? >> >> > SNES problem. It can be linear solver error, max its, or line search > failure. See attached. > > I see that I want to unset TS->errorifstepfailed. I see how to

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
> > > > You're in PetscTraceBackErrorHandler which is way too late. Back up. > What caused the error? > > SNES problem. It can be linear solver error, max its, or line search failure. See attached. I see that I want to unset TS->errorifstepfailed. I see how to do that with SNES, Will look again

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: > I am trying to recover from a failed time step with this code. But PETSc > does a hard abort with this code: > > } else { > /* do not print error messages since process 0 will print them, sleep > before aborting so will not accidently kill process 0*/ > PetscSleep(

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
I am trying to recover from a failed time step with this code. But PETSc does a hard abort with this code: } else { /* do not print error messages since process 0 will print them, sleep before aborting so will not accidently kill process 0*/ PetscSleep(10.0); abort(); } Can I g

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
On Wed, Nov 1, 2017 at 2:34 PM, Jed Brown wrote: > Mark Adams writes: > > > Oh, Maybe the Jacobian has Nans or Infs even though the last time step > > survived. Maybe it was going crazy. I'll check > > If that is the case you would use TSSetFunctionDomainError(). > Ah, very cool. Good to know.

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: > Oh, Maybe the Jacobian has Nans or Infs even though the last time step > survived. Maybe it was going crazy. I'll check If that is the case you would use TSSetFunctionDomainError(). > On Wed, Nov 1, 2017 at 2:13 PM, Mark Adams wrote: > >> Yea, I don't understand the linear

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
Oh, Maybe the Jacobian has Nans or Infs even though the last time step survived. Maybe it was going crazy. I'll check On Wed, Nov 1, 2017 at 2:13 PM, Mark Adams wrote: > Yea, I don't understand the linear solve error: > > -ts_monitor -ts_type beuler -pc_type lu -pc_factor_mat_solver_package > mu

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
Yea, I don't understand the linear solve error: -ts_monitor -ts_type beuler -pc_type lu -pc_factor_mat_solver_package mumps -ksp_type preonly -snes_monitor -snes_rtol 1.e-10 -snes_stol 1.e-10 -snes_converged_reason -snes_atol 1.e-18 -snes_converged_reason Maybe mumps? On Wed, Nov 1, 2017 at 2:02

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: > On Wed, Nov 1, 2017 at 1:46 PM, Jed Brown wrote: > >> Mark Adams writes: >> >> > I have added some code in a TS post step method to look at the number of >> > nonlinear iterations and cut the time step if it took too many SNES >> > iterations. That helped but now I want to

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
On Wed, Nov 1, 2017 at 1:46 PM, Jed Brown wrote: > Mark Adams writes: > > > I have added some code in a TS post step method to look at the number of > > nonlinear iterations and cut the time step if it took too many SNES > > iterations. That helped but now I want to go one step further and > re

Re: [petsc-dev] Recover from TS failure

2017-11-01 Thread Jed Brown
Mark Adams writes: > I have added some code in a TS post step method to look at the number of > nonlinear iterations and cut the time step if it took too many SNES > iterations. That helped but now I want to go one step further and recover > from a failed time step (see appended error message).

[petsc-dev] Recover from TS failure

2017-11-01 Thread Mark Adams
I have added some code in a TS post step method to look at the number of nonlinear iterations and cut the time step if it took too many SNES iterations. That helped but now I want to go one step further and recover from a failed time step (see appended error message). How can/should I recover fro