While trying to use -fp_trap with a debugger on a local system we used the debug build of our and PETSc code and all solvers are working fine (left and right PC and fgmres also) and no NaNs either. Interrogating the optimized version of the code now…
On Mon, May 2, 2022 at 11:11 AM Matthew Knepley <[email protected]> wrote: > On Mon, May 2, 2022 at 1:56 PM Amneet Bhalla <[email protected]> > wrote: > >> Thanks Matt and Barry. Since we are using MATSHELL, we printed max norm >> of || b || and || A*x ||, the two things that have all the physics of the >> problem. Both are coming out to be finite (no NANs). Perhaps there is a NaN >> in PCSHELL. But this is counter-intuitive because the true residual norm || >> b - A*x || has a NaN where there is no PC application. We checked the >> velocity and pressure field in VisIt and they seem to be reasonable and >> matching the analytical solution well. Need to try it in a debugger or >> -fp_trap next. >> > > I would recommend -fp_trap. It will just fail at the first NaN so it is > really easy to find. You can also send the code and I can run it. > > >> The behavior is similar to what is discussed in this thread: >> https://www.mail-archive.com/[email protected]/msg34602.html >> > > That turned out to be a bug in their code. > > Thanks, > > Matt > > >> On Mon, May 2, 2022 at 7:56 AM Barry Smith <[email protected]> wrote: >> >>> >>> >>> On May 2, 2022, at 8:12 AM, Matthew Knepley <[email protected]> wrote: >>> >>> On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy < >>> [email protected]> wrote: >>> >>>> Thank you. I have a couple of questions. I am solving the low Mach >>>> Navier-Stokes system using a projection preconditioner (pc_shell type) with >>>> GMRES being the outer solver and Richardson being the Krylov >>>> preconditioner. The solver diverges when ksp_pc_type is "right”: >>>> >>>> Linear stokes_ solve did not converge due to DIVERGED_NANORINF >>>> iterations 0 >>>> >>> >>> NaN can always be tracked back. I recommend tracing it back to the first >>> NaN produced. My guess is that your equation of state if producing a NaN. >>> >>> >>> You can run in the debugger with -ksp_error_if_not_converged or >>> -fp_trap to see when the Nan first appears. If the problem does not appear >>> on one rank or you need to use mpiexec to start the program you can use the >>> option -start_in_debugger to have the program started up in the debugger >>> https://petsc.org/main/docs/manualpages/Sys/PetscAttachDebugger.html >>> >>> >>> >>> >>> >>> >>> Also, we have an example of low Mach flow in TS ex76. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> and it converges when ksp_pc_type is "left": >>>> >>>> Residual norms for stokes_ solve. >>>> 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm >>>> -nan ||r(i)||/||b|| -nan >>>> 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm >>>> -nan ||r(i)||/||b|| -nan >>>> 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm >>>> -nan ||r(i)||/||b|| -nan >>>> Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 >>>> >>>> I am curious to know why this is happening. The solver also diverges >>>> with "FGMRES" as the outer solver (which supports only right >>>> preconditioning). >>>> >>>> 2. Is it also possible to not get "-nan" when || b || = 0? >>>> >>>> >>>> Regards, >>>> Rama >>>> >>>> On Sun, May 1, 2022 at 12:12 AM Dave May <[email protected]> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Sun 1. May 2022 at 07:03, Amneet Bhalla <[email protected]> >>>>> wrote: >>>>> >>>>>> How about using a fixed number of Richardson iterations as a Krylov >>>>>> preconditioner to a GMRES solver? >>>>>> >>>>> >>>>> That is fine. >>>>> >>>>> Would that lead to a linear operation? >>>>>> >>>>> >>>>> Yes. >>>>> >>>>> >>>>> >>>>>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown <[email protected]> wrote: >>>>>> >>>>>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, >>>>>>> etc.) is a nonlinear operation. >>>>>>> >>>>>>> A fixed number of iterations of a method with a fixed polynomial, >>>>>>> such as Chebyshev, is a linear operation so you don't need a flexible >>>>>>> outer >>>>>>> method. >>>>>>> >>>>>>> Ramakrishnan Thirumalaisamy <[email protected]> writes: >>>>>>> >>>>>>> > Hi, >>>>>>> > >>>>>>> > I have a Krylov solver with a preconditioner that is also a Krylov >>>>>>> solver. >>>>>>> > I know I can use "fgmres" for the outer solver but can I use gmres >>>>>>> for the >>>>>>> > outer solver with a fixed number of iterations in the Krylov >>>>>>> > preconditioners? >>>>>>> > >>>>>>> > >>>>>>> > Thanks, >>>>>>> > Rama >>>>>>> >>>>>> -- >>>>>> --Amneet >>>>>> >>>>>> >>>>>> >>>>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> <http://www.cse.buffalo.edu/~knepley/> >>> >>> >>> >> >> -- >> --Amneet >> >> >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> > -- --Amneet
