Barry, thank you for the tips. Besides the trust region method, I have also tested line search methods. -snes_linesearch_type cp worked robustly. Other line search types didn't converge, except for basic. I'll spend some more time to check if my Jacobian is wrong or if -snes_mf_operator has some problem for my problem.
On Tue, Apr 29, 2014 at 5:02 PM, Barry Smith <[email protected]> wrote: > > On Apr 29, 2014, at 9:19 AM, Norihiro Watanabe <[email protected]> > wrote: > > > Hi Barry, > > > > Is it possible that -snes_mf_operator makes convergence of linear solves > slower if unknowns are poorly scaled for multiphysics problems? I gave up > to check Jacobian for the large problem because it takes too long time. > Instead I tested it with several different small size problems and noticed > scaling of my unknowns makes a difference in FD Jacobian. My unknowns are > two kinds: pressure (1e7) and temperature (1e3). > > This is not really a huge difference in scaling. In fact it seems > pretty minor to me. > > > I scaled pressure by 1e-5 and got the following different results from > -snes_type test > > Having a good scaling is always a good idea. > > > > without scaling > > Norm of matrix ratio 3.81371e-05 difference 6.42349e+06 (user-defined > state) > > > > with scaling > > Norm of matrix ratio 8.69182e-09 difference 1463.98 (user-defined state) > > > > which may suggest that a differentiate parameter h is not properly set > if unknowns are poorly scaled. I also tested the scaling for the large > problem with keeping fluid properties constant and got the following > different convergence behaviours: > > > > without -snes_mf_operator > > 0 SNES Function norm 4.425457683773e+04 > > 0 KSP Residual norm 44254.6 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 2 > > 1 KSP Residual norm 0.000168321 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 2 > > 2 KSP Residual norm 8.18977e-06 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 2 > > 3 KSP Residual norm 4.75882e-07 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > 4 KSP Residual norm 4.06286e-08 > > Linear solve converged due to CONVERGED_RTOL iterations 4 > > 1 SNES Function norm 2.229156139237e+05 > > I don’t like this. Your SNES function norm has increased. I would avoid > using a line search of basic and want to see some real decrease in the > residual norm. > > Have you thought about doing some grid sequencing or some other > continuation method to solve the nonlinear system? > > You could also try running with quad precision (./configure > —with-precision=__float128 and gnu compilers) > > In my experience most failures of Newton do not come from “difficult > physics” but rather from some error in the model (making it crazy), or the > function or Jacobian evaluation. > > Barry > > > > > with -snes_mf_operator > > 0 SNES Function norm 4.425457683883e+04 > > 0 KSP Residual norm 44254.6 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 2 > > 1 KSP Residual norm 5255.66 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > 2 KSP Residual norm 1646.58 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > 3 KSP Residual norm 1518.05 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > ... > > 42 KSP Residual norm 0.656962 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > 43 KSP Residual norm 0.462202 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > > with -snes_mf_operator and scaling of pressure by 1e-5 > > 0 SNES Function norm 4.425457683773e+04 > > 0 KSP Residual norm 44254.6 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 2 > > 1 KSP Residual norm 1883.94 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > 2 KSP Residual norm 893.88 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > ... > > 42 KSP Residual norm 6.66081e-08 > > Linear solve converged due to CONVERGED_ITS iterations 1 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > 43 KSP Residual norm 2.17062e-08 > > Linear solve converged due to CONVERGED_RTOL iterations 43 > > 1 SNES Function norm 2.200867439822e+05 > > > > > > > > > > > > > > > > > > On Mon, Apr 28, 2014 at 5:59 PM, Barry Smith <[email protected]> wrote: > > > > It will take a very long time > > > > On Apr 28, 2014, at 9:14 AM, Norihiro Watanabe <[email protected]> > wrote: > > > > > I cannot surely say my Jacobian for this particular problem is > correct, as I have not checked it. For a smaller problem, I've already > checked its correctness using -snes_type test or -snes_compare_explicit > (but linear solve and nonlinear solve with FD one need a few more > iterations than with my Jacobian). To make it sure, now I started > -snes_type test for the problem and will update you once it finished. By > the way, I'm waiting the calculation for more than three hours now. Is it > usual for a large problem (>1e6 dof) or is there something wrong? > > > > > > > > > > > > > > > > > > On Mon, Apr 28, 2014 at 6:34 AM, Barry Smith <[email protected]> > wrote: > > > > > > I have run your code. I changed to use -snes_type newtonls and also > -snes_mf_operator there is something wrong with your Jacobian: > > > > > > Without -snes_mf_operator > > > 0 SNES Function norm 1.821611413735e+03 > > > 0 KSP Residual norm 1821.61 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 1 KSP Residual norm 0.000167024 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 2 KSP Residual norm 7.66595e-06 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 3 KSP Residual norm 4.4581e-07 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 4 KSP Residual norm 3.77537e-08 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 5 KSP Residual norm 2.20453e-09 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 6 KSP Residual norm 1.711e-10 > > > Linear solve converged due to CONVERGED_RTOL iterations 6 > > > > > > with -snes_mf_operator > > > > > > 0 SNES Function norm 1.821611413735e+03 > > > 0 KSP Residual norm 1821.61 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 1 KSP Residual norm 1796.39 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 2 KSP Residual norm 1786.2 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 3 KSP Residual norm 1741.11 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 4 KSP Residual norm 1733.92 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 5 KSP Residual norm 1726.57 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 6 KSP Residual norm 1725.35 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 7 KSP Residual norm 1723.89 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 8 KSP Residual norm 1715.41 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 9 KSP Residual norm 1713.72 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > > 10 KSP Residual norm 1702.84 > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > > > > … > > > > > > This means your Jacobian is wrong. Your first order of business is > to fix your Jacobian. I noticed in previous emails your discussion with Jed > about switching to MatGetLocalSubMatrix() and using -snes_type test YOU > NEED TO DO THIS. You will get no where with an incorrect Jacobian. You need > to fix your Jacobian before you do anything else! No amount of other > options or methods will help you with a wrong Jacobian! Once you have a > correct Jacobian if you still have convergence problems let us know and we > can make further suggestions. > > > > > > Barry > > > > > > On Apr 25, 2014, at 7:31 AM, Norihiro Watanabe <[email protected]> > wrote: > > > > > > > Hi, > > > > > > > > In my simulation, nonlinear solve with the trust regtion method got > stagnent after linear solve (see output below). Is it possible that the > method goes to inifite loop? Is there any parameter to avoid this situation? > > > > > > > > 0 SNES Function norm 1.828728087153e+03 > > > > 0 KSP Residual norm 91.2735 > > > > Linear solve converged due to CONVERGED_ITS iterations 1 > > > > Linear solve converged due to CONVERGED_RTOL iterations 3 > > > > 1 KSP Residual norm 3.42223 > > > > Linear solve converged due to CONVERGED_STEP_LENGTH iterations 1 > > > > > > > > > > > > Thank you in advance, > > > > Nori > > > > > > > > > > > > > > > -- > > > Norihiro Watanabe > > > > > > > > > > -- > > Norihiro Watanabe > > -- Norihiro Watanabe
