Have you tried -pc_type hypre and then -pc_type gamg ? Where does your problem come from? What physical models and what type of discretization?
Barry > On Sep 1, 2017, at 4:31 PM, zakaryah . <[email protected]> wrote: > > You were right - I don't think the Jacobian has errors but there was a > mistake in the function evaluation. Now when I run with -snes_type newtonls > -snes_monitor -ksp_monitor -ksp_monitor_true_residual -ksp_converged_reason > -snes_converged_reason -pc_type lu -snes_linesearch_monitor the SNES > converges fairly quickly and there are no problems with the line search. I > also ran with -snes_type test -snes_test_display, at various initial guesses, > and in all cases the hand-coded Jacobian agreed well with the finite > difference Jacobian. > > Unfortunately, I need to solve the system on a grid which is too large for LU > decomp. When I try running the same grid with the same options except > without -pc_type lu, everything converges fine. However, when I increase the > grid size, even by a factor 2 in each dimension, the KSP no longer converges > with the default options (neither the preconditioned residual nor the true > residual decrease from about iteration 6000 onward). Should I move on to > looking into the linear solver? > > To check if the linear system is singular, I ran the problem on the small > grid (KSP converges here but not for 2^3-fold larger grid) with -pc_type svd > -pc_svd_monitor and got the following output: > > 0 SNES Function norm 1.259563818203e+03 > > SVD: condition number 9.125223106399e+02, 0 of 4590 singular values are > (nearly) zero > > SVD: smallest singular values: 6.205544702362e+00 8.361112027794e+00 > 9.327934960932e+00 1.250439191766e+01 1.373952564946e+01 > > SVD: largest singular values : 5.497346539524e+03 5.517949695999e+03 > 5.539227197966e+03 5.596380525169e+03 5.662697990578e+03 > > 0 KSP Residual norm 7.340750993927e+00 > > 0 KSP preconditioned resid norm 7.340750993927e+00 true resid norm > 1.259563818203e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 8.452960708813e-13 > > 1 KSP preconditioned resid norm 8.452960708813e-13 true resid norm > 2.092403440031e-10 ||r(i)||/||b|| 1.661212722842e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: Using full step: fnorm 1.259563818203e+03 gnorm > 2.909228222342e+02 > > 1 SNES Function norm 2.909228222342e+02 > > SVD: condition number 8.837760047818e+02, 0 of 4590 singular values are > (nearly) zero > > SVD: smallest singular values: 6.853997087392e+00 8.414157285275e+00 > 9.547538541816e+00 1.307676522991e+01 1.424251871761e+01 > > SVD: largest singular values : 5.674169364836e+03 5.711489755811e+03 > 5.754496402805e+03 5.905177592591e+03 6.057398162681e+03 > > 0 KSP Residual norm 6.537083346419e-01 > > 0 KSP preconditioned resid norm 6.537083346419e-01 true resid norm > 2.909228222342e+02 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 4.766960746477e-14 > > 1 KSP preconditioned resid norm 4.766960746477e-14 true resid norm > 2.206377081276e-11 ||r(i)||/||b|| 7.584063238254e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: Using full step: fnorm 2.909228222342e+02 gnorm > 8.519091421012e+00 > > 2 SNES Function norm 8.519091421012e+00 > > SVD: condition number 8.866866174487e+02, 0 of 4590 singular values are > (nearly) zero > > SVD: smallest singular values: 6.754153695550e+00 8.386828076859e+00 > 9.514891596031e+00 1.302462235577e+01 1.424263827525e+01 > > SVD: largest singular values : 5.671563211881e+03 5.685626194828e+03 > 5.739548236997e+03 5.833938566429e+03 5.988817694036e+03 > > 0 KSP Residual norm 3.040350335176e-02 > > 0 KSP preconditioned resid norm 3.040350335176e-02 true resid norm > 8.519091421012e+00 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.285565040836e-15 > > 1 KSP preconditioned resid norm 3.285565040836e-15 true resid norm > 9.005553787171e-13 ||r(i)||/||b|| 1.057102611314e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: Using full step: fnorm 8.519091421012e+00 gnorm > 3.086711801483e-02 > > 3 SNES Function norm 3.086711801483e-02 > > SVD: condition number 8.863731497353e+02, 0 of 4590 singular values are > (nearly) zero > > SVD: smallest singular values: 6.750850248606e+00 8.386167701356e+00 > 9.513896745592e+00 1.302218359138e+01 1.424165997875e+01 > > SVD: largest singular values : 5.669769588361e+03 5.683785644290e+03 > 5.736938066065e+03 5.828997334185e+03 5.983772398249e+03 > > 0 KSP Residual norm 3.152844718957e-05 > > 0 KSP preconditioned resid norm 3.152844718957e-05 true resid norm > 3.086711801483e-02 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.048057745094e-18 > > 1 KSP preconditioned resid norm 3.048057745094e-18 true resid norm > 9.890968076924e-16 ||r(i)||/||b|| 3.204370447598e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: Using full step: fnorm 3.086711801483e-02 gnorm > 2.527921600901e-07 > > 4 SNES Function norm 2.527921600901e-07 > > Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 4 > > > > I suppose it is possible that the linear system becomes more singular as the > grid size increases but I think it will take forever to run SVD on it. I > have not tried other PC or KSP methods. Any advice? > > > On Wed, Aug 30, 2017 at 12:37 PM, Matthew Knepley <[email protected]> wrote: > On Wed, Aug 30, 2017 at 12:21 AM, zakaryah . <[email protected]> wrote: > OK, this is very helpful (and not totally clear from the "Why is Newton's > method not converging" FAQ at this address: > https://scicomp.stackexchange.com/questions/30/why-is-newtons-method-not-converging. > Here's the output, using the intermediate grid and the options -snes_type > newtonls -snes_monitor -ksp_monitor -ksp_monitor_true_residual > -ksp_converged_reason -snes_converged_reason -pc_type lu > -snes_linesearch_monitor: > > 0 SNES Function norm 1.370293318432e+04 > > 0 KSP Residual norm 7.457516441709e+02 > > 0 KSP preconditioned resid norm 7.457516441709e+02 true resid norm > 1.370293318432e+04 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 1.244742959480e-10 > > 1 KSP preconditioned resid norm 1.244742959480e-10 true resid norm > 9.866304569403e-09 ||r(i)||/||b|| 7.200140609815e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 6.541326653047e+05 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 8.963755251044e+04 lambda=5.0000000000000003e-02 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.788913078169e+04 lambda=2.5000000000000001e-02 > > Line search: Cubically determined step, current gnorm > 1.340565818376e+04 lambda=1.2500000000000001e-02 > > Your Jacobian is wrong. You solved the Newton linear system, which should be > guaranteed to give a residual reduction, but > you get nothing. Thus we conclude that your Jacobian is wrong, giving a wrong > descent direction. I would recommend simplifying > the problem until you can check the Jacobian entries by hand. > > Thanks, > > Matt > 1 SNES Function norm 1.340565818376e+04 > > 0 KSP Residual norm 3.837938728414e+02 > > 0 KSP preconditioned resid norm 3.837938728414e+02 true resid norm > 1.340565818376e+04 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.935624425879e-11 > > 1 KSP preconditioned resid norm 3.935624425879e-11 true resid norm > 4.012917478400e-09 ||r(i)||/||b|| 2.993450544086e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 7.752009107979e+05 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 9.942192482364e+04 lambda=5.0000000000000003e-02 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.724069009956e+04 lambda=2.5000000000000001e-02 > > Line search: Cubically determined step, current gnorm > 1.272263847462e+04 lambda=1.2500000000000001e-02 > > 2 SNES Function norm 1.272263847462e+04 > > 0 KSP Residual norm 8.610741245938e+01 > > 0 KSP preconditioned resid norm 8.610741245938e+01 true resid norm > 1.272263847462e+04 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.953708398506e-12 > > 1 KSP preconditioned resid norm 3.953708398506e-12 true resid norm > 1.259105691297e-09 ||r(i)||/||b|| 9.896576828850e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 1.126216236986e+04 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 3 SNES Function norm 1.126216236986e+04 > > 0 KSP Residual norm 6.100581225849e+01 > > 0 KSP preconditioned resid norm 6.100581225849e+01 true resid norm > 1.126216236986e+04 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.714526144776e-12 > > 1 KSP preconditioned resid norm 3.714526144776e-12 true resid norm > 1.150524854441e-09 ||r(i)||/||b|| 1.021584325156e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 9.992994468502e+03 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 4 SNES Function norm 9.992994468502e+03 > > 0 KSP Residual norm 5.452156705070e+01 > > 0 KSP preconditioned resid norm 5.452156705070e+01 true resid norm > 9.992994468502e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 2.345517221383e-12 > > 1 KSP preconditioned resid norm 2.345517221383e-12 true resid norm > 6.645776037775e-10 ||r(i)||/||b|| 6.650435020976e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 8.954956134809e+03 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 5 SNES Function norm 8.954956134809e+03 > > 0 KSP Residual norm 1.869342714623e+02 > > 0 KSP preconditioned resid norm 1.869342714623e+02 true resid norm > 8.954956134809e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 6.952075013553e-12 > > 1 KSP preconditioned resid norm 6.952075013553e-12 true resid norm > 2.332836306210e-09 ||r(i)||/||b|| 2.605078429298e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 9.366608314830e+04 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.702858832608e+04 lambda=5.0000000000000003e-02 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 9.141708166107e+03 lambda=2.5000000000000001e-02 > > Line search: Cubically determined step, current gnorm > 8.847002105227e+03 lambda=1.2500000000000001e-02 > > 6 SNES Function norm 8.847002105227e+03 > > 0 KSP Residual norm 4.136774315749e+01 > > 0 KSP preconditioned resid norm 4.136774315749e+01 true resid norm > 8.847002105227e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 2.340140336645e-12 > > 1 KSP preconditioned resid norm 2.340140336645e-12 true resid norm > 6.952836046439e-10 ||r(i)||/||b|| 7.858974106416e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 7.928904942575e+03 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 7 SNES Function norm 7.928904942575e+03 > > 0 KSP Residual norm 9.421715655897e+01 > > 0 KSP preconditioned resid norm 9.421715655897e+01 true resid norm > 7.928904942575e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 5.662350656880e-12 > > 1 KSP preconditioned resid norm 5.662350656880e-12 true resid norm > 1.323447274435e-09 ||r(i)||/||b|| 1.669142566370e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 4.679455846837e+04 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.024301325332e+04 lambda=5.0000000000000003e-02 > > Line search: Cubically determined step, current gnorm > 7.882357228991e+03 lambda=2.5000000000000001e-02 > > 8 SNES Function norm 7.882357228991e+03 > > 0 KSP Residual norm 3.101373992455e+01 > > 0 KSP preconditioned resid norm 3.101373992455e+01 true resid norm > 7.882357228991e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 1.176793992597e-12 > > 1 KSP preconditioned resid norm 1.176793992597e-12 true resid norm > 4.948698048884e-10 ||r(i)||/||b|| 6.278195602050e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: Using full step: fnorm 7.882357228991e+03 gnorm > 5.263287196819e+03 > > 9 SNES Function norm 5.263287196819e+03 > > 0 KSP Residual norm 8.541947236232e+00 > > 0 KSP preconditioned resid norm 8.541947236232e+00 true resid norm > 5.263287196819e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 1.741221887649e-12 > > 1 KSP preconditioned resid norm 1.741221887649e-12 true resid norm > 9.735248249306e-10 ||r(i)||/||b|| 1.849651726242e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 4.836381737060e+03 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 10 SNES Function norm 4.836381737060e+03 > > 0 KSP Residual norm 1.926518638579e+01 > > 0 KSP preconditioned resid norm 1.926518638579e+01 true resid norm > 4.836381737060e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 1.278653333162e-11 > > 1 KSP preconditioned resid norm 1.278653333162e-11 true resid norm > 1.211659620475e-08 ||r(i)||/||b|| 2.505301868938e-12 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 4.550187794983e+03 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 11 SNES Function norm 4.550187794983e+03 > > 0 KSP Residual norm 5.492305435155e+00 > > 0 KSP preconditioned resid norm 5.492305435155e+00 true resid norm > 4.550187794983e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.941029797735e-13 > > 1 KSP preconditioned resid norm 3.941029797735e-13 true resid norm > 2.123615113926e-10 ||r(i)||/||b|| 4.667093336823e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 3.210238905481e+03 > > Line search: Quadratically determined step, > lambda=4.4544417493894700e-01 > > 12 SNES Function norm 3.210238905481e+03 > > 0 KSP Residual norm 7.578202966222e+00 > > 0 KSP preconditioned resid norm 7.578202966222e+00 true resid norm > 3.210238905481e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.800509410151e-12 > > 1 KSP preconditioned resid norm 3.800509410151e-12 true resid norm > 3.408323024026e-09 ||r(i)||/||b|| 1.061703855812e-12 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 2.924559961092e+03 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 13 SNES Function norm 2.924559961092e+03 > > 0 KSP Residual norm 5.076980628354e+00 > > 0 KSP preconditioned resid norm 5.076980628354e+00 true resid norm > 2.924559961092e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 1.468522311180e-13 > > 1 KSP preconditioned resid norm 1.468522311180e-13 true resid norm > 9.059064389128e-11 ||r(i)||/||b|| 3.097582032733e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 2.389633730237e+03 > > Line search: Quadratically determined step, > lambda=2.1257217805235396e-01 > > 14 SNES Function norm 2.389633730237e+03 > > 0 KSP Residual norm 5.166598232781e+00 > > 0 KSP preconditioned resid norm 5.166598232781e+00 true resid norm > 2.389633730237e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 2.249406847764e-13 > > 1 KSP preconditioned resid norm 2.249406847764e-13 true resid norm > 4.780078098549e-11 ||r(i)||/||b|| 2.000339231098e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 2.093485770827e+03 > > Line search: Quadratically determined step, > lambda=1.5993239844479040e-01 > > 15 SNES Function norm 2.093485770827e+03 > > 0 KSP Residual norm 8.445727235216e+00 > > 0 KSP preconditioned resid norm 8.445727235216e+00 true resid norm > 2.093485770827e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 1.917214128329e-13 > > 1 KSP preconditioned resid norm 1.917214128329e-13 true resid norm > 1.577089911021e-10 ||r(i)||/||b|| 7.533320421842e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 1.967949189728e+03 > > Line search: Quadratically determined step, > lambda=1.0000000000000001e-01 > > 16 SNES Function norm 1.967949189728e+03 > > 0 KSP Residual norm 2.548962277036e+01 > > 0 KSP preconditioned resid norm 2.548962277036e+01 true resid norm > 1.967949189728e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 4.393171339150e-12 > > 1 KSP preconditioned resid norm 4.393171339150e-12 true resid norm > 6.268541493766e-10 ||r(i)||/||b|| 3.185316738097e-13 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 3.445905389608e+03 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 2.129006719528e+03 lambda=5.0000000000000003e-02 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.969320033677e+03 lambda=1.8287150237426619e-02 > > Line search: Cubically determined step, current gnorm > 1.963119931876e+03 lambda=8.7403113229804347e-03 > > 17 SNES Function norm 1.963119931876e+03 > > 0 KSP Residual norm 1.189834379636e+02 > > 0 KSP preconditioned resid norm 1.189834379636e+02 true resid norm > 1.963119931876e+03 ||r(i)||/||b|| 1.000000000000e+00 > > 1 KSP Residual norm 3.213937715681e-11 > > 1 KSP preconditioned resid norm 3.213937715681e-11 true resid norm > 8.137767291258e-08 ||r(i)||/||b|| 4.145323553147e-11 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Line search: gnorm after quadratic fit 1.329932469429e+05 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 2.079971894561e+04 lambda=5.0000000000000003e-02 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 4.672632539568e+03 lambda=2.5000000000000001e-02 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 2.337320012333e+03 lambda=1.2500000000000001e-02 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.989655494060e+03 lambda=3.7908486903933652e-03 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.964061592872e+03 lambda=4.3979444116724767e-04 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963285343925e+03 lambda=9.8760011176842895e-05 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963156689078e+03 lambda=2.3385231527184855e-05 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963128672876e+03 lambda=5.6482781216089357e-06 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963122042807e+03 lambda=1.3692247365910180e-06 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963120443658e+03 lambda=3.3226543340386796e-07 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963120056069e+03 lambda=8.0648305027594934e-08 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119962021e+03 lambda=1.9576315045380916e-08 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119939193e+03 lambda=4.7519586695347587e-09 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119933652e+03 lambda=1.1534950726729967e-09 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119932307e+03 lambda=2.8000026447574534e-10 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119931981e+03 lambda=6.7967445495582766e-11 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119931902e+03 lambda=1.6498100190994443e-11 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119931882e+03 lambda=4.0046243381676681e-12 > > Line search: Cubic step no good, shrinking lambda, current gnorm > 1.963119931878e+03 lambda=9.7209812173761137e-13 > > Line search: unable to find good step length! After 19 tries > > Line search: fnorm=1.9631199318761367e+03, > gnorm=1.9631199318776339e+03, ynorm=1.1898343796305618e+02, > minlambda=9.9999999999999998e-13, lambda=9.7209812173761137e-13, initial > slope=-3.8538398668951751e+06 > > Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH iterations 17 > > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > > http://www.caam.rice.edu/~mk51/ >
