Hi Andreas, I'm pretty sure this is a bug on my side, directly related to the ls->reason
Can you try fixing src/tao/linesearch/impls/armijo/armijo.c so it sets this correctly? before (line 275): /* Successful termination, update memory */ armP->lastReference = ref; /* Successful termination, update memory */ ls->reason = TAOLINESEARCH_SUCCESS; armP->lastReference = ref; On Thu, Jun 11, 2015 at 3:26 PM, Andreas Mang <[email protected]> wrote: > Hey Jason: > > Thanks for looking into this. In the meantime I have checked against > your elliptic tao example using an Armijo linesearch. It works (i.e. > converges) as you suggested in an earlier email even though it returns the > “wrong" flag. For my problem it does even choke if I set the regularization > parameter to 1E6 (essentially solving a quadratic problem). > > A final question before I continue the struggle by myself: It says > “gradient steps: 1” instead of “gradient steps: 0” in the outputs (Armijo > vs. More-Thuente / Unit). Does it start doing gradient evaluations? Maybe > this helps me to further poke my code. > > I’ll continue to look into this. I’ll come back to you if I discover > that the problem is on the PETSc side of things and I can reproduce the > problem with a toy example. > > Thanks for your time! /Andreas > > > On Jun 11, 2015, at 3:03 PM, Jason Sarich <[email protected]> wrote: > > Hi Andreas, > > I don't see anything obviously wrong. If the function is very flat, you > can try setting -tao_ls_armijo_sigma to a smaller number. If you continue > to have problems, please let me know. It would definitely help if you have > an example you could send me that reproduces this behavior. > > Jason > > > > > On Thu, Jun 11, 2015 at 1:12 PM, Andreas Mang <[email protected]> > wrote: > >> Hey Jason: >> >> The line search fails. If I use Armijo I get >> >> TaoLineSearch Object: >> type: armijo >> maxf=30, ftol=1e-10, gtol=0.0001 >> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 >> memsize=1 >> maximum function evaluations=30 >> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9 >> total number of function evaluations=1 >> total number of gradient evaluations=1 >> total number of function/gradient evaluations=0 >> Termination reason: 0 >> >> The parameters seem to be the default ones also suggested by Nocedal >> and Wright. So I did not change anything. The termination reason is >> equivalent to TAOLINESEARCH_CONTINUE_ITERATING. I am not checking the >> reason directly. I guess it starts reducing the step size after that. I can >> see that my objective function get’s evaluated (as expected); however, the >> objective values increase (from what I see when monitoring the evaluations >> of my objective). This leads to a failure in the line search and made >> (still makes) me believe there is a bug on my side (which I have not found >> yet). However, if I use a unit step it converges (relative change of the >> gradient e.g. to 1E-9; see bottom of this email). If I use More & Thuente, >> same thing. No reduction in step size necessary. >> >> If you suggest that I should do some further testing on simpler >> problems, I’m happy to do so. After looking at the code, I just felt like >> there obviously is something wrong in the line-search implementation. >> >> Thanks for your help. >> /Andreas >> >> *Here’s the output after the first iteration (where the Armijo line >> search fails):* >> >> TaoLineSearch Object: >> type: armijo >> maxf=30, ftol=1e-10, gtol=0.0001 >> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 >> memsize=1 >> maximum function evaluations=30 >> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9 >> total number of function evaluations=1 >> total number of gradient evaluations=1 >> total number of function/gradient evaluations=0 >> Termination reason: 0 >> TaoLineSearch Object: >> type: armijo >> maxf=30, ftol=1e-10, gtol=0.0001 >> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 >> memsize=1 >> maximum function evaluations=30 >> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9 >> total number of function evaluations=30 >> total number of gradient evaluations=0 >> total number of function/gradient evaluations=0 >> Termination reason: 4 >> >> *With final output (end of optimization):* >> >> Tao Object: 1 MPI processes >> type: nls >> Newton steps: 1 >> BFGS steps: 0 >> Scaled gradient steps: 0 >> Gradient steps: 1 >> nls ksp atol: 0 >> nls ksp rtol: 1 >> nls ksp ctol: 0 >> nls ksp negc: 0 >> nls ksp dtol: 0 >> nls ksp iter: 0 >> nls ksp othr: 0 >> TaoLineSearch Object: 1 MPI processes >> type: armijo >> KSP Object: 1 MPI processes >> type: cg >> total KSP iterations: 21 >> convergence tolerances: fatol=0, frtol=0 >> convergence tolerances: gatol=0, steptol=0, gttol=0.0001 >> Residual in Function/Gradient:=0.038741 >> Objective value=0.639121 >> total number of iterations=0, (max: 50) >> total number of function evaluations=31, max: 10000 >> total number of gradient evaluations=1, max: 10000 >> total number of function/gradient evaluations=1, (max: 10000) >> total number of Hessian evaluations=1 >> Solver terminated: -6 Line Search Failure >> >> *This is without line-search (unit step size):* >> >> Tao Object: 1 MPI processes >> type: nls >> Newton steps: 3 >> BFGS steps: 0 >> Scaled gradient steps: 0 >> Gradient steps: 0 >> nls ksp atol: 0 >> nls ksp rtol: 3 >> nls ksp ctol: 0 >> nls ksp negc: 0 >> nls ksp dtol: 0 >> nls ksp iter: 0 >> nls ksp othr: 0 >> TaoLineSearch Object: 1 MPI processes >> type: unit >> KSP Object: 1 MPI processes >> type: cg >> total KSP iterations: 71 >> convergence tolerances: fatol=0, frtol=0 >> convergence tolerances: gatol=0, steptol=0, gttol=0.0001 >> Residual in Function/Gradient:=1.91135e-11 >> Objective value=0.160914 >> total number of iterations=3, (max: 50) >> total number of function/gradient evaluations=4, (max: 10000) >> total number of Hessian evaluations=3 >> Solution converged: ||g(X)||/||g(X0)|| <= gttol >> >> >> >> On Jun 11, 2015, at 12:44 PM, Jason Sarich <[email protected]> >> wrote: >> >> Hi Andreas, >> >> Yes it looks like a bug that the reason is never set, but the line >> should still terminate. Is the problem you are having with the line search >> itself, or is it failing because you are checking this ls->reason directly? >> >> Jason Sarich >> >> >> On Thu, Jun 11, 2015 at 9:53 AM, Andreas Mang <[email protected]> >> wrote: >> >>> Hi guys: >>> >>> I have a problem with the TAO Armijo line search (petsc-3.5.4). My >>> algorithm works if I use the More & Thuente line search (default). I have >>> numerically checked the gradient of my objective. It’s correct. I am happy >>> to write a small snippet of code and do an easy test if you guys disagree, >>> but from what I’ve seen in the line search code it seems obvious to me that >>> there is a bug. Am I missing something or are you not setting >>> >>> ls->reason >>> >>> to >>> >>> TAOLINESEARCH_SUCCESS >>> >>> if the Armijo condition is fulfilled (TaoLineSearchApply_Armijo in >>> armijo.c; line 118 - 302)?! >>> >>> It seems to me that ls->reason is and will remain to be set to >>> >>> TAOLINESEARCH_CONTINUE_ITERATING >>> >>> if everything works (i.e. I don’t hit one of the exceptions). Does this >>> make sense? If not I’ll invest the time and put together a simple test case >>> and, if that works, continue to check my code. >>> >>> /Andreas >>> >>> >>> >> >> > >
