Hey Jason:
The line search fails. If I use Armijo I get
TaoLineSearch Object:
type: armijo
maxf=30, ftol=1e-10, gtol=0.0001
Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1
maximum function evaluations=30
tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9
total number of function evaluations=1
total number of gradient evaluations=1
total number of function/gradient evaluations=0
Termination reason: 0
The parameters seem to be the default ones also suggested by Nocedal and
Wright. So I did not change anything. The termination reason is equivalent to
TAOLINESEARCH_CONTINUE_ITERATING. I am not checking the reason directly. I
guess it starts reducing the step size after that. I can see that my objective
function get’s evaluated (as expected); however, the objective values increase
(from what I see when monitoring the evaluations of my objective). This leads
to a failure in the line search and made (still makes) me believe there is a
bug on my side (which I have not found yet). However, if I use a unit step it
converges (relative change of the gradient e.g. to 1E-9; see bottom of this
email). If I use More & Thuente, same thing. No reduction in step size
necessary.
If you suggest that I should do some further testing on simpler problems, I’m
happy to do so. After looking at the code, I just felt like there obviously is
something wrong in the line-search implementation.
Thanks for your help.
/Andreas
Here’s the output after the first iteration (where the Armijo line search
fails):
TaoLineSearch Object:
type: armijo
maxf=30, ftol=1e-10, gtol=0.0001
Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1
maximum function evaluations=30
tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9
total number of function evaluations=1
total number of gradient evaluations=1
total number of function/gradient evaluations=0
Termination reason: 0
TaoLineSearch Object:
type: armijo
maxf=30, ftol=1e-10, gtol=0.0001
Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1
maximum function evaluations=30
tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9
total number of function evaluations=30
total number of gradient evaluations=0
total number of function/gradient evaluations=0
Termination reason: 4
With final output (end of optimization):
Tao Object: 1 MPI processes
type: nls
Newton steps: 1
BFGS steps: 0
Scaled gradient steps: 0
Gradient steps: 1
nls ksp atol: 0
nls ksp rtol: 1
nls ksp ctol: 0
nls ksp negc: 0
nls ksp dtol: 0
nls ksp iter: 0
nls ksp othr: 0
TaoLineSearch Object: 1 MPI processes
type: armijo
KSP Object: 1 MPI processes
type: cg
total KSP iterations: 21
convergence tolerances: fatol=0, frtol=0
convergence tolerances: gatol=0, steptol=0, gttol=0.0001
Residual in Function/Gradient:=0.038741
Objective value=0.639121
total number of iterations=0, (max: 50)
total number of function evaluations=31, max: 10000
total number of gradient evaluations=1, max: 10000
total number of function/gradient evaluations=1, (max: 10000)
total number of Hessian evaluations=1
Solver terminated: -6 Line Search Failure
This is without line-search (unit step size):
Tao Object: 1 MPI processes
type: nls
Newton steps: 3
BFGS steps: 0
Scaled gradient steps: 0
Gradient steps: 0
nls ksp atol: 0
nls ksp rtol: 3
nls ksp ctol: 0
nls ksp negc: 0
nls ksp dtol: 0
nls ksp iter: 0
nls ksp othr: 0
TaoLineSearch Object: 1 MPI processes
type: unit
KSP Object: 1 MPI processes
type: cg
total KSP iterations: 71
convergence tolerances: fatol=0, frtol=0
convergence tolerances: gatol=0, steptol=0, gttol=0.0001
Residual in Function/Gradient:=1.91135e-11
Objective value=0.160914
total number of iterations=3, (max: 50)
total number of function/gradient evaluations=4, (max: 10000)
total number of Hessian evaluations=3
Solution converged: ||g(X)||/||g(X0)|| <= gttol
> On Jun 11, 2015, at 12:44 PM, Jason Sarich <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi Andreas,
>
> Yes it looks like a bug that the reason is never set, but the line should
> still terminate. Is the problem you are having with the line search itself,
> or is it failing because you are checking this ls->reason directly?
>
> Jason Sarich
>
>
> On Thu, Jun 11, 2015 at 9:53 AM, Andreas Mang <[email protected]
> <mailto:[email protected]>> wrote:
> Hi guys:
>
> I have a problem with the TAO Armijo line search (petsc-3.5.4). My algorithm
> works if I use the More & Thuente line search (default). I have numerically
> checked the gradient of my objective. It’s correct. I am happy to write a
> small snippet of code and do an easy test if you guys disagree, but from what
> I’ve seen in the line search code it seems obvious to me that there is a bug.
> Am I missing something or are you not setting
>
> ls->reason
>
> to
>
> TAOLINESEARCH_SUCCESS
>
> if the Armijo condition is fulfilled (TaoLineSearchApply_Armijo in armijo.c;
> line 118 - 302)?!
>
> It seems to me that ls->reason is and will remain to be set to
>
> TAOLINESEARCH_CONTINUE_ITERATING
>
> if everything works (i.e. I don’t hit one of the exceptions). Does this make
> sense? If not I’ll invest the time and put together a simple test case and,
> if that works, continue to check my code.
>
> /Andreas
>
>
>
