That is very interesting, thanks for responding. And apologies for not checking the source paper before posting.
However, given what you've said, I think there is something I am not understanding about stopping criteria on the objective function. Is this an appropriate forum to ask the question? If not, I'd be very grateful if you could direct me to a web resource for asking questions about NLopt. If this is an appropriate place, the question follows: I've got an optimisation problem that I'm trying to solve with COBYLA. My problem is a bit lengthy to reproduce here, but I can duplicate the issue on a much simpler problem. If my understanding is correct (and it may not be) then f_tol_rel and f_tol_abs don't appear to be working for this algorithm. On my machine (Ubuntu 14.04, Julia 0.4, NLopt 0.2.3) the code at the end of this post will print the following sequence of objective function values as the final five steps in COBYLA: 1.161 1.074 1.004 1.017 1.038 Note that in the code, I have f_tol_rel set to 0.5 and f_tol_abs set to 0.5, so by my understanding, both stopping criteria should have halted the algorithm at any of the above 4 steps. In fact, they should have kicked in at any of the final 8 steps. So the question is: what am I missing here? Code follows: using NLopt function objective_function(param::Vector{Float64}, grad::Vector{Float64}) obj_func_value = param[1]^2 + param[2]^2 + 1.0 println("Objective func value = " * string(obj_func_value)) println("Parameter value = " * string(param)) return(obj_func_value) end opt1 = Opt(:LN_COBYLA, 2) lower_bounds!(opt1, [-10.0, -10.0]) upper_bounds!(opt1, [10.0, 10.0]) ftol_rel!(opt1, 0.5) ftol_abs!(opt1, 0.5) min_objective!(opt1, objective_function) (fObjOpt, paramOpt, flag) = optimize(opt1, [9.0, 9.0]) Cheers and thanks again for responding, Colin On Thursday, 7 January 2016 14:42:56 UTC+11, Steven G. Johnson wrote: > > > > On Wednesday, January 6, 2016 at 7:47:07 PM UTC-7, colint...@gmail.com > wrote: >> >> Actually, I spoke to soon. >> >> While your suggestion is very effective for convergence routines that >> call the objective function once at each step, it can lead to some pretty >> confusing results if the objective function is called multiple times at >> each step. For example, some of the routines for derivative-free >> optimisation such as LN_COBYLA will do this in order to construct a linear >> approximation. So for these routines, one has to fairly exhaustively filter >> the printed output in order to get the actual objective function and >> parameter values at a given step. It becomes pretty much impossible as the >> dimension of the problem increases. >> > > That's not how COBYLA works, except in the initialization phase. After > the first N+1 steps in N dimension, it uses the memory of the previous > steps to update its first derivative approximation, rather than doing N+1 > evaluations on each step which would be very expensive. > > (Avoiding the necessity of doing N+1 evaluations on each step is pretty > much the whole point of using a specialized derivative-free algorithm > rather than using a gradient-based algorithm where you use finite > differences.) > > In consequence, just printing the objective function usually gives a > pretty good idea of how it is doing. Often, I will print the both the > current objective function and the best value found so far. >