Hi, The best solution would be to identify where the problem is in the FORTRAN code and correct it. However, this problem of premature termination due to absolute function convergence is highly unlikely to occur in practice. As John Nash noted, this is going to be highly unlikely for multi-dimensional parameters (it is also unlikely for one-dimensional problem). However, unless we understand the source of the problem, we cannot feel comfortable in saying with absolute certainty that this will not occur for n > 1. Therefore, I would suggest that either we fix the problem at its source or we set abs.tol=0, since there is little harm in doing so.
Ravi. ____________________________________________________________________ Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvarad...@jhmi.edu ----- Original Message ----- From: Duncan Murdoch <murdoch.dun...@gmail.com> Date: Saturday, July 10, 2010 7:32 am Subject: Re: [R] Not nice behaviour of nlminb (windows 32 bit, version 2.11.1) To: Ravi Varadhan <rvarad...@jhmi.edu> Cc: Matthew Killeya <matthewkill...@googlemail.com>, Peter Ehlers <ehl...@ucalgary.ca>, r-help@r-project.org, ba...@stat.wisc.edu > Ravi Varadhan wrote: > >Hi, > > > >The absolute function stopping criterion is not meant for any > positive objective function. It is meant for functions whose minimum > is 0. Here is what David Gay's documentation from PORT says: > > > >"6 - absolute function convergence: |f (x)| < V(AFCTOL) = V(31). > This test is only of interest in > >problems where f (x) = 0 means a ‘‘perfect fit’’, such as nonlinear > least-squares problems." > > > > Okay, I've taken a more careful look at the docs, and they do say > that the return code 6 does not necessarily indicate convergence: > "The desirable return codes are 3, 4, 5, and sometimes 6". So we > shouldn't by default terminate on it, we should allow users to choose > that if they want faster convergence to perfect fits. > > Would changing the default for abs.tol to zero be a reasonable solution? > > Duncan Murdoch > >For example, let us try a positive objective function: > > > > > >>nlminb( obj = function(x) x^2 + 1, start=1, lower=-Inf, upper=Inf, > control=list(trace=TRUE)) > > 0: 2.0000000: 1.00000 > > 1: 1.0000000: 0.00000 > > 2: 1.0000000: 0.00000 > >$par > >[1] 0 > > > >$objective > >[1] 1 > > > >$convergence > >[1] 0 > > > >$message > >[1] "relative convergence (4)" > > > >$iterations > >[1] 2 > > > >$evaluations > >function gradient 3 2 > > > >Here the absolute function criterion does not kicks in. > > > >Now let us try a function whose minimum value is 0. > > > > > >>nlminb( obj = function(x) x^2, start=6, grad=function(x) 2*x, > lower=-Inf, upper=Inf, control=list(trace=TRUE) ) > >> > > 0: 36.000000: 6.00000 > > 1: 4.0000000: 2.00000 > > 2: 4.9303807e-32: 2.22045e-16 > >$par > >[1] 2.220446e-16 > > > >$objective > >[1] 4.930381e-32 > > > >$convergence > >[1] 0 > > > >$message > >[1] "absolute function convergence (6)" > > > >$iterations > >[1] 2 > > > >$evaluations > >function gradient 4 3 > >We see that convergence is attained and that the stoppage is due to > absolute function criterion. > >Suppose, we now set abs.tol=0: > > > > > >>nlminb( obj = function(x) x^2, start=6, grad=function(x) 2*x, > lower=-Inf, upper=Inf, control=list(trace=TRUE, abs.tol=0) ) > >> > > 0: 36.000000: 6.00000 > > 1: 4.0000000: 2.00000 > > 2: 4.9303807e-32: 2.22045e-16 > > 3: 2.4308653e-63: -4.93038e-32 > > 4: 2.9962729e-95: -5.47382e-48 > > 5:1.4772766e-126: 1.21543e-63 > > 6:1.8208840e-158: 1.34940e-79 > > 7:8.9776511e-190: -2.99627e-95 > > 8:1.1065809e-221: -3.32653e-111 > > 9:5.4558652e-253: 7.38638e-127 > > 10:6.7248731e-285: 8.20053e-143 > > 11:3.3156184e-316: -1.82088e-158 > > 12: 0.0000000: -2.02159e-174 > > 13: 0.0000000: -2.02159e-174 > >$par > >[1] -2.021587e-174 > > > >$objective > >[1] 0 > > > >$convergence > >[1] 0 > > > >$message > >[1] "X-convergence (3)" > > > >$iterations > >[1] 13 > > > >$evaluations > >function gradient 15 13 > > Now, we see that it takes a while to stop, eventhough it is clear > that convergence has been attained after 2 iterations. This > demonstrates the need for the absolute function criterion for obj > functions whose minimum is exactly 0. Although, there is nothing > wrong with setting abs.tol=0, except for some loss of computational > efficiency. > >Now, let us get back to Matthew' example: > > > > > >>nlminb( obj = function(x) x, start=1, lower=-2, upper=2, > control=list(trace=TRUE)) > > 0: 1.0000000: 1.00000 > > 1: 0.0000000: 0.00000 > >$par > >[1] 0 > > > >$objective > >[1] 0 > > > >$convergence > >[1] 0 > > > >$message > >[1] "absolute function convergence (6)" > > > >$iterations > >[1] 1 > > > >$evaluations > >function gradient 2 2 > > > >>nlminb( obj = function(x) x, start=1, lower=-2, upper=2, > control=list(trace=TRUE, abs.tol=0)) > > 0: 1.0000000: 1.00000 > > 1: 0.0000000: 0.00000 > > 2: -2.0000000: -2.00000 > > 3: -2.0000000: -2.00000 > >$par > >[1] -2 > > > >$objective > >[1] -2 > > > >$convergence > >[1] 0 > > > >$message > >[1] "both X-convergence and relative convergence (5)" > > > >$iterations > >[1] 3 > > > >$evaluations > >function gradient 3 3 > > > >Thus it is evident that setting abs.tol=0 is a reasonable, general > solution for functions whose minimum value is non-zero, because it > protects against premature termination at iteration `n' whenever > |f(x_n)| < abs.tol. The only limitation being that of loss of > efficiency in problems where f(x*) = 0. where x* is the local minimum. > > > >Ravi. > >____________________________________________________________________ > > > >Ravi Varadhan, Ph.D. > >Assistant Professor, > >Division of Geriatric Medicine and Gerontology > >School of Medicine > >Johns Hopkins University > > > >Ph. (410) 502-2619 > >email: rvarad...@jhmi.edu > > > > > >----- Original Message ----- > >From: Duncan Murdoch <murdoch.dun...@gmail.com> > >Date: Friday, July 9, 2010 6:54 pm > >Subject: Re: [R] Not nice behaviour of nlminb (windows 32 bit, > version 2.11.1) > >To: Matthew Killeya <matthewkill...@googlemail.com> > >Cc: Peter Ehlers <ehl...@ucalgary.ca>, Ravi Varadhan > <rvarad...@jhmi.edu>, r-help@r-project.org, ba...@stat.wisc.edu > > > > > > > >>On 09/07/2010 6:09 PM, Matthew Killeya wrote: > >> >Yes clearly a bug... there are numerous variations ... problem > seems to be > >> >for a linear function whenever the first function valuation is 1. > >> > Not at all. You can get the same problem on a quadratic that > happens to have a zero at an inconvenient place, e.g. > >> nlminb( obj = function(x) x^2-25, start=6, lower=-Inf, upper=Inf > ) > >> Ravi's workaround of setting the abs.tol to zero fixes this > example, but I think it's pretty clear from the documentation that the > whole thing was designed for positive objective functions, so I > wouldn't count on his workaround solving all the problems. > >> Duncan Murdoch > >> >e.g. two more examples: > >> > nlminb( obj = function(x) x+1, start=0, lower=-Inf, upper=Inf ) > >> > nlminb( obj = function(x) x+2, start=-1, lower=-Inf, upper=Inf ) > >> > > >> >(I wasn't sure where best to report a bug, so emailed the help list) > >> > > >> >On 9 July 2010 22:10, Peter Ehlers <ehl...@ucalgary.ca> wrote: > >> > > >> > >>Actually, it looks like any value other than 1.0 > >> >>(and in (lower, upper)) for start will work. > >> >> > >> >> -Peter Ehlers > >> >> > >> >> > >> >>On 2010-07-09 14:45, Ravi Varadhan wrote: > >> >> > >> >> >>>Setting abs.tol = 0 works! This turns-off the absolute > function > >> >>>convergence > >> >>>criterion. > >> >>> > >> >>> > >> >>> nlminb( objective=function(x) x, start=1, lower=-2, upper=2, > >> >>> control=list(abs.tol=0)) > >> >>>$par > >> >>>[1] -2 > >> >>> > >> >>>$objective > >> >>>[1] -2 > >> >>> > >> >>>$convergence > >> >>>[1] 0 > >> >>> > >> >>>$message > >> >>>[1] "both X-convergence and relative convergence (5)" > >> >>> > >> >>>$iterations > >> >>>[1] 3 > >> >>> > >> >>>$evaluations > >> >>>function gradient > >> >>> 3 3 > >> >>> > >> >>> > >> >>>This is clearly a bug. > >> >>> > >> >>> > >> >>>Ravi. > >> >>> > >> >>>-----Original Message----- > >> >>>From: r-help-boun...@r-project.org [ > >> >>>On > >> >>>Behalf Of Ravi Varadhan > >> >>>Sent: Friday, July 09, 2010 4:42 PM > >> >>>To: 'Duncan Murdoch'; 'Matthew Killeya' > >> >>>Cc: r-help@r-project.org; ba...@stat.wisc.edu > >> >>>Subject: Re: [R] Not nice behaviour of nlminb (windows 32 bit, > version > >> >>>2.11.1) > >> >>> > >> >>>Duncan, `nlminb' is not intended for non-negative functions > only. There > >> >>>is > >> >>>indeed something strange happening in the algorithm! > >> >>> > >> >>>start<- 1.0 # converges to wrong minimum > >> >>> > >> >>>startp<- 1.0 + .Machine$double.eps # correct > >> >>> > >> >>>startm<- 1.0 - .Machine$double.eps # correct > >> >>> > >> >>> nlminb( objective=obj, start=start, lower=-2, upper=2) > >> >>> $par > >> >>>[1] 0 > >> >>> > >> >>>$objective > >> >>>[1] 0 > >> >>> > >> >>>$convergence > >> >>>[1] 0 > >> >>> > >> >>>$message > >> >>>[1] "absolute function convergence (6)" > >> >>> > >> >>>$iterations > >> >>>[1] 1 > >> >>> > >> >>>$evaluations > >> >>>function gradient > >> >>> 2 2 > >> >>> > >> >>> > >> >>> >>>>nlminb( objective=obj, start=startp, lower=-2, upper=2) > >> >>>> > >> >>>> >>>$par > >> >>>[1] -2 > >> >>> > >> >>>$objective > >> >>>[1] -2 > >> >>> > >> >>>$convergence > >> >>>[1] 0 > >> >>> > >> >>>$message > >> >>>[1] "both X-convergence and relative convergence (5)" > >> >>> > >> >>>$iterations > >> >>>[1] 3 > >> >>> > >> >>>$evaluations > >> >>>function gradient > >> >>> 3 3 > >> >>> > >> >>> > >> >>> >>>>nlminb( objective=obj, start=startm, lower=-2, upper=2) > >> >>>> > >> >>>> >>>$par > >> >>>[1] -2 > >> >>> > >> >>>$objective > >> >>>[1] -2 > >> >>> > >> >>>$convergence > >> >>>[1] 0 > >> >>> > >> >>>$message > >> >>>[1] "both X-convergence and relative convergence (5)" > >> >>> > >> >>>$iterations > >> >>>[1] 3 > >> >>> > >> >>>$evaluations > >> >>>function gradient > >> >>> 3 3 > >> >>> > >> >>> > >> >>> From the convergence message the `absolute function > convergence' seems to > >> >>> be > >> >>>the culprit, although I do not understand why that stopping > criterion is > >> >>>becoming effective, when the algorithm is started at x=1, but > not at any > >> >>>other values. The documentation in IPORT makes it clear that this > >> >>>criterion > >> >>>is effective only for functions where f(x*) = 0, where x* is a > local > >> >>>minimum. In this example, x=0 is not a local minimum for f(x), > so that > >> >>>criterion should not apply. > >> >>> > >> >>> > >> >>>Ravi. > >> >>> > >> >>> > >> >>>-----Original Message----- > >> >>>From: r-help-boun...@r-project.org [ > >> >>>On > >> >>>Behalf Of Duncan Murdoch > >> >>>Sent: Friday, July 09, 2010 3:45 PM > >> >>>To: Matthew Killeya > >> >>>Cc: r-help@r-project.org; ba...@stat.wisc.edu > >> >>>Subject: Re: [R] Not nice behaviour of nlminb (windows 32 bit, > version > >> >>>2.11.1) > >> >>> > >> >>>On 09/07/2010 10:37 AM, Matthew Killeya wrote: > >> >>> > >> >>> >>>> nlminb( obj = function(x) x, start=1, lower=-Inf, > upper=Inf ) > >> >>>> > >> >>>> > >> >>>> >>>If you read the PORT documentation carefully, > you'll see that their > >> >>>convergence criteria are aimed at minimizing positive > functions. (They > >> >>>never state this explicitly, as far as I can see.) So one stopping > >> >>>criterion is that |f(x)|< abs.tol, and that's what it found > for you. I > >> >>>don't know if there's a way to turn this off. > >> >>> > >> >>>Doug or Deepayan, do you know if nlminb can be made to work on > functions > >> >>>that go negative? > >> >>> > >> >>>Duncan Murdoch > >> >>> > >> >>> $par > >> >>> >>>>[1] 0 > >> >>>> > >> >>>>$objective > >> >>>>[1] 0 > >> >>>> > >> >>>>$convergence > >> >>>>[1] 0 > >> >>>> > >> >>>>$message > >> >>>>[1] "absolute function convergence (6)" > >> >>>> > >> >>>>$iterations > >> >>>>[1] 1 > >> >>>> > >> >>>>$evaluations > >> >>>>function gradient > >> >>>> 2 2 > >> >>>> > >> >>>> [[alternative HTML version deleted]] > >> >>>> > >> >>>> > >> >>>> > > >> > > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
