I think the idea of parameter and intrinsic nonlinearity is due to Beale (JRSSB 1960). Was he Doug Bates' thesis advisor?
Ravi. ----- Original Message ----- From: Spencer Graves <[EMAIL PROTECTED]> Date: Saturday, September 2, 2006 2:05 pm Subject: Re: [R] nonlinear least squares fitting Trust-Region" To: RAVI VARADHAN <[EMAIL PROTECTED]> Cc: Prof Brian Ripley <[EMAIL PROTECTED]>, Martin Ivanov <[EMAIL PROTECTED]>, [email protected] > May I also suggest Bates and Watts (1988) Nonlinear > Regression > Analysis and Its Applications (Wiley). This book carefully > explains the > difference between "parameter effects" and "intrinsic" curvature > in > non-linear fitting. I don't know if this idea was original with > Bates or > Watts, but I believe that Bates' PhD dissertation made important, > original contributions to our understanding of it -- and it helped > get > him the faculty position in Statistics at the University of > Wisconsin, > where he still is. Bates is also a leading contributor to R. > > hope this helps. > spencer graves > > RAVI VARADHAN wrote: > > As suggested by Prof. Ripley, you should read a good book in the > optimization area. One that I would highly recommend is the book > by Dennis and Schnabel (1983) - Numerical methods for > unconstrained optimization, which does a great job of explaining > both "line-search" and "trust-region" approaches for achieving > globally-convergent versions of a fast numerical scheme such as > Gauss-Newton. > > > > Best, > > Ravi. > > > > ----- Original Message ----- > > From: Prof Brian Ripley <[EMAIL PROTECTED]> > > Date: Saturday, September 2, 2006 5:51 am > > Subject: Re: [R] nonlinear least squares fitting Trust-Region" > > To: Martin Ivanov <[EMAIL PROTECTED]> > > Cc: [email protected] > > > > > >> I believe people (including me) did not reply because you > appeared > >> not to > >> have done your homework. The help page for ?nls _does_ have a > >> reference > >> to the 'port' documentation, and RSiteSearch("trust region") is > >> informative and leads to an R package that does trust-region > >> optimization. > >> (So would looking in the R FAQ.) > >> > >> You say: > >> > >> > >>> Since I am not an expert in the field of optimization, I am > just > >>> conforming to what matlab documentation > >>> > >> Please note that some of the R developers are really expert in > >> that area, > >> and their advice (in the R documentation) should be taken as > >> seriously as > >> that in some commercial package that is merely commenting about > >> the very > >> sparse choice it offers. Or if R is not in your personal trust > >> region, > >> just use 'matlab'. > >> > >> Please > >> > >> 1) do not shout at your helpers: using all caps is regarded as > >> shouting. > >> 2) study and follow the posting guide. People are much more > >> likely to > >> help you if you demonstrate you have made efforts to help yourself. > >> > >> 3) read the literature. The R FAQ leads to books that cover > >> fitting > >> non-linear models in S/R in considerable detail. > >> > >> > >> On Sat, 2 Sep 2006, Martin Ivanov wrote: > >> > >> > >>> Dear Mr Graves, > >>> > >>> Thank you very much for your response. Nobody else from this > >>> > >> mailing > >> > >>> list ventured to reply to me for the two weeks since I posted > my > >>> question. "nlminb" and "optim" are just optimization > procedures. > >>> > >> What I > >> > >>> need is not just optimization, but a nonlinear CURVE FITTING > >>> > >> procedure. > >> Which is just optimization: usually by least squares (although > you > >> have > >> not actually specified that and there are better modern > >> statistical > >> ideas). > >> > >> > >>> If there is some way to perform nonlinear curve fitting with > the > >>> "Trust-Region" algorithm using any of these functions, I would > >>> > >> me much > >> > >>> obliged to you if you suggest to me how to achieve that. You > >>> > >> asked me > >> > >>> why I do not want Gauss-Newton. Since I am not an expert in > the > >>> > >> field of > >> > >>> optimization, I am just conforming to what matlab > documentation > >>> suggests, namely: "Algorithm used for the fitting procedure: > >>> Trust-Region -- This is the default algorithm and must be used > >>> > >> if you > >> > >>> specify coefficient constraints. Levenberg-Marquardt -- If the > >>> trust-region algorithm does not produce a reasonable fit, and > >>> > >> you do not > >> > >>> have coefficient constraints, you should try the Levenberg- > >>> > >> Marquardt > >> > >>> algorithm. Gauss-Newton --THIS ALGORITHM IS POTENTIALLY FASTER > >>> > >> THAN THE > >> > >>> OTHER ALGORITHMS, BUT IT ASSUMES THAT THE RESIDUALS ARE CLOSE > TO > >>> > >> ZERO. > >> > >>> IT IS INCLUDED FOR PEDAGOGICAL REASONS AND SHOULD BE THE LAST > >>> > >> CHOICE FOR > >> > >>> MOST MODELS AND DATA SETS. I browsed some literature about the > >>> > >> garchfit > >> > >>> function, but I did not see the "Trust-Region" algorithm there > >>> > >> either: > >> > >>> algorithm = c("sqp", "nlminb", "lbfgsb", "nlminb+nm", > >>> > >> "lbfgsb+nm"), > >> > >>> control = list(), title = NULL, description = NULL, ...) > >>> > >>> Thank you for your attention. I am looking forward to your reply. > >>> Regards, > >>> Martin > >>> > >>> --------------------------------------------------------------- > -- > >>> vbox7.com - ??????? ????? ???????! > >>> > >>> ______________________________________________ > >>> [email protected] mailing list > >>> https://stat.ethz.ch/mailman/listinfo/r-help > >>> PLEASE do read the posting guide http://www.R- > >>> > >> project.org/posting-guide.html > >> > >>> and provide commented, minimal, self-contained, reproducible code. > >>> > >>> > >> -- > >> Brian D. Ripley, [EMAIL PROTECTED] > >> Professor of Applied Statistics, > http://www.stats.ox.ac.uk/~ripley/>> University of Oxford, > Tel: +44 1865 272861 (self) > >> 1 South Parks Road, +44 1865 272866 (PA) > >> Oxford OX1 3TG, UK Fax: +44 1865 272595 > >> > >> ______________________________________________ > >> [email protected] mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide http://www.R- > project.org/posting- > >> guide.htmland provide commented, minimal, self-contained, > >> reproducible code. > >> > >> > > > > ______________________________________________ > > [email protected] mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide http://www.R- > project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
