I regularly optimize functions of over 1000 parameters for posterior mode computations using a variant of newton-raphson. I have some favorable conditions: the prior is pretty good, the posterior is smooth, and I can compute the gradient and hessian.
albyn On Mon, Jun 19, 2006 at 06:53:00PM +0100, Patrick Burns wrote: > Seagulls have a very different perspective to ballparks > than ants. Nonetheless, there is something that can be > said. > > There are several variables in addition to the number of > parameters that are important. These include: > > * The complexity of the likelihood > > * The number of observations in the dataset > > * How close to the optimum is close enough > > * Your patience > > The latter is undoubtedly the most important of all. It > matters a lot whether you think a minute is a long time > or only periods measured in weeks. > > The optimization strategy can also have a big effect. If > you are using a derivative-based optimizer, then the number > of parameters can have a big impact. Typically one iteration > in such algorithms requires p+1 function calls, where p is the > number of parameters. Since more iterations are generally > required with more parameters, the speed can decrease > rapidly as the number of parameters increases. > > One strategy to deal with a large number of parameters is to > start with something like a genetic algorithm. Once the genetic > algorithm has a pretty good solution, then switch to a derivative- > based algorithm to finish. The amount to run the initial > algorithm before switching depends on the problem, the quality > of the two optimizers, and probably other things. > > With this switching strategy and at least a modicum of patience, > problems with thousands of parameters may be feasible to solve. > > Patrick Burns > [EMAIL PROTECTED] > +44 (0)20 8525 0696 > http://www.burns-stat.com > (home of S Poetry and "A Guide for the Unwilling S User") > > Federico Calboli wrote: > > >Hi All, > > > >I would like to know, is there a *ballpark* figure for how many > >parameters the minimisation routines can cope with? > > > >I'm asking because I was asked if I knew. > > > >Cheers, > > > >Federico > > > >-- > >Federico C. F. Calboli > >Department of Epidemiology and Public Health > >Imperial College, St. Mary's Campus > >Norfolk Place, London W2 1PG > > > >Tel +44 (0)20 75941602 Fax +44 (0)20 75943193 > > > >f.calboli [.a.t] imperial.ac.uk > >f.calboli [.a.t] gmail.com > > > >______________________________________________ > >R-help@stat.math.ethz.ch mailing list > >https://stat.ethz.ch/mailman/listinfo/r-help > >PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > > > > > > > > > > ______________________________________________ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html ______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html