[R] maximum likelihood estimation of 5 parameters
Hi Guys, it would be great if you could help me with a MLE problem in R.
I am trying to evaluate the maximum likelihood estimates of theta = (a1,
b1, a2, b2, P) which defines a mixture of a Poisson distribution and two
gamma prior distributions (where the Poisson means have a gamma
distribution, actually 2 gammas and P is the mixing factor). The likelihood
function for theta is L(theta) = Pi,j{P f(Nij; a1, b1, Eij) + (1 – P) f(Nij;
a2, b2, Eij),}
The maximum likelihood estimate of theta is the vector that maximizes the
above equation (the values of N and E are given). The authors of the article
I read say that the maximization involves an iterative search in the five
dimensional parameter space, where each iteration involves computing
log[L(theta)] and its first and second-order derivatives. In test runs it is
suggested that the maximization typically takes between 5 and 15 iterations
from the starting point theta = (a1 = 0.2, b1 = 0.1, a2 = 2, b2 = 4, P =
1/3).
Now I have done maximization of a gamma-poisson mixture before (1 poisson, 1
gamma) successfully and I could determine correctly alpha (a) and beta(a).
But this one above is giving me ridiculously large unusable values (for
example P should not be above 1 and sometimes I get values of 500!) or even
negative values! I know the values I should be obtaining with my samples
shouldn't be far from the staring points. Is there a way to help me solve
this issue? Thanks.
--
View this message in context:
http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf2925364.html#a8177473
Sent from the R help mailing list archive at Nabble.com.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] maximum likelihood estimation of 5 parameters
Franco,
You can provide lower and upper bounds on the parameters if you use optim
with method=L-BFGS-B.
Hth, Ingmar
From: francogrex [EMAIL PROTECTED]
Date: Fri, 5 Jan 2007 04:54:50 -0800 (PST)
To: r-help@stat.math.ethz.ch
Subject: [R] maximum likelihood estimation of 5 parameters
Hi Guys, it would be great if you could help me with a MLE problem in R.
I
am trying to evaluate the maximum likelihood estimates of theta = (a1,
b1,
a2, b2, P) which defines a mixture of a Poisson distribution and two
gamma
prior distributions (where the Poisson means have a gamma
distribution,
actually 2 gammas and P is the mixing factor). The likelihood
function for
theta is L(theta) = Pi,j{P f(Nij; a1, b1, Eij) + (1 P) f(Nij;
a2, b2, Eij),}
The maximum likelihood estimate of theta is the vector that maximizes
the
above equation (the values of N and E are given). The authors of the
article
I read say that the maximization involves an iterative search in the
five
dimensional parameter space, where each iteration involves
computing
log[L(theta)] and its first and second-order derivatives. In test
runs it is
suggested that the maximization typically takes between 5 and 15
iterations
from the starting point theta = (a1 = 0.2, b1 = 0.1, a2 = 2, b2 =
4, P =
1/3).
Now I have done maximization of a gamma-poisson mixture before
(1 poisson, 1
gamma) successfully and I could determine correctly alpha (a)
and beta(a).
But this one above is giving me ridiculously large unusable
values (for
example P should not be above 1 and sometimes I get values of
500!) or even
negative values! I know the values I should be obtaining with my
samples
shouldn't be far from the staring points. Is there a way to help me
solve
this issue? Thanks.
--
View this message in context:
http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf2925364.
html#a8177473
Sent from the R help mailing list archive at
Nabble.com.
__
[EMAIL PROTECTED]
thz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do
read the posting guide http://www.R-project.org/posting-guide.html
and provide
commented, minimal, self-contained, reproducible code.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] maximum likelihood estimation of 5 parameters
Franco, You can provide lower and upper bounds on the parameters if you use optim with method=L-BFGS-B. Hth, Ingmar Thanks, but when I use L-BFGS-B it tells me that there is an error in optim(start, f, method = method, hessian = TRUE, ...) : L-BFGS-B needs finite values of 'fn' -- View this message in context: http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf2925364.html#a8180120 Sent from the R help mailing list archive at Nabble.com. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] maximum likelihood estimation of 5 parameters
On Fri, 5 Jan 2007, francogrex wrote: [quoting Ingmar Vissar without attribution, contrary to the posting guide.] Franco, You can provide lower and upper bounds on the parameters if you use optim with method=L-BFGS-B. Hth, Ingmar Thanks, but when I use L-BFGS-B it tells me that there is an error in optim(start, f, method = method, hessian = TRUE, ...) : L-BFGS-B needs finite values of 'fn' It sounds as if you have ignored the advice to scale the problem via control options 'fnscale' and 'parscale'. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] maximum likelihood estimation of 5 parameters
Franco, Is it possible that you have failed to provide the negative of loglikelihood to optim, since optim, by default, minimizes a function? If you want to do this withput redefining the log-likelihood, you should set fnscale= -1 (as hinted by Prof. Ripley). This would turn the problem into a maximization problem. If this doesn't work, you should provide more details (a reproducible code with actual error message). Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: [EMAIL PROTECTED] Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of francogrex Sent: Friday, January 05, 2007 10:42 AM To: r-help@stat.math.ethz.ch Subject: Re: [R] maximum likelihood estimation of 5 parameters Franco, You can provide lower and upper bounds on the parameters if you use optim with method=L-BFGS-B. Hth, Ingmar Thanks, but when I use L-BFGS-B it tells me that there is an error in optim(start, f, method = method, hessian = TRUE, ...) : L-BFGS-B needs finite values of 'fn' -- View this message in context: http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf292536 4.html#a8180120 Sent from the R help mailing list archive at Nabble.com. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] maximum likelihood estimation of 5 parameters
Using the inverse logistic transform to replace p by exp(xp)/(1+exp(xp)) allows
unconstrained fitting of xp. There may still be problems where xp tends to + or
- infinity depending on starting values.
francogrex [EMAIL PROTECTED] 01/05/07 11:54 PM
Hi Guys, it would be great if you could help me with a MLE problem in R.
I am trying to evaluate the maximum likelihood estimates of theta = (a1,
b1, a2, b2, P) which defines a mixture of a Poisson distribution and two
gamma prior distributions (where the Poisson means have a gamma
distribution, actually 2 gammas and P is the mixing factor). The likelihood
function for theta is L(theta) = Pi,j{P f(Nij; a1, b1, Eij) + (1 * P) f(Nij;
a2, b2, Eij),}
The maximum likelihood estimate of theta is the vector that maximizes the
above equation (the values of N and E are given). The authors of the article
I read say that the maximization involves an iterative search in the five
dimensional parameter space, where each iteration involves computing
log[L(theta)] and its first and second-order derivatives. In test runs it is
suggested that the maximization typically takes between 5 and 15 iterations
from the starting point theta = (a1 = 0.2, b1 = 0.1, a2 = 2, b2 = 4, P =
1/3).
Now I have done maximization of a gamma-poisson mixture before (1 poisson, 1
gamma) successfully and I could determine correctly alpha (a) and beta(a).
But this one above is giving me ridiculously large unusable values (for
example P should not be above 1 and sometimes I get values of 500!) or even
negative values! I know the values I should be obtaining with my samples
shouldn't be far from the staring points. Is there a way to help me solve
this issue? Thanks.
--
View this message in context:
http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf2925364.html#a8177473
Sent from the R help mailing list archive at Nabble.com.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] maximum likelihood estimation of 5 parameters
No I have not forgotten to use a negative fnscale to optimize, so as you
suggest I will post some parts of the code I am running to show you the
errors:
n
[1] 3 1 4 54 6 58 20 14 3 14 4 65 1 7 9 10 2 4
66
[20] 5 9 7 12 7 55 105 2 5 10 55 5 28 1 1 6 2 1
30
[39] 6 49 7 21 8 7
e
[1] 21.763201 1.209070 4.836270 32.644798 19.546600 24.584400 30.226700
[8] 6.045340 14.010100 3.113350 21.015100 12.583100 15.826200 19.458401
[15] 3.891690 1.329970 0.241814 3.143580 13.057900 0.725441 18.136000
[22] 2.187660 6.319900 1.701510 29.654900 36.460999 7.292190 1.215370
[29] 3.209070 19.995001 11.972300 3.455920 0.138539 0.113350 1.360200
[36] 1.889170 1.518890 18.226700 4.050380 27.340099 1.181360 16.370300
[43] 20.589399 25.314899
fr-function(a1,b1,a2,b2,p){
+
+ w-((gamma(a1+n)))/((gamma(a1)*factorial(n))*(1+(e/b1)^a1)*(1+(b1/e)^n))
+ z-((gamma(a2+n)))/((gamma(a2)*factorial(n))*(1+(e/b2)^a2)*(1+(b2/e)^n))
+
+ sum (log( (p*w)+ ((1-p)*z) ))
+
+ }
mle((fr),
start=list(a1=0.2,b1=0.1,a2=2,b2=4,p=0.33),method=BFGS,control=list(fnscale=-1))
Error in optim(start, f, method = method, hessian = TRUE, ...) :
non-finite finite-difference value [2]
And with the L-BFGS-B:
Error in optim(start, f, method = method, hessian = TRUE, ...) :
L-BFGS-B needs finite values of 'fn'
AND WITH NELDER-MEAD it doesn't work either (same error), but when I change
intial parameters (though I shouldn't, it gives something very weird
(negatives or sometimes huge values).
Call:
mle(minuslogl = (fr), start = list(a1 = 1, b1 = 1, a2 = 10, b2 = 10,
p = 0.9), method = Nelder-Mead, control = list(fnscale = -1))
Coefficients:
a1 b1 a2 b2 p
-2.5035823 0.6236359 26.5562988 12.9604112 -0.1383767
Thanks
Ravi Varadhan wrote:
Franco,
Is it possible that you have failed to provide the negative of
loglikelihood
to optim, since optim, by default, minimizes a function? If you want to
do this withput redefining the log-likelihood, you should set fnscale= -1
(as hinted by Prof. Ripley). This would turn the problem into a
maximization problem.
If this doesn't work, you should provide more details (a reproducible code
with actual error message).
Ravi.
---
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: [EMAIL PROTECTED]
Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of francogrex
Sent: Friday, January 05, 2007 10:42 AM
To: r-help@stat.math.ethz.ch
Subject: Re: [R] maximum likelihood estimation of 5 parameters
Franco,
You can provide lower and upper bounds on the parameters if you use optim
with method=L-BFGS-B.
Hth, Ingmar
Thanks, but when I use L-BFGS-B it tells me that there is an error in
optim(start, f, method = method, hessian = TRUE, ...) : L-BFGS-B needs
finite values of 'fn'
--
View this message in context:
http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf292536
4.html#a8180120
Sent from the R help mailing list archive at Nabble.com.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
View this message in context:
http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf2925364.html#a8186869
Sent from the R help mailing list archive at Nabble.com.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] maximum likelihood estimation of 5 parameters
] maximum likelihood estimation of 5 parameters Franco, You can provide lower and upper bounds on the parameters if you use optim with method=L-BFGS-B. Hth, Ingmar Thanks, but when I use L-BFGS-B it tells me that there is an error in optim(start, f, method = method, hessian = TRUE, ...) : L-BFGS-B needs finite values of 'fn' -- View this message in context: http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf292536 4.html#a8180120 Sent from the R help mailing list archive at Nabble.com. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- View this message in context: http://www.nabble.com/maximum-likelihood-estimation-of-5-parameters-tf2925364.html#a8186869 Sent from the R help mailing list archive at Nabble.com. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Charles C. Berry(858) 534-2098 Dept of Family/Preventive Medicine E mailto:[EMAIL PROTECTED] UC San Diego http://biostat.ucsd.edu/~cberry/ La Jolla, San Diego 92093-0717 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
