I'm trying to write a program to perform a fitting procedure for
logistic regression but I've come to a point where my implementation
doesn't give the expected answers (comparing to book results). I'm an
engineer rather than a mathematician or statistician but I really want
to understand how the process works.
I've been using 3 books to understand the process - see references at
the bottom.
At first I implemented the algorithm for multivariate linear
regression, which works. I then extended this in an attempt to fit
more general models as described in (3). The results do converge but
not to the answers I expect and I don't understand why.
There are some parts of the algorithm described in (3)-p40 I'm unsure
of:
- How do you calculate initial estimates for the Beta (coefficient)
values? I've hard coded initial estimates and found values which
either always cause divergence or always converge to the same answer.
- What's the best measure to use as a test for stopping the
iterations? I've been looking at the change in R-squared from the
linear regression result.
- How do you use the calculated quadratic weight to regress the
adjusted DV (z) on the covariates. I've been multiplying the IV values
by the weight - is this correct?
- When converting from the standardized coefficients to the unstandard
coefficients I've been using the formula:
Bi = Bis * Sy / Si
where: Bi is the i'th Beta value
Bis is the i'th standardized Beta value
Sy is the adjusted DV (z) standard deviation
Si is the standard deviation of the i'th weighted IV (i.e IV *
w)
Is this correct, or should I be using the actual DV or IV standard
deviations?
It may be that a mistake in any or all of these areas is my problem.
Any help would be appreciated.
References:
(1) "Using Multivariate Statistics", Tabachnick & Fidell, 4th Edition
(gives a method for solving general linear equations for multivariate
regression)
(2) "Applied Logistic Regression", Hosmer & Lemeshow, 2nd Edition
(3) "Generalized Linear Models", McCullagh & Nelder, 2nd Edition
Thanks in advance,
Darren.
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
