On Fri, 2 Jun 2000, Arie Beresteanu wrote:
> Estimation of linear (multivariate) regression with equality
> constraints on the coefficients is a well known problem (at least for
> me). What about if the constrains are inequalities? More specifically:
> Y=Xb+e
> s.t.
> Qb<=q
>
> where Q is a matrix and q is a vector. (for example Y=b0+b1*X1+b2*X2+e
> s.t. b1+2*b2>=0 )
>
> How do I solve that?
Intriguing problem. I suspect that a general
answer, if there is one, requires imposing the constraint first, then
doing the usual least-squares calculus. But here's one way of thinking
about it, in case you find it helpful.
Presumably you want a least-squares (LS) estimate, or set of
estimates. The usual LS solution, with no constraints on the b's,
either happens to meet your desired constraint (e.g., b1+2*b2 >= 0),
or it doesn't. If it does, well and good, and you can proceed to
whatever the next step is.
If it does not, then what (I suppose!) you want is the set of
estimates that gives the smallest sum of squared residuals, subject to
the criterion stated in the constraint, but with the equality (since --
I think this is right -- the ordinary LS solution lies outside the region
specified in the constraint, the equality ought to be the closest part of
that region to the LS solution. I suppose there may exist
(pathological?) constraints for which this is not true, but
"b1+2*b2 >= 0" surely isn't one of them; and I suspect your more
general conmstraint ( Qb <= q for some vector q ) doesn't generate
such nastinesses either.).
So: rewrite the equations (the "normal equations"). First,
write an expression for the sum of squared residuals SSR; then impose
the constraint (b1 + 2*b2 = 0, or whatever) and work through THAT
algebra to a simplified version (that now contains either b1 or b2
but not both); then take partial derivatives of the re-expressed SSR
with respect to all the parameters remaining to be estimated, set each
partial derivative = 0, and solve the resulting set of simultaneous
equations for the parameter estimates.
> How do I test the constraint?
Well, since you will have built the contraint into the
solutions, I shouldn't think a test would be necessary -- or, for that
matter, make much sense.
> Is there something on MatLab/STATA/SAS for that?
I don't know. Maybe someone else does.
-- DFB.
------------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED]
MSC #29, Plymouth, NH 03264 603-535-2597
184 Nashua Road, Bedford, NH 03110 603-471-7128
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
