[
https://issues.apache.org/jira/browse/MATH-607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060774#comment-13060774
]
greg sterijevski commented on MATH-607:
---------------------------------------
Phil,
underlying solver is QR or Gaussian this info would exist. If the underlying
method is SVD, then we would register the rank reduction, but we would not
be able to attribute it to a particular column in the design matrix.
I am probably in agreement with with making RegressionResults concrete, but
there were a couple of considerations which forced me to interface.
Say that I begin with the following augmented matrix:
| X'X X'Y|
| X'Y Y'Y|
where X is the design matrix ( nobs x nreg ), Y is the dependent variable
(nobs x 1 )
On a copy of the cross products matrix (the thing above), I get the
following via gaussian elimination:
| inv(X'X) -beta|
| -beta e'e|
inv(X'X) is the inverse of the X'X matrix. -beta is the OLS vector of
slopes. e'e is the sum of squared errors.
Getting most of the info (that RegressionResults surfaces) is simply a
matter of indexing. All I need to do in this case is write a wrapper around
a symmetric matrix which implements the interface.
I suppose that there could be constructor which took the matrix above and
did the indexing, but that seems too dirty. Furthermore, there are probably
other optimized formats for OLS which have similar aspects. I wanted to keep
the door open to other schemes, without making (potentially large) copies of
variance matrices, standard errors and so forth a necessity.
On the name of the getter for number of observations, I am okay with
whatever you feel is a better name.
So you are saying the UpdatingOLSRegression be an abstract class? There are
not that many methods in the interface. That would be okay if were sure that
subclasses always overrode either the regress(...) methods or the
addObservations(...) methods. I worry that you might get have a base class
full of nothing but abstract functions.
So, modulo the one name change, I propose to just change these to classes
> Current Multiple Regression Object does calculations with all data incore.
> There are non incore techniques which would be useful with large datasets.
> -----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MATH-607
> URL: https://issues.apache.org/jira/browse/MATH-607
> Project: Commons Math
> Issue Type: New Feature
> Affects Versions: 3.0
> Environment: Java
> Reporter: greg sterijevski
> Labels: Gentleman's, QR, Regression, Updating, decomposition,
> lemma
> Fix For: 3.0
>
> Attachments: updating_reg_ifaces
>
> Original Estimate: 840h
> Remaining Estimate: 840h
>
> The current multiple regression class does a QR decomposition on the complete
> data set. This necessitates the loading incore of the complete dataset. For
> large datasets, or large datasets and a requirement to do datamining or
> stepwise regression this is not practical. There are techniques which form
> the normal equations on the fly, as well as ones which form the QR
> decomposition on an update basis. I am proposing, first, the specification of
> an "UpdatingLinearRegression" interface which defines basic functionality all
> such techniques must fulfill.
> Related to this 'updating' regression, the results of running a regression on
> some subset of the data should be encapsulated in an immutable object. This
> is to ensure that subsequent additions of observations do not corrupt or
> render inconsistent parameter estimates. I am calling this interface
> "RegressionResults".
> Once the community has reached a consensus on the interface, work on the
> concrete implementation of these techniques will take place.
> Thanks,
> -Greg
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
