[
https://issues.apache.org/jira/browse/SPARK-9834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14732440#comment-14732440
]
DB Tsai commented on SPARK-9834:
--------------------------------
In fact, for linear regression, if the # of features is small, X^TX is the only
required stats for implementing one pass elastic net. Once X^TX is computed, we
can use local solver to optimize the objective function without going through
the data.
Here is the algorithm Kun implemented when he was an intern at Alpine Data
Labs. http://arxiv.org/pdf/1307.0048v1.pdf
Maybe what we can implement is the following. For the # of features < 4096, we
compute the X^TX first. Then if no L1, we compute the model using exact normal
equation, if there is L1, we compute the model using LBFGS with X^TX without
going through the data again.
> Normal equation solver for ordinary least squares
> -------------------------------------------------
>
> Key: SPARK-9834
> URL: https://issues.apache.org/jira/browse/SPARK-9834
> Project: Spark
> Issue Type: New Feature
> Components: ML
> Reporter: Xiangrui Meng
> Assignee: Xiangrui Meng
>
> Add normal equation solver for ordinary least squares with not many features.
> The approach requires one pass to collect AtA and Atb, then solve the problem
> on driver. It works well when the problem is not very ill-conditioned and not
> having many columns. It also provides R-like summary statistics.
> We can hide this implementation under LinearRegression. It is triggered when
> there are no more than, e.g., 4096 features.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
