[jira] [Commented] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties
[ https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638392#comment-15638392 ] Apache Spark commented on SPARK-17748: -- User 'jkbradley' has created a pull request for this issue: https://github.com/apache/spark/pull/15779 > One-pass algorithm for linear regression with L1 and elastic-net penalties > -- > > Key: SPARK-17748 > URL: https://issues.apache.org/jira/browse/SPARK-17748 > Project: Spark > Issue Type: New Feature > Components: ML >Reporter: Seth Hendrickson >Assignee: Seth Hendrickson > Fix For: 2.1.0 > > > Currently linear regression uses weighted least squares to solve the normal > equations locally on the driver when the dimensionality is small (<4096). > Weighted least squares uses a Cholesky decomposition to solve the problem > with L2 regularization (which has a closed-form solution). We can support > L1/elasticnet penalties by solving the equations locally using OWL-QN solver. > Also note that Cholesky does not handle singular covariance matrices, but > L-BFGS and OWL-QN are capable of providing reasonable solutions. This patch > can also add support for solving singular covariance matrices by also adding > L-BFGS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties
[ https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15605719#comment-15605719 ] Apache Spark commented on SPARK-17748: -- User 'yanboliang' has created a pull request for this issue: https://github.com/apache/spark/pull/15625 > One-pass algorithm for linear regression with L1 and elastic-net penalties > -- > > Key: SPARK-17748 > URL: https://issues.apache.org/jira/browse/SPARK-17748 > Project: Spark > Issue Type: New Feature > Components: ML >Reporter: Seth Hendrickson >Assignee: Seth Hendrickson > Fix For: 2.1.0 > > > Currently linear regression uses weighted least squares to solve the normal > equations locally on the driver when the dimensionality is small (<4096). > Weighted least squares uses a Cholesky decomposition to solve the problem > with L2 regularization (which has a closed-form solution). We can support > L1/elasticnet penalties by solving the equations locally using OWL-QN solver. > Also note that Cholesky does not handle singular covariance matrices, but > L-BFGS and OWL-QN are capable of providing reasonable solutions. This patch > can also add support for solving singular covariance matrices by also adding > L-BFGS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties
[ https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604805#comment-15604805 ] Apache Spark commented on SPARK-17748: -- User 'yanboliang' has created a pull request for this issue: https://github.com/apache/spark/pull/15621 > One-pass algorithm for linear regression with L1 and elastic-net penalties > -- > > Key: SPARK-17748 > URL: https://issues.apache.org/jira/browse/SPARK-17748 > Project: Spark > Issue Type: New Feature > Components: ML >Reporter: Seth Hendrickson >Assignee: Seth Hendrickson > Fix For: 2.1.0 > > > Currently linear regression uses weighted least squares to solve the normal > equations locally on the driver when the dimensionality is small (<4096). > Weighted least squares uses a Cholesky decomposition to solve the problem > with L2 regularization (which has a closed-form solution). We can support > L1/elasticnet penalties by solving the equations locally using OWL-QN solver. > Also note that Cholesky does not handle singular covariance matrices, but > L-BFGS and OWL-QN are capable of providing reasonable solutions. This patch > can also add support for solving singular covariance matrices by also adding > L-BFGS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties
[ https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576770#comment-15576770 ] Apache Spark commented on SPARK-17748: -- User 'sethah' has created a pull request for this issue: https://github.com/apache/spark/pull/15394 > One-pass algorithm for linear regression with L1 and elastic-net penalties > -- > > Key: SPARK-17748 > URL: https://issues.apache.org/jira/browse/SPARK-17748 > Project: Spark > Issue Type: New Feature > Components: ML >Reporter: Seth Hendrickson >Assignee: Seth Hendrickson > > Currently linear regression uses weighted least squares to solve the normal > equations locally on the driver when the dimensionality is small (<4096). > Weighted least squares uses a Cholesky decomposition to solve the problem > with L2 regularization (which has a closed-form solution). We can support > L1/elasticnet penalties by solving the equations locally using OWL-QN solver. > Also note that Cholesky does not handle singular covariance matrices, but > L-BFGS and OWL-QN are capable of providing reasonable solutions. This patch > can also add support for solving singular covariance matrices by also adding > L-BFGS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties
[
https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536781#comment-15536781
]
Seth Hendrickson commented on SPARK-17748:
--
I am working on this currently. The basic plan is to refactor WLS so that it
has a pluggable solver for the normal equations. We can implement a new
interface like
{code:java}
trait NormalEquationSolver {
def solve(
bBar: Double,
bbBar: Double,
abBar: DenseVector,
aaBar: DenseVector,
aBar: DenseVector): NormalEquationSolution
}
class CholeskySolver extends NormalEquationsSolver
class QuasiNewtonSolver extends NormalEquationSolver
{code}
If others have thoughts on the design please comment, otherwise I will continue
working on this and submit a PR reasonably soon.
> One-pass algorithm for linear regression with L1 and elastic-net penalties
> --
>
> Key: SPARK-17748
> URL: https://issues.apache.org/jira/browse/SPARK-17748
> Project: Spark
> Issue Type: Bug
> Components: ML
>Reporter: Seth Hendrickson
>
> Currently linear regression uses weighted least squares to solve the normal
> equations locally on the driver when the dimensionality is small (<4096).
> Weighted least squares uses a Cholesky decomposition to solve the problem
> with L2 regularization (which has a closed-form solution). We can support
> L1/elasticnet penalties by solving the equations locally using OWL-QN solver.
> Also note that Cholesky does not handle singular covariance matrices, but
> L-BFGS and OWL-QN are capable of providing reasonable solutions. This patch
> can also add support for solving singular covariance matrices by also adding
> L-BFGS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
