[jira] [Comment Edited] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties
[ https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536781#comment-15536781 ] Seth Hendrickson edited comment on SPARK-17748 at 9/30/16 7:16 PM: --- I am working on this currently. The basic plan is to refactor WLS so that it has a pluggable solver for the normal equations. We can implement a new interface like {code:java} trait NormalEquationSolver { def solve( bBar: Double, bbBar: Double, abBar: DenseVector, aaBar: DenseVector, aBar: DenseVector): NormalEquationSolution } class CholeskySolver extends NormalEquationsSolver class QuasiNewtonSolver extends NormalEquationSolver {code} If others have thoughts on the design please comment, otherwise I will continue working on this and submit a PR reasonably soon. cc [~srowen] [~yanboliang] [~dbtsai] was (Author: sethah): I am working on this currently. The basic plan is to refactor WLS so that it has a pluggable solver for the normal equations. We can implement a new interface like {code:java} trait NormalEquationSolver { def solve( bBar: Double, bbBar: Double, abBar: DenseVector, aaBar: DenseVector, aBar: DenseVector): NormalEquationSolution } class CholeskySolver extends NormalEquationsSolver class QuasiNewtonSolver extends NormalEquationSolver {code} If others have thoughts on the design please comment, otherwise I will continue working on this and submit a PR reasonably soon. cc [~srowen] [~yanboliang] > One-pass algorithm for linear regression with L1 and elastic-net penalties > -- > > Key: SPARK-17748 > URL: https://issues.apache.org/jira/browse/SPARK-17748 > Project: Spark > Issue Type: Bug > Components: ML >Reporter: Seth Hendrickson > > Currently linear regression uses weighted least squares to solve the normal > equations locally on the driver when the dimensionality is small (<4096). > Weighted least squares uses a Cholesky decomposition to solve the problem > with L2 regularization (which has a closed-form solution). We can support > L1/elasticnet penalties by solving the equations locally using OWL-QN solver. > Also note that Cholesky does not handle singular covariance matrices, but > L-BFGS and OWL-QN are capable of providing reasonable solutions. This patch > can also add support for solving singular covariance matrices by also adding > L-BFGS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties
[ https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536781#comment-15536781 ] Seth Hendrickson edited comment on SPARK-17748 at 9/30/16 7:16 PM: --- I am working on this currently. The basic plan is to refactor WLS so that it has a pluggable solver for the normal equations. We can implement a new interface like {code:java} trait NormalEquationSolver { def solve( bBar: Double, bbBar: Double, abBar: DenseVector, aaBar: DenseVector, aBar: DenseVector): NormalEquationSolution } class CholeskySolver extends NormalEquationsSolver class QuasiNewtonSolver extends NormalEquationSolver {code} If others have thoughts on the design please comment, otherwise I will continue working on this and submit a PR reasonably soon. cc [~srowen] [~yanboliang] was (Author: sethah): I am working on this currently. The basic plan is to refactor WLS so that it has a pluggable solver for the normal equations. We can implement a new interface like {code:java} trait NormalEquationSolver { def solve( bBar: Double, bbBar: Double, abBar: DenseVector, aaBar: DenseVector, aBar: DenseVector): NormalEquationSolution } class CholeskySolver extends NormalEquationsSolver class QuasiNewtonSolver extends NormalEquationSolver {code} If others have thoughts on the design please comment, otherwise I will continue working on this and submit a PR reasonably soon. > One-pass algorithm for linear regression with L1 and elastic-net penalties > -- > > Key: SPARK-17748 > URL: https://issues.apache.org/jira/browse/SPARK-17748 > Project: Spark > Issue Type: Bug > Components: ML >Reporter: Seth Hendrickson > > Currently linear regression uses weighted least squares to solve the normal > equations locally on the driver when the dimensionality is small (<4096). > Weighted least squares uses a Cholesky decomposition to solve the problem > with L2 regularization (which has a closed-form solution). We can support > L1/elasticnet penalties by solving the equations locally using OWL-QN solver. > Also note that Cholesky does not handle singular covariance matrices, but > L-BFGS and OWL-QN are capable of providing reasonable solutions. This patch > can also add support for solving singular covariance matrices by also adding > L-BFGS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org