GitHub user coderxiang opened a pull request:
https://github.com/apache/spark/pull/458
[SPARK-1543][MLlib] Add ADMM for solving Lasso (and elastic net) problem
This PR introduces the Alternating Direction Method of Multipliers (ADMM)
for solving Lasso (elastic net, in fact) in mllib.
ADMM is capable of solving a class of composite minimization problems in a
distributed way. Specifically for Lasso (if only L1-regularization) or
elastic-net (both L1- and L2- regularization), it requires solving independent
systems of linear equations on each partition and a soft-threholding operation
on the driver. Unlike SGD, it is a deterministic algorithm (except for the
random partition). Details can be found in the [S. Boyd's
paper](http://www.stanford.edu/~boyd/papers/admm_distr_stats.html).
The linear algebra operations mainly rely on the Breeze library,
particularly, it applies `breeze.linalg.cholesky` to perform cholesky
decomposition on each partition to solve the linear system.
I tried to follow the organization of existing Lasso implementation.
However, as ADMM is also a good fit for similar optimization problems, e.g.,
(sparse) logistic regression, it may worth to re-organize and put ADMM into a
separate section.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/coderxiang/spark admmlasso
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/458.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #458
----
commit 6c06c710dd098a3280fe5a9a5d5f490f6dcd3b9c
Author: lebesgue <[email protected]>
Date: 2014-04-20T21:37:28Z
add admm for lasso
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
