GitHub user mengxr opened a pull request:
https://github.com/apache/spark/pull/1849
[SPARK-2923][MLLIB] Implement some basic BLAS routines
Having some basic BLAS operations implemented in MLlib can help simplify
the current implementation and improve some performance.
Tested on my local machine:
~~~
bin/spark-submit --class
org.apache.spark.examples.mllib.BinaryClassification \
examples/target/scala-*/spark-examples-*.jar --algorithm LR --regType L2 \
--regParam 1.0 --numIterations 1000
~/share/data/rcv1.binary/rcv1_train.binary
~~~
1. before: ~1m
2. after: ~30s
CC: @jkbradley
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mengxr/spark ml-blas
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1849.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1849
----
commit e8c326ddb5697ca06e901a600b2287578214b161
Author: Xiangrui Meng <[email protected]>
Date: 2014-07-15T20:16:22Z
axpy
commit 07db0bb4d25bf26411310259f4bed376a0057d85
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-07T17:18:43Z
Merge branch 'master' into ml-blas
commit cbb8273e27ca1c25326b24a54ffb49bc1909885f
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-07T19:24:03Z
add daxpy test
commit 14e664595ce6adfa21a8085b8fd13a21b88a07d6
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T02:21:07Z
add ddot
commit 7f78186335a4e0d8ecee0319de5f7deda1a011c8
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T03:26:19Z
add zeros to Vectors; add dscal and dcopy to BLAS
commit d2d7d3c2fb6a6fad2848bce413e8f83e4a2281ce
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T03:28:21Z
update gradient and lbfgs
commit 735eb23fa3f4f9a364c4fb35139db03add7db45d
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T04:52:15Z
remove d from BLAS routines
commit 3b882b1d23456636fb76bd67e41008d8d808c043
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T05:03:35Z
use blas.scal in gradient
commit 48d01d20787cf98b892963ee0d3fbb4f6633d063
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T05:16:30Z
add tests for zeros and copy
commit 4cfaac46cf0f1a40919af6e7935f4c734ab6625c
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T05:18:07Z
add apache header
commit c2a38bcccc95b5fbbae4d022857746252e65006d
Author: Xiangrui Meng <[email protected]>
Date: 2014-08-08T05:23:25Z
enhance dot tests
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
