GitHub user mengxr opened a pull request:
https://github.com/apache/spark/pull/364
[SPARK-1225, 1241] [MLLIB] Add AreaUnderCurve and
BinaryClassificationEvaluator
This PR implements a generic version of `AreaUnderCurve` using the
`RDD.sliding` implementation from https://github.com/apache/spark/pull/136 . It
also contains refactoring of https://github.com/apache/spark/pull/160 for
binary classification evaluation.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mengxr/spark auc
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/364.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #364
----
commit d2a600d5c0ab8a068cb23bdd422645d8b1a39f0b
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-13T08:47:45Z
add sliding to rdd
commit 5ee6001471b1897400fef1e35b5e10fbfb47395f
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-13T18:49:04Z
add TODO
commit 65461b21b012c8688d2747a039a721fb859bf9d3
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-13T20:14:22Z
Merge branch 'sliding' into auc
commit c1c6c2228a446ed42bf4382d4703309865f6dc54
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-13T20:47:11Z
add AreaUnderCurve
commit 284d991cf8c79a1ef7db79a9caa35a238e02338a
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-15T17:12:41Z
change SlidedRDD to SlidingRDD
commit 9916202e0c6bc9d183bc35f3f16302bb7fbbb644
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-15T17:46:35Z
change RDD.sliding return type to RDD[Seq[T]]
commit db6cb30da9ef7ce5ca473f32e709aedb2eeabc34
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-15T17:59:13Z
remove unnecessary toSeq
commit cab9a52349a7ffcefeae7660836a6ea1b77d910f
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-15T18:06:32Z
use last for the last element
commit a9b250a22e61192fd7c90b936b5eb798d1a5039e
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-22T00:52:44Z
move sliding to mllib
commit a92086513c976479b1b68255967a72bd4af8f5c2
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-31T21:26:44Z
Merge branch 'sliding' into auc
commit 221ebced1b36b0b625ce1bc19316f310a7e9f44c
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-31T22:03:08Z
add a new test to sliding
commit aa7e278d589fb342dd505c23b35a789eb1f7ed55
Author: Xiangrui Meng <[email protected]>
Date: 2014-03-31T22:30:25Z
add initial version of binary classification evaluator
commit dda82d5253f448b3e3f37ba712d420fe942efd26
Author: Xiangrui Meng <[email protected]>
Date: 2014-04-08T22:51:51Z
add confusion matrix
commit 8f78958cf366ae2bdecbf987bfa6f23d29c36c71
Author: Xiangrui Meng <[email protected]>
Date: 2014-04-08T23:29:53Z
add PredictionAndResponse
commit 3d71525d05ef3b5619c9af8d436ec585d648c1c9
Author: Xiangrui Meng <[email protected]>
Date: 2014-04-09T01:12:39Z
move binary evalution classes to evaluation.binary
commit ca31da590e25a8b18e347534a07b5e8392e1036e
Author: Xiangrui Meng <[email protected]>
Date: 2014-04-09T01:13:20Z
remove PredictionAndResponse
commit 9dc35182725c8dca5293cee7ab7dccca9a258c06
Author: Xiangrui Meng <[email protected]>
Date: 2014-04-09T02:16:52Z
add tests for BinaryClassificationEvaluator
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
