GitHub user jkbradley opened a pull request:
https://github.com/apache/spark/pull/17165
[DO NOT MERGE][TESTING] Vince shieh spark 17498
Temp PR to reproduce Jenkins compilation error
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jkbradley/spark VinceShieh-spark-17498
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17165.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17165
----
commit b970728f48f22f0c2789a941c1fe1ac6b94a3b49
Author: VinceShieh <[email protected]>
Date: 2017-02-10T05:50:30Z
[SPARK-17498][ML] StringIndexer handles unseen labels
This PR is an enhancement to ML StringIndexer. Before this PR, String
Indexer only supports "skip"/"error" options
to deal with unseen records. But sometimes those unseen records might still
be useful in certain use cases, so user
would like to keep the unseen labels. This PR enables StringIndexer to
support keeping unseen labels as indices
[numLabels].
'''Before
StringIndexer().setHandleInvalid("skip")
StringIndexer().setHandleInvalid("error")
'''After
support the third option "keep"
StringIndexer().setHandleInvalid("keep")
Signed-off-by: VinceShieh <[email protected]>
commit 5d4b07f517cdf52e5b3b0b786e1dba1993659b2e
Author: VinceShieh <[email protected]>
Date: 2017-02-10T07:02:44Z
fix compilation issue
Signed-off-by: VinceShieh <[email protected]>
commit 0eb7f0784a71cb695f4d936255abbe8ad30bd95d
Author: VinceShieh <[email protected]>
Date: 2017-02-10T08:16:57Z
code refactoring
Signed-off-by: VinceShieh <[email protected]>
commit 9a4174579aa811c99a81967dd829e506c0096ccd
Author: VinceShieh <[email protected]>
Date: 2017-02-10T09:08:30Z
add exclusion rules in mima to pass binary compability check
Signed-off-by: VinceShieh <[email protected]>
commit 1736057d055ad4a01dac3e9e79950bfcd9b91e1e
Author: VinceShieh <[email protected]>
Date: 2017-02-10T09:33:31Z
update document
Signed-off-by: VinceShieh <[email protected]>
commit ebe9ddb0dc3dd597d435f8a641fce790b4033a64
Author: VinceShieh <[email protected]>
Date: 2017-02-10T09:37:43Z
Revert "add exclusion rules in mima to pass binary compability check"
This reverts commit 9a4174579aa811c99a81967dd829e506c0096ccd.
commit 27c1b10f25db851cd1e670bd6a0d6e6f59c2ce1e
Author: VinceShieh <[email protected]>
Date: 2017-02-10T09:42:56Z
Mima changes to pass binary compatibility check
Signed-off-by: VinceShieh <[email protected]>
commit 9bcaffc19e7a11d31aa6bb9ebbcd96367fc1cd38
Author: VinceShieh <[email protected]>
Date: 2017-03-01T02:09:36Z
update
Signed-off-by: VinceShieh <[email protected]>
commit 4dc10e6390b30fa8df9789479430e0a3f7c65c39
Author: VinceShieh <[email protected]>
Date: 2017-03-01T02:16:29Z
update target version
Signed-off-by: VinceShieh <[email protected]>
commit fa24e433c3f9fe6f76fe0a55df4551881f194d7b
Author: VinceShieh <[email protected]>
Date: 2017-03-01T02:26:43Z
fix compilation on
val (filteredDataset, keepInvalid) = getHandleInvalid match {
case ..
}
Signed-off-by: VinceShieh <[email protected]>
commit 67f02d565685dc4b9be2709783539f7af1ea1bb5
Author: Joseph K. Bradley <[email protected]>
Date: 2017-03-04T20:08:21Z
remove scala existentials import
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
