GitHub user jkbradley opened a pull request:
https://github.com/apache/spark/pull/12430
[SPARK-14671][ML] Pipeline setStages should handle subclasses of
PipelineStage
## What changes were proposed in this pull request?
Pipeline.setStages failed for some code examples which worked in 1.5 but
fail in 1.6. This tends to occur when using a mix of transformers from
ml.feature. It is because Java Arrays are non-covariant and the addition of
MLWritable to some transformers means the stages0/1 arrays above are not of
type Array[PipelineStage]. This PR modifies the following to accept subclasses
of PipelineStage:
* Pipeline.setStages()
* Params.w()
## How was this patch tested?
Unit test which fails to compile before this fix.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jkbradley/spark pipeline-setstages
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12430.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12430
----
commit e15a0a2bb5da059718b1b82538eba10928b9dd3b
Author: Joseph K. Bradley <[email protected]>
Date: 2016-04-14T23:52:05Z
Added unit test for Pipeline.setStages which failed beforehand but works
now, with fix for arrays being non-covariant
commit f2d65998fb8c7176141902bc58d370eb3e2245c8
Author: Joseph K. Bradley <[email protected]>
Date: 2016-04-15T00:13:34Z
removed old comment
commit 3401570a8aa737e04a214736ced2e9fe62ec7692
Author: Joseph K. Bradley <[email protected]>
Date: 2016-04-15T21:29:35Z
reverted some style fixes for easier backport
commit c7e4cb7a3c79bb5f6093ffaf1d4debac5cf23c3a
Author: Joseph K. Bradley <[email protected]>
Date: 2016-04-15T21:32:57Z
reverted more style fixes for easier backport
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
