GitHub user BryanCutler opened a pull request:
https://github.com/apache/spark/pull/18982
[SPARK-21685][PYTHON][ML] PySpark Params isSet state should not change
after transform
## What changes were proposed in this pull request?
Currently when a PySpark Model is transformed, default params that have not
been explicitly set are then set on the Java side on the call to
`wrapper._transfer_values_to_java`. This incorrectly changes the state of the
Param as it should still be marked as a default value only.
## How was this patch tested?
Added a new test to verify that when transferring Params to Java, default
params have their state preserved.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/BryanCutler/spark
pyspark-ml-param-to-java-defaults-SPARK-21685
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18982.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18982
----
commit bbe3ef70d2acb3647171621b02b4f73c001d5caf
Author: Bryan Cutler <[email protected]>
Date: 2017-08-17T23:59:43Z
added regression test for preserving Param set state when transferring to
Java
commit 12030569df3aa92a227d6e7d06ceeb5121d853f9
Author: Bryan Cutler <[email protected]>
Date: 2017-08-17T23:59:54Z
changed _transfer_params_to_java to not set param in Java unless explicitly
set
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
