GitHub user ajaysaini725 opened a pull request:
https://github.com/apache/spark/pull/18746
Implemented UnaryTransformer in Python
## What changes were proposed in this pull request?
Implemented UnaryTransformer in Python
(Please fill in changes proposed in this fix)
## How was this patch tested?
UnaryTransformer is an abstract class with only two pre-implemented
functions provided. It wasn't clear how to test this without specifically
instantiating a UnaryTransformer so no unit tests were provided. In Scala,
UnaryTransformer is not tested so as long as the Python implementation follows
the method used in Scala, it should be fine. It would be good to include
examples in the docs of specific instances of UnaryTransformer. When combined
with the change in: https://github.com/apache/spark/pull/18742, this patch
makes it possible to implement custom Python-only UnaryTransformers and persist
them in Python (i.e. without any Java dependence).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ajaysaini725/spark AddPythonUnaryTransformer
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18746.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18746
----
commit 10ab7bc336c00fd6e010cd4301d03eed31b0c95c
Author: Ajay Saini <[email protected]>
Date: 2017-07-10T21:06:06Z
Started
commit 4fe94202819024ad82661004f59922f88cc3b7c8
Author: Ajay Saini <[email protected]>
Date: 2017-07-10T21:06:06Z
Started
commit 7d25c70d4882daea0a7d6dcc2318c06682c00eca
Author: Ajay Saini <[email protected]>
Date: 2017-07-27T05:01:39Z
Implemented Unary Transforme in Python.
commit 960de95330f31c46bdfde8f3f7d634d0edac18d4
Author: Ajay Saini <[email protected]>
Date: 2017-07-27T05:02:50Z
Fixed merge conflict.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
