GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/20162
[SPARK-22965] [PySpark] [SQL] Add deterministic parameter to
registerJavaFunction
## What changes were proposed in this pull request?
To register a JAVA UDF in PySpark, users are unable to specify the
registered UDF is not deterministic. The proposal is to add the extra parameter
deterministic at the end of the function registerJavaFunction
Below is an example.
```
>>> from pyspark.sql.types import DoubleType
>>> sqlContext.registerJavaFunction("javaRand",
... "test.org.apache.spark.sql.JavaRandUDF", DoubleType(),
deterministic=False)
>>> sqlContext.sql("SELECT javaRand(3)").collect()
[Row(UDF:javaRand(3)=3.12345)]
```
## How was this patch tested?
Added test cases
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark registerJava
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20162.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20162
----
commit cf0076b9257d65605ed3153f0b59cd89cdb145fc
Author: gatorsmile <gatorsmile@...>
Date: 2018-01-04T23:57:51Z
fix
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
