Xiao Li created SPARK-22965:
-------------------------------
Summary: Add deterministic parameter to registerJavaFunction
Key: SPARK-22965
URL: https://issues.apache.org/jira/browse/SPARK-22965
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 2.3.0
Reporter: Xiao Li
Assignee: Xiao Li
To register a JAVA UDF in PySpark, users are unable to specify the registered
UDF is not deterministic. The proposal is to add the extra parameter
deterministic at the end of the function registerJavaFunction
Below is an example.
{noformat}
>>> from pyspark.sql.types import DoubleType
>>> sqlContext.registerJavaFunction("javaRand",
... "test.org.apache.spark.sql.JavaRandUDF", DoubleType(),
deterministic=False)
>>> sqlContext.sql("SELECT javaRand(3)").collect() # doctest: +SKIP
[Row(UDF:javaRand(3)=3.12345)]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
