Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/20162#discussion_r159785534
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala ---
@@ -601,15 +601,32 @@ class UDFRegistration private[sql] (functionRegistry:
FunctionRegistry) extends
//////////////////////////////////////////////////////////////////////////////////////////////
/**
- * Register a Java UDF class using reflection, for use from pyspark
+ * Register a deterministic Java UDF class using reflection, for use
from pyspark
*
* @param name udf name
* @param className fully qualified class name of udf
* @param returnDataType return type of udf. If it is null, spark would
try to infer
* via reflection.
*/
private[sql] def registerJava(name: String, className: String,
returnDataType: DataType): Unit = {
--- End diff --
Although this is private and only used for PySpark, the JAVA users still
could call it. For not breaking the existing behavior, I keep the original one
unchanged. Also added a test case.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
