Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20909#discussion_r180134340
--- Diff: python/pyspark/sql/udf.py ---
@@ -385,8 +385,23 @@ def registerJavaUDAF(self, name, javaClassName):
def _test():
import doctest
+ import os
+ import os.path
+ import glob
from pyspark.sql import SparkSession
import pyspark.sql.udf
+
+ SPARK_HOME = os.environ["SPARK_HOME"]
+ filename_pattern = "sql/core/target/scala-*/test-classes/" + \
+ "test/org/apache/spark/sql/JavaStringLength.class"
+ if not glob.glob(os.path.join(SPARK_HOME, filename_pattern)):
+ # if test udf files are not compiled, then skip the below doctests
+ # TODO: Need to communicate with outside world that these tests
+ # have been skipped.
+ m = pyspark.sql.udf
+
m.__dict__["UDFRegistration"].__dict__["registerJavaFunction"].__doc__ = ""
+ m.__dict__["UDFRegistration"].__dict__["registerJavaUDAF"].__doc__
= ""
--- End diff --
ah.. hmm... yea this one was the last resort I was thinking ... let me
investigate other possible ways for some more days.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
