Yikun commented on a change in pull request #32867:
URL: https://github.com/apache/spark/pull/32867#discussion_r652488153
##########
File path: dev/sparktestsupport/modules.py
##########
@@ -16,13 +16,65 @@
#
from functools import total_ordering
+from importlib import import_module
+import inspect
import itertools
import os
+from pkgutil import iter_modules
import re
+import unittest
+
+from sparktestsupport import SPARK_HOME
+
all_modules = []
+def _contain_unittests_class(module_name):
+ """
+ Check if the module with specific module_name has classes are derived from
unittest.TestCase.
+
+ Such as:
+ pyspark.tests.test_appsubmit, it will return True, because there is
SparkSubmitTests which is
+ included under the module of pyspark.tests.test_appsubmit, inherits from
unittest.TestCase.
+ ``
+
+ :param module_name: the complete name of module to be checked.
+ :return: True if contains unittest classes otherwise False.
+ An ``ModuleNotFoundError`` will raise if the module is not found
+ """
+ _module = import_module(module_name)
Review comment:
@HyukjinKwon the module import based discoverd depends on pyspark and
serveral dependency of pyspark, so it's a bit hard to discover the modules.
Do you have any idea on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
