This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b6ec1ce41d8 [SPARK-41346][CONNECT][TESTS][FOLLOWUP] Fix
`test_connect_function` to import `PandasOnSparkTestCase` properly
b6ec1ce41d8 is described below
commit b6ec1ce41d8c35a3e9d3ab74c026a7419ded2eae
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Mon Dec 5 22:58:23 2022 -0800
[SPARK-41346][CONNECT][TESTS][FOLLOWUP] Fix `test_connect_function` to
import `PandasOnSparkTestCase` properly
### What changes were proposed in this pull request?
This PR aims to fix `test_connect_function` to import
`PandasOnSparkTestCase` properly. If we handle `import` properly, the test
cases are ignored properly because `should_test_connect` assumes `have_pandas`
https://github.com/apache/spark/blob/97976a5cc915597fd2606602d18c52c075a03bf6/python/pyspark/testing/connectutils.py#L49
### Why are the changes needed?
SPARK-41346 imported `PandasOnSparkTestCase` outside of `if have_pandas:`.
https://github.com/apache/spark/blob/97976a5cc915597fd2606602d18c52c075a03bf6/python/pyspark/sql/tests/connect/test_connect_function.py#L25-L29
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
**BEFORE**
```
$ python/run-tests --testnames
pyspark.sql.tests.connect.test_connect_function
...
ModuleNotFoundError: No module named 'pandas'
```
**AFTER**
```
$ python/run-tests --testnames
pyspark.sql.tests.connect.test_connect_function
...
Skipped tests in pyspark.sql.tests.connect.test_connect_function with
python3.9:
test_aggregation_functions
(pyspark.sql.tests.connect.test_connect_function.SparkConnectFunctionTests) ...
skip (0.004s)
test_math_functions
(pyspark.sql.tests.connect.test_connect_function.SparkConnectFunctionTests) ...
skip (0.004s)
test_normal_functions
(pyspark.sql.tests.connect.test_connect_function.SparkConnectFunctionTests) ...
skip (0.002s)
test_sort_with_nulls_order
(pyspark.sql.tests.connect.test_connect_function.SparkConnectFunctionTests) ...
skip (0.001s)
test_sorting_functions_with_column
(pyspark.sql.tests.connect.test_connect_function.SparkConnectFunctionTests) ...
skip (0.001s)
```
Closes #38929 from dongjoon-hyun/SPARK-41346.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
python/pyspark/sql/tests/connect/test_connect_function.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/python/pyspark/sql/tests/connect/test_connect_function.py
b/python/pyspark/sql/tests/connect/test_connect_function.py
index 83d27235bdb..6d06421d084 100644
--- a/python/pyspark/sql/tests/connect/test_connect_function.py
+++ b/python/pyspark/sql/tests/connect/test_connect_function.py
@@ -24,9 +24,11 @@ from pyspark.sql import SparkSession
if have_pandas:
from pyspark.sql.connect.session import SparkSession as RemoteSparkSession
+ from pyspark.testing.pandasutils import PandasOnSparkTestCase
+else:
+ from pyspark.testing.sqlutils import ReusedSQLTestCase as
PandasOnSparkTestCase
from pyspark.sql.dataframe import DataFrame
from pyspark.testing.connectutils import should_test_connect,
connect_requirement_message
-from pyspark.testing.pandasutils import PandasOnSparkTestCase
from pyspark.testing.utils import ReusedPySparkTestCase
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
