This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 63529bf97ad0 [SPARK-47543][CONNECT][PYTHON][TESTS][FOLLOW-UP] Skip the
test if pandas and PyArrow are unavailable
63529bf97ad0 is described below
commit 63529bf97ad0bb6d46f4d88bc1425cad76add1a1
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Fri Mar 29 18:31:12 2024 +0900
[SPARK-47543][CONNECT][PYTHON][TESTS][FOLLOW-UP] Skip the test if pandas
and PyArrow are unavailable
### What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/45699 that
adds pandas/arrow check (and skips the test if not installed).
Currently, scheduled PyPy build is broken
https://github.com/apache/spark/actions/runs/8469356110/job/23208888581.
### Why are the changes needed?
To make the PySpark tests runnable without optional dependencies.
To fix up the scheduled build for PyPy.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Manually.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45772 from HyukjinKwon/SPARK-47543.
Lead-authored-by: Hyukjin Kwon <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/sql/tests/test_creation.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/python/pyspark/sql/tests/test_creation.py
b/python/pyspark/sql/tests/test_creation.py
index c9d1edb6ae6c..d35ef9323d79 100644
--- a/python/pyspark/sql/tests/test_creation.py
+++ b/python/pyspark/sql/tests/test_creation.py
@@ -20,6 +20,7 @@ from decimal import Decimal
import os
import time
import unittest
+from typing import cast
from pyspark.sql import Row
import pyspark.sql.functions as F
@@ -35,7 +36,9 @@ from pyspark.errors import (
from pyspark.testing.sqlutils import (
ReusedSQLTestCase,
have_pandas,
+ have_pyarrow,
pandas_requirement_message,
+ pyarrow_requirement_message,
)
@@ -157,6 +160,10 @@ class DataFrameCreationTestsMixin:
message_parameters={},
)
+ @unittest.skipIf(
+ not have_pandas or not have_pyarrow,
+ cast(str, pandas_requirement_message or pyarrow_requirement_message),
+ )
def test_schema_inference_from_pandas_with_dict(self):
# SPARK-47543: test for verifying if inferring `dict` as `MapType`
work properly.
import pandas as pd
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
