This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c53dac05058c [SPARK-50211][PYTHON][CONNECT][TESTS] Skip FeatureTests
if torch is not installed
c53dac05058c is described below
commit c53dac05058c48ae1edad7912e8cc82533839ca0
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Sun Nov 3 16:06:28 2024 -0800
[SPARK-50211][PYTHON][CONNECT][TESTS] Skip FeatureTests if torch is not
installed
### What changes were proposed in this pull request?
This PR proposes to skip FeatureTests if torch is not installed
### Why are the changes needed?
If tourch isn't available, we should run the tests `FeatureTests `. That's
an optional dependency.
To fix Python 3.13 build
https://github.com/apache/spark/actions/runs/11654255828/job/32447601706
### Does this PR introduce _any_ user-facing change?
No, test-only.
### How was this patch tested?
Manually tested.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #48743 from HyukjinKwon/SPARK-50211.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
python/pyspark/ml/tests/connect/test_legacy_mode_feature.py | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/python/pyspark/ml/tests/connect/test_legacy_mode_feature.py
b/python/pyspark/ml/tests/connect/test_legacy_mode_feature.py
index 4915d4706b87..d90e4a4315d5 100644
--- a/python/pyspark/ml/tests/connect/test_legacy_mode_feature.py
+++ b/python/pyspark/ml/tests/connect/test_legacy_mode_feature.py
@@ -26,7 +26,10 @@ import numpy as np
from pyspark.util import is_remote_only
from pyspark.sql import SparkSession
from pyspark.testing.connectutils import should_test_connect,
connect_requirement_message
-
+from pyspark.ml.tests.connect.test_connect_classification import (
+ have_torch,
+ torch_requirement_message,
+)
if should_test_connect:
from pyspark.ml.connect.feature import (
@@ -196,8 +199,10 @@ class FeatureTestsMixin:
@unittest.skipIf(
- not should_test_connect or is_remote_only(),
- connect_requirement_message or "pyspark-connect cannot test classic Spark",
+ not should_test_connect or not have_torch or is_remote_only(),
+ connect_requirement_message
+ or torch_requirement_message
+ or "pyspark-connect cannot test classic Spark",
)
class FeatureTests(FeatureTestsMixin, unittest.TestCase):
def setUp(self) -> None:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
