This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b7e70feef54a [SPARK-50233][PYTHON][CONNECT][TESTS] Skip PipelineTests
if torch is not installed
b7e70feef54a is described below
commit b7e70feef54ae0d1de4e012262ac055356a9d9f4
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Tue Nov 5 11:23:00 2024 -0800
[SPARK-50233][PYTHON][CONNECT][TESTS] Skip PipelineTests if torch is not
installed
### What changes were proposed in this pull request?
This PR proposes to skip PipelineTests if torch is not installed
### Why are the changes needed?
If tourch isn't available, we should run the tests `PipelineTests `. That's
an optional dependency.
To fix Python 3.13 build
https://github.com/apache/spark/actions/runs/11671728559
### Does this PR introduce _any_ user-facing change?
No, test-only.
### How was this patch tested?
Manually tested.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #48764 from HyukjinKwon/torch-req-test.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/ml/tests/connect/test_legacy_mode_pipeline.py | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/ml/tests/connect/test_legacy_mode_pipeline.py
b/python/pyspark/ml/tests/connect/test_legacy_mode_pipeline.py
index 692144148af0..9165034718d7 100644
--- a/python/pyspark/ml/tests/connect/test_legacy_mode_pipeline.py
+++ b/python/pyspark/ml/tests/connect/test_legacy_mode_pipeline.py
@@ -24,6 +24,10 @@ import numpy as np
from pyspark.util import is_remote_only
from pyspark.sql import SparkSession
from pyspark.testing.connectutils import should_test_connect,
connect_requirement_message
+from pyspark.ml.tests.connect.test_connect_classification import (
+ have_torch,
+ torch_requirement_message,
+)
if should_test_connect:
from pyspark.ml.connect.feature import StandardScaler
@@ -169,8 +173,10 @@ class PipelineTestsMixin:
@unittest.skipIf(
- not should_test_connect or is_remote_only(),
- connect_requirement_message or "pyspark-connect cannot test classic Spark",
+ not should_test_connect or not have_torch or is_remote_only(),
+ connect_requirement_message
+ or torch_requirement_message
+ or "pyspark-connect cannot test classic Spark",
)
class PipelineTests(PipelineTestsMixin, unittest.TestCase):
def setUp(self) -> None:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
