This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 920e46f02785 [SPARK-52413][PYTHON][TEST] `test_install_spark` switch
to Spark 3.5.6
920e46f02785 is described below
commit 920e46f0278506675a1ee68d4b011739b0cdbf77
Author: Cheng Pan <[email protected]>
AuthorDate: Sat Jun 7 13:11:43 2025 -0700
[SPARK-52413][PYTHON][TEST] `test_install_spark` switch to Spark 3.5.6
### What changes were proposed in this pull request?
Upgrade to Spark 3.5.6 for `test_install_spark` because Spark 3.4.4 was
removed from https://dlcdn.apache.org/spark/
The [Apache Distribution Directory](https://dlcdn.apache.org/) only
contains current software releases from the ASF projects. Older non-recommended
releases can be found on [archive site](https://archive.apache.org/dist/), the
latter is slow and fragile.
### Why are the changes needed?
Improve CI stability, to mitigate failures like
https://github.com/apache/spark/actions/runs/15485108225/job/43598093680
```
Trying to download Spark spark-3.4.4 from [https://dlcdn.apache.org/,
https://archive.apache.org/dist, https://dist.apache.org/repos/dist/release]
Downloading spark-3.4.4 for Hadoop hadoop3 from:
- https://dlcdn.apache.org//spark/spark-3.4.4/spark-3.4.4-bin-hadoop3.tgz
Failed to download spark-3.4.4 for Hadoop hadoop3 from
https://dlcdn.apache.org//spark/spark-3.4.4/spark-3.4.4-bin-hadoop3.tgz:
Downloading spark-3.4.4 for Hadoop hadoop3 from:
-
https://archive.apache.org/dist/spark/spark-3.4.4/spark-3.4.4-bin-hadoop3.tgz
Failed to download spark-3.4.4 for Hadoop hadoop3 from
https://archive.apache.org/dist/spark/spark-3.4.4/spark-3.4.4-bin-hadoop3.tgz:
Downloading spark-3.4.4 for Hadoop hadoop3 from:
-
https://dist.apache.org/repos/dist/release/spark/spark-3.4.4/spark-3.4.4-bin-hadoop3.tgz
Failed to download spark-3.4.4 for Hadoop hadoop3 from
https://dist.apache.org/repos/dist/release/spark/spark-3.4.4/spark-3.4.4-bin-hadoop3.tgz:
ERROR (270.884s)
test_package_name
(pyspark.tests.test_install_spark.SparkInstallationTestCase.test_package_name)
... ok (0.000s)
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GHA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #51103 from pan3793/SPARK-52413.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
python/pyspark/tests/test_install_spark.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/tests/test_install_spark.py
b/python/pyspark/tests/test_install_spark.py
index d46d55e02284..effbab6a9010 100644
--- a/python/pyspark/tests/test_install_spark.py
+++ b/python/pyspark/tests/test_install_spark.py
@@ -31,8 +31,8 @@ from pyspark.install import (
class SparkInstallationTestCase(unittest.TestCase):
def test_install_spark(self):
# Test only one case. Testing this is expensive because it needs to
download
- # the Spark distribution.
- spark_version, hadoop_version, hive_version =
checked_versions("3.4.4", "3", "2.3")
+ # the Spark distribution, ensure it is available at
https://dlcdn.apache.org/spark/
+ spark_version, hadoop_version, hive_version =
checked_versions("3.5.6", "3", "2.3")
with tempfile.TemporaryDirectory(prefix="test_install_spark") as
tmp_dir:
install_spark(
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
