This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new b4b6d3008443 [SPARK-53540][INFRA] Remove `pypy3` from `spark-rm` image
b4b6d3008443 is described below
commit b4b6d30084430dac9b3fc9f5fbecd15275dc1717
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Sep 9 16:21:35 2025 -0700
[SPARK-53540][INFRA] Remove `pypy3` from `spark-rm` image
This PR aims to remove `pypy3` from `spark-rm` image.
- This will reduce the `spark-rm` build time during the daily `dry-run` CI
and release process.
- https://github.com/apache/spark/actions/workflows/release.yml
- `pypy3` was added at 4.0.0 via the following and we didn't have `pypy3`
before.
- https://github.com/apache/spark/pull/46534
No. This is an update for Apache Spark release tool.
Manual review because this is not used technically.
After merging this PR, we can check the daily release dry-run CI too.
- https://github.com/apache/spark/actions/workflows/release.yml
No.
Closes #52291 from dongjoon-hyun/SPARK-53540.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 19912f85b8a5935a27fdf67d5550ed60e0c53068)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/create-release/spark-rm/Dockerfile | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/dev/create-release/spark-rm/Dockerfile
b/dev/create-release/spark-rm/Dockerfile
index a537f81a4c06..5a067fee3599 100644
--- a/dev/create-release/spark-rm/Dockerfile
+++ b/dev/create-release/spark-rm/Dockerfile
@@ -19,7 +19,6 @@
# Extends the base image with:
# - Java 17
# - Python 3.9/3.10 with required packages
-# - PyPy 3.10 for testing
FROM spark-rm-base:latest
@@ -80,14 +79,6 @@ RUN python3.9 -m pip install 'sphinx==4.5.0' mkdocs
'pydata_sphinx_theme>=0.13'
'sphinxcontrib-htmlhelp==2.0.1' 'sphinxcontrib-qthelp==1.0.3' \
'sphinxcontrib-serializinghtml==1.1.5'
-# Install PyPy 3.10 for testing
-RUN mkdir -p /usr/local/pypy/pypy3.10 && \
- curl -sqL
https://downloads.python.org/pypy/pypy3.10-v7.3.17-linux64.tar.bz2 | tar xjf -
-C /usr/local/pypy/pypy3.10 --strip-components=1 && \
- ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3.10 && \
- ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3 && \
- curl -sS https://bootstrap.pypa.io/get-pip.py | pypy3 && \
- pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.2.3' scipy coverage
matplotlib lxml
-
# Set Python 3.9 as the default (branch-4.0 uses 3.9 for docs)
RUN ln -sf "$(which python3.9)" "/usr/local/bin/python" && \
ln -sf "$(which python3.9)" "/usr/local/bin/python3"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
