This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 68fd17d09e83 [SPARK-49861][INFRA] Add `Python 3.13` to Infra docker
image
68fd17d09e83 is described below
commit 68fd17d09e83dc4a53c5cf1bf42c346e481098ca
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Oct 3 07:30:05 2024 -0700
[SPARK-49861][INFRA] Add `Python 3.13` to Infra docker image
### What changes were proposed in this pull request?
This PR aims to add `Python 3.13` to Infra docker image.
Note that SPARK-49862 `Add BASIC_PIP_PKGS and CONNECT_PIP_PKGS to Python
3.13` will track the Python package readiness.
### Why are the changes needed?
`Python 3.13` release is scheduled on next Monday, 2024-10-07.
- https://peps.python.org/pep-0719/
This is a part of `Python 3.13 Support` preparation for Apache Spark 4.0.0
on 2025 February.
### Does this PR introduce _any_ user-facing change?
No, this is an infra only change.
### How was this patch tested?
Pass the CIs and verify manually like the following.
```
$ docker run -it --rm
ghcr.io/dongjoon-hyun/apache-spark-ci-image:master-11152626644 python3.13
--version
Python 3.13.0rc3
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #48330 from dongjoon-hyun/SPARK-49861.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/infra/Dockerfile | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index a40e43bb659f..4125002cab0b 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -24,7 +24,7 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra
Image"
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""
-ENV FULL_REFRESH_DATE 20240903
+ENV FULL_REFRESH_DATE 20241002
ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
@@ -142,6 +142,16 @@ RUN python3.12 -m pip install $BASIC_PIP_PKGS
$CONNECT_PIP_PKGS lxml && \
python3.12 -m pip install torcheval && \
python3.12 -m pip cache purge
+# Install Python 3.13 at the last stage to avoid breaking the existing Python
installations
+RUN apt-get update && apt-get install -y \
+ python3.13 \
+ && rm -rf /var/lib/apt/lists/*
+RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.13
+# TODO(SPARK-49862) Add BASIC_PIP_PKGS and CONNECT_PIP_PKGS to Python 3.13
image when it supports Python 3.13
+RUN python3.13 -m pip install --ignore-installed blinker>=1.6.2 # mlflow needs
this
+RUN python3.13 -m pip install lxml && \
+ python3.13 -m pip cache purge
+
# Remove unused installation packages to free up disk space
RUN apt-get remove --purge -y 'gfortran-11' 'humanity-icon-theme' 'nodejs-doc'
|| true
RUN apt-get autoremove --purge -y
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
