This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 2121a5a31d69 [SPARK-55436][INFRA] Upgrade lint and doc test images to
Ubuntu 24.04
2121a5a31d69 is described below
commit 2121a5a31d6931eb2ce281f2da2d32370142ebb8
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Mon Feb 9 15:42:46 2026 +0900
[SPARK-55436][INFRA] Upgrade lint and doc test images to Ubuntu 24.04
### What changes were proposed in this pull request?
Upgrade lint and doc test images to Ubuntu 24.04
### Why are the changes needed?
to test against newer os version
### Does this PR introduce _any_ user-facing change?
no, infra-only
### How was this patch tested?
ci, the PR builder should cover the change
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #54220 from zhengruifeng/u24_doc_lint.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
dev/spark-test-image/docs/Dockerfile | 16 ++++++++++------
dev/spark-test-image/lint/Dockerfile | 21 ++++++++++-----------
2 files changed, 20 insertions(+), 17 deletions(-)
diff --git a/dev/spark-test-image/docs/Dockerfile
b/dev/spark-test-image/docs/Dockerfile
index 5e04b17e4b7d..db9f5b11d016 100644
--- a/dev/spark-test-image/docs/Dockerfile
+++ b/dev/spark-test-image/docs/Dockerfile
@@ -15,16 +15,16 @@
# limitations under the License.
#
-# Image for building and testing Spark branches. Based on Ubuntu 22.04.
+# Image for building and testing Spark branches. Based on Ubuntu 24.04.
# See also in https://hub.docker.com/_/ubuntu
-FROM ubuntu:jammy-20240911.1
+FROM ubuntu:noble
LABEL org.opencontainers.image.authors="Apache Spark project
<[email protected]>"
LABEL org.opencontainers.image.licenses="Apache-2.0"
LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for
Documentation"
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""
-ENV FULL_REFRESH_DATE=20250616
+ENV FULL_REFRESH_DATE=20260208
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true
@@ -63,9 +63,7 @@ RUN apt-get update && apt-get install -y \
ruby-dev \
software-properties-common \
wget \
- zlib1g-dev \
- && rm -rf /var/lib/apt/lists/*
-
+ zlib1g-dev
# See more in SPARK-39959, roxygen2 < 7.2.1
RUN Rscript -e "install.packages(c('devtools', 'knitr', 'markdown',
'rmarkdown', 'testthat'), repos='https://cloud.r-project.org/')" && \
@@ -83,6 +81,12 @@ RUN apt-get update && apt-get install -y \
&& apt-get autoremove --purge -y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
+
+# Setup virtual environment
+ENV VIRTUAL_ENV=/opt/spark-venv
+RUN python3.11 -m venv --without-pip $VIRTUAL_ENV
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11
# Should unpin 'sphinxcontrib-*' after upgrading sphinx>5
diff --git a/dev/spark-test-image/lint/Dockerfile
b/dev/spark-test-image/lint/Dockerfile
index c808b12d127c..1a2684be015d 100644
--- a/dev/spark-test-image/lint/Dockerfile
+++ b/dev/spark-test-image/lint/Dockerfile
@@ -15,16 +15,16 @@
# limitations under the License.
#
-# Image for building and testing Spark branches. Based on Ubuntu 22.04.
+# Image for building and testing Spark branches. Based on Ubuntu 24.04.
# See also in https://hub.docker.com/_/ubuntu
-FROM ubuntu:jammy-20240911.1
+FROM ubuntu:noble
LABEL org.opencontainers.image.authors="Apache Spark project
<[email protected]>"
LABEL org.opencontainers.image.licenses="Apache-2.0"
LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for Linter"
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""
-ENV FULL_REFRESH_DATE=20260129
+ENV FULL_REFRESH_DATE=20260208
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true
@@ -51,12 +51,15 @@ RUN apt-get update && apt-get install -y \
nodejs \
npm \
pkg-config \
+ python3.12 \
qpdf \
tzdata \
r-base \
software-properties-common \
wget \
zlib1g-dev \
+ && apt-get autoremove --purge -y \
+ && apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN Rscript -e "install.packages(c('devtools', 'knitr', 'markdown',
'rmarkdown', 'testthat'), repos='https://cloud.r-project.org/')" \
@@ -67,14 +70,10 @@ RUN Rscript -e "install.packages(c('devtools', 'knitr',
'markdown', 'rmarkdown',
# See more in SPARK-39735
ENV
R_LIBS_SITE="/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"
-# Install Python 3.12
-RUN add-apt-repository ppa:deadsnakes/ppa
-RUN apt-get update && apt-get install -y \
- python3.12 \
- && apt-get autoremove --purge -y \
- && apt-get clean \
- && rm -rf /var/lib/apt/lists/*
-
+# Setup virtual environment
+ENV VIRTUAL_ENV=/opt/spark-venv
+RUN python3.12 -m venv --without-pip $VIRTUAL_ENV
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.12
RUN python3.12 -m pip install \
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
