This is an automated email from the ASF dual-hosted git repository.
gaogaotiantian pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 491b63ff7825 [SPARK-57872][4.0][INFRA] Pin `lxml==4.9.4` in Dockerfile
for PyPy to recover branch-4.0 CI
491b63ff7825 is described below
commit 491b63ff7825b8ff6c7bc28a886c6b38cc6fda21
Author: Tian Gao <[email protected]>
AuthorDate: Mon Jul 6 13:34:10 2026 -0700
[SPARK-57872][4.0][INFRA] Pin `lxml==4.9.4` in Dockerfile for PyPy to
recover branch-4.0 CI
### What changes were proposed in this pull request?
This PR pins `lxml==4.9.4` in the PyPy `pip install` commands in
`dev/infra/Dockerfile` and `dev/spark-test-image/pypy-310/Dockerfile` on
branch-4.0. It backports #56950 (branch-4.1) to branch-4.0.
### Why are the changes needed?
The branch-4.0 base image build fails because pip pulls the newest `lxml`,
which builds from source and requires the libxml2/libxslt development headers
that are not available in the PyPy build stage:
```
Error: Please make sure the libxml2 and libxslt development packages are
installed.
ERROR: Failed to build 'lxml' when getting requirements to build wheel
ERROR: process "/bin/sh -c pypy3 -m pip install numpy 'six==1.16.0'
'pandas==2.2.3' scipy coverage matplotlib lxml" did not complete successfully:
exit code: 1
```
This is the same failure fixed on master by
[SPARK-56513](https://github.com/apache/spark/pull/55369), which pinned both
`meson<1.11.0` and `lxml==4.9.4`. `meson` has since been fixed upstream, so
only the `lxml` pin is needed here. `4.9.4` is consistent with the version
specified in `.github/workflows/build_and_test.yml`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
CI.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #56972 from gaogaotiantian/SPARK-57872-pin-lxml-branch-4.0.
Authored-by: Tian Gao <[email protected]>
Signed-off-by: Tian Gao <[email protected]>
---
dev/infra/Dockerfile | 2 +-
dev/spark-test-image/pypy-310/Dockerfile | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index 548389c2c437..7ea422998d28 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -94,7 +94,7 @@ RUN mkdir -p /usr/local/pypy/pypy3.10 && \
ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3.10 && \
ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | pypy3
-RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.2.3' scipy coverage
matplotlib lxml
+RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.2.3' scipy coverage
matplotlib 'lxml==4.9.4'
ARG BASIC_PIP_PKGS="numpy pyarrow>=18.0.0 six==1.16.0 pandas==2.2.3 scipy
plotly>=4.8 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0
scikit-learn>=1.3.2"
diff --git a/dev/spark-test-image/pypy-310/Dockerfile
b/dev/spark-test-image/pypy-310/Dockerfile
index cffcd434c505..13a7e8093ff9 100644
--- a/dev/spark-test-image/pypy-310/Dockerfile
+++ b/dev/spark-test-image/pypy-310/Dockerfile
@@ -71,4 +71,4 @@ RUN mkdir -p /usr/local/pypy/pypy3.10 && \
ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3.10 && \
ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | pypy3
-RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.2.3' scipy coverage
matplotlib lxml
+RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.2.3' scipy coverage
matplotlib 'lxml==4.9.4'
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
