This is an automated email from the ASF dual-hosted git repository.
gaogaotiantian pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 55dc77e5e173 [SPARK-58308][INFRA] Remove usage of ipython_genutils
55dc77e5e173 is described below
commit 55dc77e5e173a4a2bdfcdc287ab884e73942a4b5
Author: Tian Gao <[email protected]>
AuthorDate: Mon Jul 27 13:12:48 2026 -0700
[SPARK-58308][INFRA] Remove usage of ipython_genutils
### What changes were proposed in this pull request?
Removed all usage of `ipython_genutils`.
### Why are the changes needed?
This package was originally introduced by
[SPARK-38517](https://issues.apache.org/jira/browse/SPARK-38517) because of
https://github.com/jupyter/nbconvert/issues/1725 . However, `ipython_genutils`
has not been maintained for 10 years and `nbconvert` quickly removed the usage
and dependency of it. So now our doc gen CI should not rely on it either.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
CI should confirm that we don't need it anymore.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57478 from gaogaotiantian/remove-ipython_genutils.
Authored-by: Tian Gao <[email protected]>
Signed-off-by: Tian Gao <[email protected]>
(cherry picked from commit 3b28be3c85f2c577076a9b821791dfd1bf4e65e1)
Signed-off-by: Tian Gao <[email protected]>
---
dev/create-release/spark-rm/Dockerfile | 4 ++--
dev/spark-test-image/docs/Dockerfile | 3 +--
dev/spark-test-image/lint/Dockerfile | 1 -
3 files changed, 3 insertions(+), 5 deletions(-)
diff --git a/dev/create-release/spark-rm/Dockerfile
b/dev/create-release/spark-rm/Dockerfile
index e87556cf61d6..a41be64f68cc 100644
--- a/dev/create-release/spark-rm/Dockerfile
+++ b/dev/create-release/spark-rm/Dockerfile
@@ -55,10 +55,10 @@ RUN python3.11 -m pip install --ignore-installed
'blinker>=1.6.2' && \
python3.11 -m pip cache purge
# Sphinx and documentation packages
-# See 'ipython_genutils' in SPARK-38517, 'docutils' in SPARK-39421
+# See 'docutils' in SPARK-39421
RUN python3.11 -m pip install 'sphinx==8.2.3' mkdocs
'pydata_sphinx_theme>=0.13' \
sphinx-copybutton nbsphinx numpydoc markupsafe 'pyzmq<24.0.0' \
- ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.23.2' pyarrow
pandas \
+ ipython sphinx_plotly_directive 'numpy>=1.23.2' pyarrow pandas \
'plotly>=4.8' 'docutils' 'mypy==1.19.1' 'pytest==7.1.3' \
'pytest-mypy-plugins==1.9.3' 'ruff==0.14.8' 'pandas-stubs==1.2.0.53' \
'grpcio==1.76.0' 'grpc-stubs==1.24.11'
'googleapis-common-protos-stubs==2.2.0'
diff --git a/dev/spark-test-image/docs/Dockerfile
b/dev/spark-test-image/docs/Dockerfile
index 22022dfb77c4..b88036916f2c 100644
--- a/dev/spark-test-image/docs/Dockerfile
+++ b/dev/spark-test-image/docs/Dockerfile
@@ -89,10 +89,9 @@ ENV VIRTUAL_ENV=/opt/spark-venv
RUN python3.12 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
-# See 'ipython_genutils' in SPARK-38517
# See 'docutils<0.18.0' in SPARK-39421
RUN python3.12 -m pip install 'sphinx==8.2.3' mkdocs
'pydata_sphinx_theme>=0.13' sphinx-copybutton nbsphinx numpydoc markupsafe \
- ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.23.2'
'pyarrow>=23.0.0' 'pandas==2.3.3' 'plotly>=4.8' 'docutils' \
+ ipython sphinx_plotly_directive 'numpy>=1.23.2' 'pyarrow>=23.0.0'
'pandas==2.3.3' 'plotly>=4.8' 'docutils' \
'mypy==1.19.1' 'pytest==7.1.3' 'pytest-mypy-plugins==1.9.3' 'ruff==0.14.8' \
'pandas-stubs==1.2.0.53' 'grpcio==1.76.0' 'grpcio-status==1.76.0'
'protobuf==6.33.5' 'grpc-stubs==1.24.11'
'googleapis-common-protos-stubs==2.2.0' \
&& python3.12 -m pip cache purge
diff --git a/dev/spark-test-image/lint/Dockerfile
b/dev/spark-test-image/lint/Dockerfile
index ead0e6488c0f..a6c7e48d48a1 100644
--- a/dev/spark-test-image/lint/Dockerfile
+++ b/dev/spark-test-image/lint/Dockerfile
@@ -89,7 +89,6 @@ RUN python3.12 -m pip install \
'grpcio==1.76.0' \
'zstandard==0.25.0' \
'ipython' \
- 'ipython_genutils' \
'matplotlib' \
'mypy==1.19.1' \
'numpy==2.4.1' \
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]