This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 001d8b0cddc [SPARK-37554][BUILD] Add PyArrow, pandas and plotly to
release Docker image dependencies
001d8b0cddc is described below
commit 001d8b0cddcec46a44e7c6e31612dc2baada05d5
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Mon Dec 6 12:00:51 2021 +0900
[SPARK-37554][BUILD] Add PyArrow, pandas and plotly to release Docker image
dependencies
### What changes were proposed in this pull request?
This PR proposes to add plotly, pyarrow and pandas dependencies for
generating the API documentation for pandas API on Spark.
The versions of `pandas==1.1.5 pyarrow==3.0.0 plotly==5.4.0` are matched
with the current versions being used in branch-3.2 at Python 3.6.
### Why are the changes needed?
Currently, the function references for pandas API on Spark are all missing:
https://spark.apache.org/docs/latest/api/python/reference/pyspark.pandas/series.html
due to missing dependencies when building the docs.
### Does this PR introduce _any_ user-facing change?
Yes, the broken links of documentation at
https://spark.apache.org/docs/latest/api/python/reference/pyspark.pandas/series.html
will all be recovered.
### How was this patch tested?
To be honest, it has not been tested. I don't have the nerve to run Docker
releasing script for the sake of testing so I defer to the next release manager.
The combinations of the dependency versions are being tested in GitHub
Actions at `branch-3.2`.
Closes #34813 from HyukjinKwon/SPARK-37554.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 03750c046b55f60b43646c8108e5f2e540782755)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/create-release/spark-rm/Dockerfile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dev/create-release/spark-rm/Dockerfile
b/dev/create-release/spark-rm/Dockerfile
index ecfb6762d9f..83752bd941d 100644
--- a/dev/create-release/spark-rm/Dockerfile
+++ b/dev/create-release/spark-rm/Dockerfile
@@ -42,7 +42,7 @@ ARG APT_INSTALL="apt-get install --no-install-recommends -y"
# We should use the latest Sphinx version once this is fixed.
# TODO(SPARK-35375): Jinja2 3.0.0+ causes error when building with Sphinx.
# See also https://issues.apache.org/jira/browse/SPARK-35375.
-ARG PIP_PKGS="sphinx==3.0.4 mkdocs==1.1.2 numpy==1.19.4
pydata_sphinx_theme==0.4.1 ipython==7.19.0 nbsphinx==0.8.0 numpydoc==1.1.0
jinja2==2.11.3 twine==3.4.1 sphinx-plotly-directive==0.1.3"
+ARG PIP_PKGS="sphinx==3.0.4 mkdocs==1.1.2 numpy==1.19.4
pydata_sphinx_theme==0.4.1 ipython==7.19.0 nbsphinx==0.8.0 numpydoc==1.1.0
jinja2==2.11.3 twine==3.4.1 sphinx-plotly-directive==0.1.3 pandas==1.1.5
pyarrow==3.0.0 plotly==5.4.0"
ARG GEM_PKGS="bundler:2.2.9"
# Install extra needed repos and refresh.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
