This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f1d1dc1f87a [SPARK-45706][PYTHON][DOCS] Makes entire Binder build
fails fast during setting up
f1d1dc1f87a is described below
commit f1d1dc1f87a7e4accd1c3c2d824f39df05465906
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Fri Oct 27 21:20:40 2023 +0900
[SPARK-45706][PYTHON][DOCS] Makes entire Binder build fails fast during
setting up
### What changes were proposed in this pull request?
This PR proposes to make entire Binder build fails fast during setting up
to prevent the Binder image to be successfully built which it cannot be rebuilt
later on the same commit.
### Why are the changes needed?
Binder build is currently broken for Spark 3.5.0:
https://mybinder.org/v2/gh/apache/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb
Seems like we uploaded PySpark late into PyPI, and the installation steps
just slightly ignored the failure (a user triggered the first docker image for
Binder, and that's being reused at that time PySpark wasn't uploaded to PyPI).
### Does this PR introduce _any_ user-facing change?
Yes, it fixes the user-facing live notebooks (at
https://spark.apache.org/docs/latest/api/python/index.html).
### How was this patch tested?
Manually tested in my fork:
-
https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb
-
https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_connect.ipynb
-
https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_ps.ipynb
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #43553 from HyukjinKwon/SPARK-45706.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
binder/postBuild | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/binder/postBuild b/binder/postBuild
index b6bdf72324c..b1eed6a8b55 100644
--- a/binder/postBuild
+++ b/binder/postBuild
@@ -20,6 +20,11 @@
# This file is used for Binder integration to install PySpark available in
# Jupyter notebook.
+# SPARK-45706: Should fail fast. Otherwise, the Binder image is successfully
+# built, and it cannot be rebuilt.
+set -o pipefail
+set -e
+
VERSION=$(python -c "exec(open('python/pyspark/version.py').read());
print(__version__)")
TAG=$(git describe --tags --exact-match 2>/dev/null)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
