This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8423d74a76f4 [SPARK-51232][PYTHON][DOCS] Remove PySpark 3.3 and older
logic from `binder/postBuild`
8423d74a76f4 is described below
commit 8423d74a76f4a5105d4480e8691cd2ac11dc99ba
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sun Feb 16 22:28:08 2025 -0800
[SPARK-51232][PYTHON][DOCS] Remove PySpark 3.3 and older logic from
`binder/postBuild`
### What changes were proposed in this pull request?
This PR aims to remove PySpark 3.3 and older logic from `binder/postBuild`
because they are the end of life already and didn't have `PySpark Connect`
module.
### Why are the changes needed?
After simplifying by removing legacy PySpark without `PySpark Connect`
module, we are going to add additional logic to handle `PySpark Connect 4.0+`
because they are different from `PySpark Connect 3.4 ~ 3.5`.
### Does this PR introduce _any_ user-facing change?
No, this is a kind of documentation via Binder service.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #49973 from dongjoon-hyun/SPARK-51232.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
binder/postBuild | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/binder/postBuild b/binder/postBuild
index 0b326f62e8f5..92f4c70ffff5 100755
--- a/binder/postBuild
+++ b/binder/postBuild
@@ -37,11 +37,7 @@ else
SPECIFIER="<="
fi
-if [[ ! $VERSION < "3.4.0" ]]; then
- pip install plotly "pandas<2.0.0"
"pyspark[sql,ml,mllib,pandas_on_spark,connect]$SPECIFIER$VERSION"
-else
- pip install plotly "pandas<2.0.0"
"pyspark[sql,ml,mllib,pandas_on_spark]$SPECIFIER$VERSION"
-fi
+pip install plotly "pandas<2.0.0"
"pyspark[sql,ml,mllib,pandas_on_spark,connect]$SPECIFIER$VERSION"
# Set 'PYARROW_IGNORE_TIMEZONE' to suppress warnings from PyArrow.
echo "export PYARROW_IGNORE_TIMEZONE=1" >> ~/.profile
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
