This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3c0824d92068 [SPARK-50711][PS][DOCS] Upgrade the minimum version of
Pandas in PS to 2.2.0
3c0824d92068 is described below
commit 3c0824d92068e92c5672b2d2ea1eba9c5c30cc13
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Thu Jan 2 16:43:09 2025 +0800
[SPARK-50711][PS][DOCS] Upgrade the minimum version of Pandas in PS to 2.2.0
### What changes were proposed in this pull request?
Upgrade the minimum version of Pandas in PS to 2.2.0
But the version used in `require_minimum_pandas_version` is still 2.0.0,
because currently this function is used in both PS and non-PS modules.
### Why are the changes needed?
Actually, PS with Pandas < 2.2.0 has already been broken in master branch
### Does this PR introduce _any_ user-facing change?
No, doc-only
### How was this patch tested?
CI
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #49344 from zhengruifeng/ps_pd_mini.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
---
dev/requirements.txt | 2 +-
python/docs/source/getting_started/install.rst | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev/requirements.txt b/dev/requirements.txt
index 33300cc28d3c..c1ae808c5e29 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -5,7 +5,7 @@ py4j>=0.10.9.7
numpy>=1.21
pyarrow>=11.0.0
six==1.16.0
-pandas>=2.0.0
+pandas>=2.2.0
scipy
plotly>=4.8
mlflow>=2.3.1
diff --git a/python/docs/source/getting_started/install.rst
b/python/docs/source/getting_started/install.rst
index b35588a618ac..515224b52e09 100644
--- a/python/docs/source/getting_started/install.rst
+++ b/python/docs/source/getting_started/install.rst
@@ -239,7 +239,7 @@ Installable with ``pip install "pyspark[pandas_on_spark]"``.
========= ================= ================================
Package Supported version Note
========= ================= ================================
-`pandas` >=2.0.0 Required for Pandas API on Spark
+`pandas` >=2.2.0 Required for Pandas API on Spark
`pyarrow` >=11.0.0 Required for Pandas API on Spark
========= ================= ================================
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
