This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 3bdff63ad904 [SPARK-58038][DOC] Fix to use Hadoop 3.5 in PySpark
installation guide and Spark Connect overview
3bdff63ad904 is described below
commit 3bdff63ad904cc7035f1e8ce05c382cc37c7fc91
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jul 8 10:44:42 2026 -0700
[SPARK-58038][DOC] Fix to use Hadoop 3.5 in PySpark installation guide and
Spark Connect overview
### What changes were proposed in this pull request?
This PR updates the outdated Hadoop version references from `3.3` to `3.5`
in the PySpark installation guide and the Spark Connect overview.
### Why are the changes needed?
Apache Spark has used Hadoop 3.5.0 since SPARK-55657, but these documents
still refer to Hadoop 3.3.
- https://github.com/apache/spark/pull/54448
### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only change.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
Closes #57124 from dongjoon-hyun/SPARK-58038.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit e76c29d4ec0071c5685970636d92337b2ba45bb3)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
docs/spark-connect-overview.md | 2 +-
python/docs/source/getting_started/install.rst | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/spark-connect-overview.md b/docs/spark-connect-overview.md
index df0129ff0d48..4d455a8e4680 100644
--- a/docs/spark-connect-overview.md
+++ b/docs/spark-connect-overview.md
@@ -107,7 +107,7 @@ library.
First, download Spark from the
[Download Apache Spark](https://spark.apache.org/downloads.html) page. Choose
the
latest release in the release drop down at the top of the page. Then choose
your package type, typically
-“Pre-built for Apache Hadoop 3.3 and later”, and click the link to download.
+“Pre-built for Apache Hadoop 3.5 and later”, and click the link to download.
Now extract the Spark package you just downloaded on your computer, for
example:
diff --git a/python/docs/source/getting_started/install.rst
b/python/docs/source/getting_started/install.rst
index 7a299d1e9b4f..d137a19b30c9 100644
--- a/python/docs/source/getting_started/install.rst
+++ b/python/docs/source/getting_started/install.rst
@@ -62,7 +62,7 @@ For PySpark with/without a specific Hadoop version, you can
install it by using
PYSPARK_HADOOP_VERSION=3 pip install pyspark
-The default distribution uses Hadoop 3.3 and Hive 2.3. If users specify
different versions of Hadoop, the pip installation automatically
+The default distribution uses Hadoop 3.5 and Hive 2.3. If users specify
different versions of Hadoop, the pip installation automatically
downloads a different version and uses it in PySpark. Downloading it can take
a while depending on
the network and the mirror chosen. ``PYSPARK_RELEASE_MIRROR`` can be set to
manually choose the mirror for faster downloading.
@@ -79,7 +79,7 @@ It is recommended to use ``-v`` option in ``pip`` to track
the installation and
Supported values in ``PYSPARK_HADOOP_VERSION`` are:
- ``without``: Spark pre-built with user-provided Apache Hadoop
-- ``3``: Spark pre-built for Apache Hadoop 3.3 and later (default)
+- ``3``: Spark pre-built for Apache Hadoop 3.5 and later (default)
Note that this installation of PySpark with/without a specific Hadoop version
is experimental. It can change or be removed between minor releases.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
