This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new d123bc6b09c [MINOR][DOCS] Remove SparkSession constructor invocation
in the example
d123bc6b09c is described below
commit d123bc6b09cfc4f13c3a80ce575eb96c3170f6d4
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Wed Mar 22 09:19:48 2023 +0900
[MINOR][DOCS] Remove SparkSession constructor invocation in the example
### What changes were proposed in this pull request?
This PR proposes to Remove SparkSession constructor invocation in the
example.
While I am here, I piggyback and add an example of Spark Connect.
### Why are the changes needed?
SparkSession's constructor is not meant to be exposed to the end users.
This is also hidden in Scala side.
### Does this PR introduce _any_ user-facing change?
Yes, it removes the usage of SparkSession constructor in the user facing
docs.
### How was this patch tested?
Linters should verify the changes in the CI.
Closes #40505 from HyukjinKwon/minor-docs-session.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 54afa2bfffb46d80cbe9f815a37c3d3325648ef3)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/sql/session.py | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/python/pyspark/sql/session.py b/python/pyspark/sql/session.py
index 3b134b551c3..da234abf927 100644
--- a/python/pyspark/sql/session.py
+++ b/python/pyspark/sql/session.py
@@ -179,10 +179,15 @@ class SparkSession(SparkConversionMixin):
... .getOrCreate()
... )
- Create a Spark session from a Spark context.
+ Create a Spark session with Spark Connect.
- >>> sc = spark.sparkContext
- >>> spark = SparkSession(sc)
+ >>> spark = (
+ ... SparkSession.builder
+ ... .remote("sc://localhost")
+ ... .appName("Word Count")
+ ... .config("spark.some.config.option", "some-value")
+ ... .getOrCreate()
+ ... ) # doctest: +SKIP
"""
class Builder:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
