This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new 513709e8f11 [SPARK-41817][CONNECT][PYTHON][TEST] Enable the doctest
pyspark.sql.connect.readwriter.DataFrameReader.option
513709e8f11 is described below
commit 513709e8f11b4b45a604fc7c0c1bef9a2c7f5081
Author: Takuya UESHIN <[email protected]>
AuthorDate: Thu Feb 16 11:32:02 2023 +0900
[SPARK-41817][CONNECT][PYTHON][TEST] Enable the doctest
pyspark.sql.connect.readwriter.DataFrameReader.option
### What changes were proposed in this pull request?
Enables the doctest `pyspark.sql.connect.readwriter.DataFrameReader.option`.
### Why are the changes needed?
The issue described
[SPARK-41817](https://issues.apache.org/jira/browse/SPARK-41817) was already
fixed at #39545.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The enabled test.
Closes #40046 from ueshin/issues/SPARK-41817/read.
Authored-by: Takuya UESHIN <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit e3d98a7e07596c647d974703fce4cd7d41ad736e)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/sql/connect/readwriter.py | 3 +--
python/pyspark/sql/readwriter.py | 2 --
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/python/pyspark/sql/connect/readwriter.py
b/python/pyspark/sql/connect/readwriter.py
index 7ac034dc221..d43d8e5488f 100644
--- a/python/pyspark/sql/connect/readwriter.py
+++ b/python/pyspark/sql/connect/readwriter.py
@@ -692,8 +692,7 @@ def _test() -> None:
globs = pyspark.sql.connect.readwriter.__dict__.copy()
- # TODO(SPARK-41817): Support reading with schema
- del pyspark.sql.connect.readwriter.DataFrameReader.option.__doc__
+ # TODO(SPARK-42458): createDataFrame should support DDL string as schema
del pyspark.sql.connect.readwriter.DataFrameWriter.option.__doc__
globs["spark"] = (
diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py
index 19ff342ac3b..93fd938dff4 100644
--- a/python/pyspark/sql/readwriter.py
+++ b/python/pyspark/sql/readwriter.py
@@ -174,8 +174,6 @@ class DataFrameReader(OptionUtils):
Examples
--------
- >>> from pyspark.sql import SparkSession
- >>> spark = SparkSession.builder.master("local").getOrCreate()
>>> spark.read.option("key", "value")
<...readwriter.DataFrameReader object ...>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
