This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 02f12eeed0c [SPARK-41658][SPARK-41656][DOCS][FOLLOW-UP] Update JIRAs
in skipped tests' comments
02f12eeed0c is described below
commit 02f12eeed0ce27757edc83e99e05152113ea7f3c
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Tue Jan 3 14:39:45 2023 +0900
[SPARK-41658][SPARK-41656][DOCS][FOLLOW-UP] Update JIRAs in skipped tests'
comments
### What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/39347 and
https://github.com/apache/spark/pull/39347, which updates the invalid JIRAs
linked in the comments.
### Why are the changes needed?
To track the issues properly, and reenable skipped tests in the future.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Comment-only. Linter in CI should verify them. I also manually checked it
in my local.
Closes #39354 from HyukjinKwon/SPARK-41658-followup.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/sql/connect/dataframe.py | 4 ++--
python/pyspark/sql/connect/functions.py | 5 ++---
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/python/pyspark/sql/connect/dataframe.py
b/python/pyspark/sql/connect/dataframe.py
index 0a69b6317f8..57c9e801c22 100644
--- a/python/pyspark/sql/connect/dataframe.py
+++ b/python/pyspark/sql/connect/dataframe.py
@@ -1394,7 +1394,7 @@ def _test() -> None:
sc, options={"spark.app.name": "sql.connect.dataframe tests"}
)
- # TODO(SPARK-41819): Implement RDD.getNumPartitions
+ # Spark Connect does not support RDD but the tests depend on them.
del pyspark.sql.connect.dataframe.DataFrame.coalesce.__doc__
del pyspark.sql.connect.dataframe.DataFrame.repartition.__doc__
@@ -1420,7 +1420,7 @@ def _test() -> None:
del pyspark.sql.connect.dataframe.DataFrame.replace.__doc__
del pyspark.sql.connect.dataframe.DataFrame.intersect.__doc__
- # TODO(SPARK-41826): Implement Dataframe.readStream
+ # TODO(SPARK-41625): Support Structured Streaming
del pyspark.sql.connect.dataframe.DataFrame.isStreaming.__doc__
# TODO(SPARK-41827): groupBy requires all cols be Column or str
diff --git a/python/pyspark/sql/connect/functions.py
b/python/pyspark/sql/connect/functions.py
index f2d3aa64728..6e688271a3f 100644
--- a/python/pyspark/sql/connect/functions.py
+++ b/python/pyspark/sql/connect/functions.py
@@ -2344,6 +2344,8 @@ def _test() -> None:
globs["_spark"] = PySparkSession(
sc, options={"spark.app.name": "sql.connect.functions tests"}
)
+ # Spark Connect does not support Spark Context but the test depends on
that.
+ del pyspark.sql.connect.functions.monotonically_increasing_id.__doc__
# TODO(SPARK-41833): fix collect() output
del pyspark.sql.connect.functions.array.__doc__
@@ -2406,9 +2408,6 @@ def _test() -> None:
# TODO(SPARK-41836): Implement `transform_values` function
del pyspark.sql.connect.functions.transform_values.__doc__
- # TODO(SPARK-41839): Implement SparkSession.sparkContext
- del pyspark.sql.connect.functions.monotonically_increasing_id.__doc__
-
# TODO(SPARK-41840): Fix 'Column' object is not callable
del pyspark.sql.connect.functions.first.__doc__
del pyspark.sql.connect.functions.last.__doc__
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
