dongjoon-hyun commented on PR #46129:
URL: https://github.com/apache/spark/pull/46129#issuecomment-2068736545
`classic` sounds like a too limited wording because it has no clear meaning
and not-extensible in a long-term perspective.
--
This is an automated message from the Apache Git
HyukjinKwon closed pull request #46129: [SPARK-47909][PYTHON][CONNECT] Parent
DataFrame class for Spark Connect and Spark Classic
URL: https://github.com/apache/spark/pull/46129
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
HyukjinKwon commented on PR #46129:
URL: https://github.com/apache/spark/pull/46129#issuecomment-2068279142
Merged to master.
I will followup if there are more comments to address.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573466682
##
python/pyspark/sql/tests/connect/test_connect_plan.py:
##
@@ -333,6 +333,11 @@ def test_observe(self):
from pyspark.sql.connect.observation import
HyukjinKwon commented on PR #46129:
URL: https://github.com/apache/spark/pull/46129#issuecomment-2067819605
Should be ready for a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573465793
##
python/pyspark/sql/classic/dataframe.py:
##
@@ -0,0 +1,1974 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573465384
##
python/pyspark/sql/connect/session.py:
##
@@ -325,7 +325,7 @@ def active(cls) -> "SparkSession":
active.__doc__ = PySparkSession.active.__doc__
-
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573465371
##
python/pyspark/sql/connect/session.py:
##
@@ -325,7 +325,7 @@ def active(cls) -> "SparkSession":
active.__doc__ = PySparkSession.active.__doc__
-
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573095371
##
python/pyspark/sql/classic/dataframe.py:
##
@@ -0,0 +1,1974 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573095413
##
python/pyspark/sql/classic/dataframe.py:
##
@@ -0,0 +1,1974 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573095371
##
python/pyspark/sql/classic/dataframe.py:
##
@@ -0,0 +1,1974 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573094548
##
python/pyspark/sql/utils.py:
##
@@ -302,6 +302,33 @@ def wrapped(*args: Any, **kwargs: Any) -> Any:
return cast(FuncT, wrapped)
+def
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1573094799
##
python/pyspark/sql/connect/session.py:
##
@@ -325,7 +325,7 @@ def active(cls) -> "SparkSession":
active.__doc__ = PySparkSession.active.__doc__
-
ueshin commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1572787315
##
python/pyspark/sql/dataframe.py:
##
@@ -139,51 +123,29 @@ class DataFrame(PandasMapOpsMixin, PandasConversionMixin):
created via using the constructor.
"""
HyukjinKwon commented on PR #46129:
URL: https://github.com/apache/spark/pull/46129#issuecomment-2067140476
Will fix up the tests soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1572006465
##
python/pyspark/sql/connect/dataframe.py:
##
@@ -2306,7 +2183,7 @@ def _test() -> None:
)
(failure_count, test_count) = doctest.testmod(
-
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1571898280
##
python/pyspark/sql/classic/dataframe.py:
##
@@ -0,0 +1,1952 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
HyukjinKwon commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1571898280
##
python/pyspark/sql/classic/dataframe.py:
##
@@ -0,0 +1,1952 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
zhengruifeng commented on code in PR #46129:
URL: https://github.com/apache/spark/pull/46129#discussion_r1571874148
##
python/pyspark/sql/classic/dataframe.py:
##
@@ -0,0 +1,1952 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
HyukjinKwon commented on PR #46129:
URL: https://github.com/apache/spark/pull/46129#issuecomment-2065807249
cc @ueshin @zhengruifeng @allisonwang-db @xinrong-meng @itholic @hvanhovell
@grundprinzip
--
This is an automated message from the Apache Git Service.
To respond to the message,
HyukjinKwon opened a new pull request, #46129:
URL: https://github.com/apache/spark/pull/46129
### What changes were proposed in this pull request?
This PR proposes to have a parent `pyspark.sql.DataFrame` class which
`pyspark.sql.connect.dataframe.DataFrame` and
21 matches
Mail list logo
