zhengruifeng commented on code in PR #46023:
URL: https://github.com/apache/spark/pull/46023#discussion_r1562034412
##########
python/pyspark/sql/tests/test_readwriter.py:
##########
@@ -252,6 +252,11 @@ def test_create_without_provider(self):
):
df.writeTo("test_table").create()
+ def test_table_overwrite(self):
Review Comment:
before this PR, this parity test fails with
```
ERROR [0.846s]: test_table_overwrite
(pyspark.sql.tests.connect.test_parity_readwriter.ReadwriterV2ParityTests.test_table_overwrite)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/tests/test_readwriter.py",
line 258, in test_table_overwrite
df.writeTo("test_table").overwrite(lit(True))
File
"/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/readwriter.py", line
920, in overwrite
self._spark.client.execute_command(
File
"/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/client/core.py",
line 1054, in execute_command
data, _, _, _, properties = self._execute_and_fetch(req, observations or
{})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/client/core.py",
line 1416, in _execute_and_fetch
for response in self._execute_and_fetch_as_iterator(
File
"/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/client/core.py",
line 1393, in _execute_and_fetch_as_iterator
self._handle_error(error)
File
"/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/client/core.py",
line 1663, in _handle_error
self._handle_rpc_error(error)
File
"/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/client/core.py",
line 1735, in _handle_rpc_error
raise convert_exception(
pyspark.errors.exceptions.connect.SparkConnectGrpcException:
(org.apache.spark.sql.connect.common.InvalidPlanInput) Expression with ID: 0 is
not supported
JVM stacktrace:
org.apache.spark.sql.connect.common.InvalidPlanInput
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
