WweiL commented on code in PR #40691:
URL: https://github.com/apache/spark/pull/40691#discussion_r1163151275
##########
python/pyspark/sql/dataframe.py:
##########
@@ -535,7 +535,7 @@ def writeStream(self) -> DataStreamWriter:
>>> with tempfile.TemporaryDirectory() as d:
... # Create a table with Rate source.
... df.writeStream.toTable(
- ... "my_table", checkpointLocation=d) # doctest: +ELLIPSIS
Review Comment:
My understanding is that in doctest the result will be kind of regex checked
if this flag is set. Like the line below right now is `
<...streaming.query.StreamingQuery object at 0x...>`, but before it was `
<pyspark.sql.streaming.query.StreamingQuery object at 0x...>`, which would
conflict with connect's test, which returns
`<pyspark.sql.connect.streaming.query.StreamingQuery object at 0x...>`
So to make this test works for both connect and non-connect, we enable this
regex-like check and replace below with ...
But it doesn't matter anyway, as we enabled the flag in test options in the
`__main__` method below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
