andygrove commented on issue #2970:
URL:
https://github.com/apache/datafusion-comet/issues/2970#issuecomment-6218851668
Overwrite mode works on `main`, as @dwsmith1983 found. I checked it at
c230bf4e98: a native Parquet write with `mode("overwrite")` into a directory
that already held an earlier write replaced it, on Spark 3.5.9
(`CometNativeWriteExec`) and on Spark 4.1.3 (`CometWriteFilesExec`). The second
write succeeded, and the directory read back only its 5 rows.
#4946 added the `SaveMode.Overwrite` handling on Spark 3.x, which
`SaveMode.Overwrite replaces existing parquet data` in
`CometParquetWriterSuite` covers, and on Spark 4.0+ #5763 leaves the save mode
to Spark. Closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
