comphead commented on code in PR #5584:
URL: https://github.com/apache/arrow-datafusion/pull/5584#discussion_r1138908503
##########
datafusion/core/src/dataframe.rs:
##########
@@ -930,10 +932,11 @@ impl DataFrame {
self,
path: &str,
writer_properties: Option<WriterProperties>,
+ save_mode: FileWriterSaveMode,
Review Comment:
Right. @alamb @metesynnada
I was thinking initially to reuse `WriterProperties` optional key value
metadata, so it can has arbitrary keys and values, including savemode and
whatever. But `WriterProperties` is created for Parquet only in arrow-rs. CSV
writer has its write properties as top struct fields
https://github.com/comphead/arrow-rs/blob/master/arrow-csv/src/writer.rs#L82
Ideally is to place such struct to arrow-rs and introduce it to all writers,
to make them consistent. This will allow us to avoid conflicts(if arrow-rs
decides to add their own saveMode), it will mess with ours defined in separate
structure.
But defining such struct in arrow-rs might be too breaking, so yes, lets try
to do it in DF and backport to arrow-rs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
