alamb commented on code in PR #15747:
URL: https://github.com/apache/datafusion/pull/15747#discussion_r2049392845
##########
datafusion/datasource/src/write/mod.rs:
##########
@@ -88,6 +91,21 @@ pub async fn create_writer(
file_compression_type.convert_async_writer(buf_writer)
}
+/// Returns an [`AsyncWrite`] which writes to the given object store location
Review Comment:
And then maybe mark `create_writer` as deprecated per this page:
https://datafusion.apache.org/contributor-guide/api-health.html#deprecation-guidelines
(next release will be 48.0.0)
And update `create_writer` to use the new builder?
##########
datafusion/datasource/src/write/mod.rs:
##########
@@ -88,6 +91,21 @@ pub async fn create_writer(
file_compression_type.convert_async_writer(buf_writer)
}
+/// Returns an [`AsyncWrite`] which writes to the given object store location
Review Comment:
If we are going to add a new API, can we please add a Builder style one here
so future additional options are easier ?
So instead of
```rust
let mut object_store_writer = create_writer_with_size(
FileCompressionType::UNCOMPRESSED,
&path,
Arc::clone(&object_store),
context
.session_config()
.options()
.execution
.objectstore_writer_buffer_size,
```
Something more like
```rust
let mut object_store_writer = ObjectStoreWriterBuilder::new(
FileCompressionType::UNCOMPRESSED,
&path,
Arc::clone(&object_store)
).with_buffer_size(
context
.session_config()
.options()
.execution
.objectstore_writer_buffer_size,
).build();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
