tustvold commented on code in PR #5205:
URL: https://github.com/apache/arrow-rs/pull/5205#discussion_r1423827219
##########
object_store/src/multipart.rs:
##########
@@ -316,3 +319,273 @@ pub trait MultiPartStore: Send + Sync + 'static {
/// Aborts a multipart upload
async fn abort_multipart(&self, path: &Path, id: &MultipartId) ->
Result<()>;
}
+
+/// Wrapper around a [`MultiPartStore`] that implements [`AsyncWrite`]
+///
+/// Data will be uploaded in fixed size chunks of 10 MiB in parallel,
+/// up to the configured maximum concurrency
+/// The multipart upload will only be created when more than the 10 MiB is
written,
+/// otherwise the data will be uploaded directly as PUT.
+pub struct LazyWriteMultiPart {
Review Comment:
Why can't this just call ObjectStore::put_multipart to obtain a boxed
AsyncWrite if/when it decides to take this path? That would simpler, faster and
work for all stores?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
