crepererum commented on code in PR #4971:
URL: https://github.com/apache/arrow-rs/pull/4971#discussion_r1370006109
##########
object_store/src/multipart.rs:
##########
@@ -263,3 +267,42 @@ impl<T: PutPart> std::fmt::Debug for WriteMultiPart<T> {
.finish()
}
}
+
+/// A low-level interface for interacting with multipart upload APIs
+///
+/// Most use-cases should prefer [`ObjectStore::put_multipart`] as this is
supported by more
+/// backends, including [`LocalFileSystem`], and automatically handles
uploading fixed
+/// size parts of sufficient size in parallel
+///
+/// [`ObjectStore::put_multipart`]: crate::ObjectStore::put_multipart
+/// [`LocalFileSystem`]: crate::local::LocalFileSystem
+#[async_trait]
+pub trait MultiPartStore: Send + Sync + 'static {
+ /// Creates a new multipart upload, returning the [`MultipartId`]
+ async fn create_multipart(&self, path: &Path) -> Result<MultipartId>;
+
+ /// Uploads a new part with index `part_idx`
+ ///
+ /// Most stores require that all parts apart from the final are at least 5
MiB. Additionally
+ /// some stores require all parts excluding the last to be the same size,
e.g. [R2]
+ ///
+ /// [R2]:
https://developers.cloudflare.com/r2/objects/multipart-objects/#limitations
+ async fn put_part(
+ &self,
+ path: &Path,
+ id: &MultipartId,
+ part_idx: usize,
Review Comment:
So `part_idx` goes from `0..N` (N being the number of parts, are holes
allowed?) and the list passed to the complete call must have the same order as
if I would order the results of PUT calls by the `part_id` arg?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
