clbarnes commented on code in PR #5281:
URL: https://github.com/apache/arrow-rs/pull/5281#discussion_r1442766285
##########
object_store/src/local.rs:
##########
@@ -1082,6 +1097,58 @@ fn convert_walkdir_result(
}
}
+
+/// Download a remote object to a local [`File`]
+pub async fn upload(store: &dyn ObjectStore, location: &Path, opts:
PutOptions, file: &mut std::fs::File) -> Result<()> {
Review Comment:
Yes, because most file systems don't actually have an async API. I guess
it's a balance of that overhead vs having this function block in the middle.
Another possibility would be for these upload and download functions to be
replaced with something like
```rust
pub async fn copy_between_stores(
src_store: &dyn ObjectStore, src_location: &Path, get_opts: GetOptions,
tgt_store: &dyn ObjectStore, tgt_location: &Path, put_opts: PutOptions,
) -> Result<usize> {...}
```
and if either store is a local file system, that's fine. Then we re-use any
optimisations we have elsewhere in the crate and it's more flexible, if a bit
more verbose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
