clbarnes commented on code in PR #5281:
URL: https://github.com/apache/arrow-rs/pull/5281#discussion_r1442766285
##########
object_store/src/local.rs:
##########
@@ -1082,6 +1097,58 @@ fn convert_walkdir_result(
}
}
+
+/// Download a remote object to a local [`File`]
+pub async fn upload(store: &dyn ObjectStore, location: &Path, opts:
PutOptions, file: &mut std::fs::File) -> Result<()> {
Review Comment:
Yes, because most file systems don't actually have an async API. I guess
it's a balance of that overhead vs having this function block in the middle.
Another possibility would be for these upload and download methods to do
something like
```rust
pub async fn upload(src_store: &dyn ObjectStore, src_location: &Path,
get_opts: GetOptions, tgt_store: &dyn ObjectStore, tgt_location: &Path,
put_opts: PutOptions) -> Result<usize> {...}
```
and if the target store is a local file system, that's fine. Then we re-use
any optimisations we have elsewhere in the crate and it's more flexible, if a
bit more verbose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
