midnattsol opened a new issue, #6832: URL: https://github.com/apache/arrow-rs/issues/6832
This issue is related with [#5277](https://github.com/apache/arrow-rs/issues/5277). Basically it is created to split the task of creating a **download** and **upload** methods in two different tasks with two different PR for easy review. The original issue was > Is your feature request related to a problem or challenge? Please describe what you are trying to do. > As a user, I would like to utilize the object_store to put and retrieve files. Currently, I can employ the put and get methods for this purpose. However, I am responsible for handling any issues that arise. For instance, when uploading a file to the store, I must verify its size to determine whether multipart upload is necessary. Similarly, when retrieving a large file, I cannot download it in its entirety before writing it to the local file system, as this would cause excessive memory usage. Instead, I may need to use get_range and write specific portions of the file locally. Moreover, downloading different ranges could potentially be executed in parallel. Regards -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
