wjones127 opened a new issue, #2185: URL: https://github.com/apache/arrow-datafusion/issues/2185
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** We are looking at improving the filesystem / object store support in delta-rs, but it seems like it would be better to work on that inside of datafusion's data-access crate instead of doing all that work in delta-rs. delta-rs currently has file system support for local fs, gcs, s3, and adls, with just reading and write whole files. I think we'll want to add streaming reads and writes. **Describe the solution you'd like** Design and implement a streaming write interface into the `ObjectStore` trait. **Describe alternatives you've considered** We could do that work in delta-rs and then contribute it back here later. But it might not transfer well. For example, the current delta-rs S3 filesystem use rusoto, while the datafusion object store uses the AWS SDK. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
