tustvold commented on issue #5427: URL: https://github.com/apache/arrow-rs/issues/5427#issuecomment-1963382878
I think I would prefer this to be implemented externally. The parquet crate is designed to work well with the IO abstractions of object_store, and introducing a second different IO abstraction will present us with conflicting design constraints. Ultimately filesystem style abstractions don't work well for performing vectored IO against files in object storage, as is needed by a parquet reader. https://github.com/apache/opendal/issues/3675 is a manifestation of this. By keeping object_store as the recommended first-party IO abstraction we can ensure users get the performance they expect out of the box. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
