alamb commented on PR #10354: URL: https://github.com/apache/arrow-rs/pull/10354#issuecomment-5109512534
In my opinion, it is hard to justify why the parquet crate has a built in integration for one I/O system but not others (OpenDAL being an obvious example, but there are others). The argument that earlier on we needed an integration to help inform API design is reasonable, but I think that time has past now I personally think separating out the I/O concerns from the lower level decoder logic will make the parquet reader easier to use and customize downstream systems (e.g. to do prefetching of I/O for example). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
