alamb commented on issue #5427: URL: https://github.com/apache/arrow-rs/issues/5427#issuecomment-1964820328
> but mainly that encouraging people to use object_store ensures they get the performance and behaviour they expect, with the common maintainer base allowing for efficient triage when people inevitably run into issues. I am not sure that everyone has the same performance expectations or that object_store is the best interface for all uses of reading parquet. I would that @Xuanwo and the rest OpenDAL team would be the ones to triage issues related to reading/writing to parquet using OpenDAL and thus I do think having it in a separate crate would be good. > It is already a significant undertaking supporting what we do currently, I'm less than enthusiastic about adding new storage backends and IO abstractions with very different behaviours and performance characteristics. Right, this is why I was thinking it would actually help maintenance if we separated the IO parts (`object_store`) from the parquet encoding/decoding parts. That way it would be clear where the responsibilities lay -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
