JigaoLuo commented on issue #8378: URL: https://github.com/apache/arrow-rs/issues/8378#issuecomment-3320377269
I couldn’t agree more. There are already some papers that systematically study Parquet internals: data types, compression ratios, and distribution characteristics like cardinality. But one aspect I feel is still missing is **a reader-friendly Parquet configuration optimized for high-performance reading**. That’s also how I interpreted your perspective on Parquet and proprietary formats. I can imagine scenarios where lakehouse consumers experience slow reads and instinctively blame the reader implementation, when in fact the bottleneck lies in how the Parquet-file-config was written. Once I wrap up the current query engine work, I’d love to pursue a more systematic study on this topic if possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
