Dandandan commented on PR #20839: URL: https://github.com/apache/datafusion/pull/20839#issuecomment-4034155970
> Thanks @Dandandan just yesterday I was looking into datafusion parquet reads takes more memory comparing to jvm+native and came to some hotspots, one of them as you mentioned in `get_byte_ranges` > > let me try to run my test case with this PR and see benefits In theory this change shouldn't change anything currently, do you think it does? When running locally, I found we can make some queries a bit faster by coalescing adjacent ranges (this happens sometimes and saves IO requests), but left this for a future PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
