JkSelf commented on issue #7746: URL: https://github.com/apache/incubator-gluten/issues/7746#issuecomment-2655472542
@Yohahaha `date 1024`, data was read using the native libhdfs3 library, which does not take advantage of short-circuit reads. However, `date 1119`, data was accessed using the JVM-based libhdfs, which benefits from short-circuit reads. I enabled short-circuit reads in my local test, resulting in a 1.09x performance improvement as noted above. And you can re-test with enabling short circuit read. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
