FelixYBW commented on PR #7697: URL: https://github.com/apache/incubator-gluten/pull/7697#issuecomment-2443421048
Just talked with Ke. Here is a summary: 1. Previously we link to libhdfs3 at compile time, it can be static or dynamic link. 2. Velox currently loads libhdfs library (can be jvm libhdfs or libhdfs3) at run time, record the pointer to libhdfs functions, then indirectly call the function when accessing hdfd. 3. libhdfs shows much worse performance than libhdfs3 4. libhdfs3 has a ~7% regression compared to link way in 1. The PR is to confirm 4. Also we need to make clear the root cause of 3. Thank you! Ke. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
