PHILO-HE commented on issue #8469: URL: https://github.com/apache/incubator-gluten/issues/8469#issuecomment-2586239466
@rajatma1993, > I want to clarify my understanding regarding performance improvements related to AVX support. It seems AVX itself doesn’t directly bring performance gains but rather enables native libraries that leverage AVX to provide performance improvements, depending on the underlying machine's capabilities. Is this correct? Yes. As I mentioned, some libs like simdjson have some avx-specific code for explicit optimization. And we can also ask compiler to do auto-vectorization by setting O2 or O3 flag along with the avx flag. But it is not enabled in Gluten/Velox. >Also, regarding the recommendation to set -march=native: Does this mean users need to explicitly handle this flag during their build process when setting up Gluten? Is it correct to assume that -march=native is not provided by default? If `-march=native` is used, we assume that same CPU architecture is used at both compiling time and runtime. According to one user's feedback, the `-march=native` setting causes fatal error due to the use of some older CPU architecture in their cluster, which is essentially caused by the incompatibility of compiler-optimized binary code. So to avoid such issue, `-march=native` is not set by default in Gluten. We will provide one option to let user decide the setting. >Also can you help me getting to know files/folders if any, where I can see that what are the Native libs is used in this Project ? For velox backend, please go to check cpp/ folder where you can see the native libs in using, such as Velox, Arrow, RE2, etc. As you know, Velox is the most important native lib, so you can also check the transitive dependencies from Velox code repo: https://github.com/facebookincubator/velox. You may find any optimization opportunity on native execution from these direct/transitive native dependencies. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
