GitHub user acvictor created a discussion: Dynamic vs static linking of libstdc++ and libgcc for Gluten libraries
Hi Gluten community, We're deploying Gluten/Velox in a production environment where we're encountering library conflicts with other components that dynamically link libstdc++ and libgcc. We'd like to get the community's input on switching from static to dynamic linking for these libraries. Context: - We ship on a single, controlled OS environment - Other components in our runtime environment are dynamically linking libstdc++/libgcc - We see a crash during Gluten plugin initialization after loading the .so files and somewhere during NativeBackendInitializer,java flow that goes away when we switch to dynamic linking. Crash stack ``` # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x0000772957b1ee94, pid=5535, tid=5603 # # JRE version: OpenJDK Runtime Environment Microsoft-12574458 (21.0.9+10) (build 21.0.9+10-LTS) # Java VM: OpenJDK 64-Bit Server VM Microsoft-12574458 (21.0.9+10-LTS, mixed mode, sharing, tiered, compressed class ptrs, parallel gc, linux-amd64) # Problematic frame: # C [libgluten.so+0xd1ee94] std::__codecvt_utf16_base<char16_t>::do_unshift(__mbstate_t&, char*, char*, char*&) const+0x4 # # Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %d" (or dumping to /mnt/var/hadoop/tmp/nm-local-dir/usercache/trusted-service-user/appcache/application_1764833388392_0001/container_1764833388392_0001_02_000001/core.5535) # # If you would like to submit a bug report, please visit: # https://github.com/microsoft/openjdk/issues # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # ``` - Gluten currently uses static linking via CMake flags: ``` set(CMAKE_EXE_LINKER_FLAGS "-static-libstdc++ -static-libgcc") set(CMAKE_SHARED_LINKER_FLAGS "-static-libstdc++ -static-libgcc") ``` Questions: - Are there known issues with dynamic linking of libstdc++/libgcc for Gluten/Velox? We understand static linking provides portability across different OS versions, but in our case we have single OS. - Production risks: Beyond the obvious GLIBCXX version compatibility (which we can control), what other risks should we consider? Thread-local storage, exception handling across library boundaries, etc.? Has anyone else in the community deployed with dynamic linking? Thanks in advance for your insights! cc @zhli1142015 @weiting-chen GitHub link: https://github.com/apache/incubator-gluten/discussions/11282 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
