laurentgo commented on PR #41497: URL: https://github.com/apache/arrow/pull/41497#issuecomment-2098704050
> > It seems incorrect imho that we rely on importing the JVM symbols on things which should be self contained? Isn't that more of a flaw in how we are building/packaging the gandiva library? > > Not exactly. We do not rely on them logic is to use syms imported by gandiva. Problem is someone forgot about RTLD_GLOBAL flag when writing jni loader and it is just luck that jvm imports the same lib that gandiva does (libgcc). This bug should have beed discovered much earlier but it was hidden by jvm imported lib. I can of understand what you mean: because Gandiva actually looks for [symbols exported by the process](https://github.com/apache/arrow/blob/03f8ae754ede16f118ccdba0abb593b1461024aa/cpp/src/gandiva/engine.cc#L183) to perform symbol resolution, gandiva library needs to be loaded as RTLD_GLOBAL for its own symbols (and the symbols brought by its own dependencies) to work. Yes it fixes the issue, but it seems to be at the expense of polluting the global namespace and possibly allowing symbols exported by the JVM to be used instead of those packaged with gandiva (in case of the same symbol exported in multiple modules). Hence my question: shouldn't gandiva symbol resolution be more restrictive and scoped to only the symbols loaded by the gandiva library (which would mean being able to get the dlopen handle or equivalent, which looks not straightforward to do on Linux) I'd would also point out that the test program attached to #40839 also fails with Arrow 16.0 but with a totally different error, and as I don't see any ARM64 tests in the list of checks, I wonder if other bugs are going unnoticed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
