> On Dec 17, 2018, at 9:38 PM, Antoine Pitrou <anto...@python.org> wrote:
>
>
> Apparently you're using System.load() to load the compiled helpers
> functions. It seems it may load the library using RTLD_LOCAL, which
> means it won't be seen from other dynamic libraries loaded in the same
> process.
>
> According to this SO entry, the solution would be to use
> NativeLibrary.getInstance() instead:
Thanks a lot. That worked like a charm !
> https://stackoverflow.com/questions/5425034/java-load-shared-librariees-with-dependencies
>
> Regards
>
> Antoine.
>
>
> Le 17/12/2018 à 16:48, Ravindra Pindikura a écrit :
>>
>>
>>> On Dec 17, 2018, at 6:39 PM, Antoine Pitrou <anto...@python.org> wrote:
>>>
>>>
>>> Is LoadLibraryPermanently(nullptr) called after the libgandiva_jni.so is
>>> loaded? Perhaps you can try LoadLibraryPermanently("libgandiva_jni.so")
>>> explicitly?
>>
>> We are already doing this inside gandiva.
>>
>> https://github.com/apache/arrow/blob/master/cpp/src/gandiva/engine.cc#L61
>> <https://github.com/apache/arrow/blob/master/cpp/src/gandiva/engine.cc#L61>
>>
>>>
>>>
>>> Le 17/12/2018 à 13:50, Ravindra Pindikura a écrit :
>>>> Antoine,
>>>>
>>>> I set ARROW_GANDIVA_STATIC_LIBSTDCPP and modified the jni/CMakeLists.txt
>>>> to export all symbols (i.e removed the version-script parameter in
>>>> LINK_FLAGS).
>>>>
>>>> $ nm ./debug/libgandiva_jni.so|grep _ZNSt8ios_base4InitC1Ev
>>>> 0000000001df34f0 T _ZNSt8ios_base4InitC1Ev
>>>>
>>>> $ nm --defined-only ./debug/libgandiva_jni.so|grep _ZNSt8ios_base4InitC1Ev
>>>> 0000000001df34f0 T _ZNSt8ios_base4InitC1Ev
>>>>
>>>> I still get the same “could not be resolved!” Error when I run the java
>>>> test. It appears like when the library is loaded via jni, some symbols are
>>>> getting hidden from the JIT.
>>>>
>>>> Of the four sub-cases of the test :
>>>>
>>>> 1. a cpp executable (shared/static), the decimal functions invoked from
>>>> CPP fns
>>>> 2. a cpp executable (shared/static), the decimal functions invoked from
>>>> JIT code
>>>> 3. a java executable, the decimal functions invoked from CPP Fns
>>>> 4. a java executable, the decimal functions invoked from JIT code
>>>>
>>>> 1, 2 & 3 work good. Only 4 causes the symbol resolution issue.
>>>>
>>>> Thanks & regards,
>>>> Ravindra.
>>>>
>>>>
>>>>> On Dec 16, 2018, at 6:34 PM, Antoine Pitrou <anto...@python.org> wrote:
>>>>>
>>>>>
>>>>> In other words, see if:
>>>>> nm --defined-only libgandiva.a | grep _ZNSt8ios_base4InitC1Ev
>>>>>
>>>>> outputs anything.
>>>>>
>>>>> Regards
>>>>>
>>>>> Antoine.
>>>>>
>>>>>
>>>>> Le 16/12/2018 à 14:03, Antoine Pitrou a écrit :
>>>>>>
>>>>>> Does the static gandiva lib bundle libstdc++? Otherwise you'll probably
>>>>>> need to load it explicitly in the Java process.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Antoine.
>>>>>>
>>>>>>
>>>>>> Le 16/12/2018 à 04:53, Ravindra Pindikura a écrit :
>>>>>>> I think this happens if any of the cpp files that are used for IR
>>>>>>> generation (clang —emit-llvm) includes <iostream>. For decimal support,
>>>>>>> I’m generating IR code from arrow/util/decimal.cc <http://decimal.cc/>
>>>>>>> which includes <iostream> (many times in the header file chain).
>>>>>>>
>>>>>>>> On Dec 16, 2018, at 9:14 AM, Ravindra Pindikura <ravin...@dremio.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I’ve been working on adding decimal support in gandiva - I’m hitting a
>>>>>>>> symbol resolution error with std::ios_base::Init with the java tests
>>>>>>>> on ubuntu. Any ideas on what I may be doing wrong ?
>>>>>>>>
>>>>>>>> LLVM ERROR: Program used external function '_ZNSt8ios_base4InitC1Ev'
>>>>>>>> which could not be resolved!
>>>>>>>>
>>>>>>>>
>>>>>>>> I’ve the same test (add two decimals) in three forms :
>>>>>>>>
>>>>>>>> 1. Cpp test executable with shared gandiva lib
>>>>>>>>
>>>>>>>> Works on both ubuntu and Mac
>>>>>>>>
>>>>>>>> 2. Cpp test executable with static gandiva lib
>>>>>>>>
>>>>>>>> Works on both ubuntu and Mac
>>>>>>>>
>>>>>>>> 3. Java test (internally uses jni over static gandiva lib)
>>>>>>>>
>>>>>>>> Works on Mac, but fails on ubuntu !!
>>>>>>>>
>>>>>>>> Ubuntu failure : https://travis-ci.org/apache/arrow/jobs/468562791
>>>>>>>>
>>>>>>>> I tried switching to ORC JIT (from MC JIT) but that didn’t help. In
>>>>>>>> the JIT, the symbol is not found in the process address space for
>>>>>>>> ubuntu+java. For all the other cases, the symbol is found.
>>>>>>>>
>>>>>>>> https://github.com/apache/arrow/pull/2942/files#diff-5ef29a5cabe384645dbf7a9c704aceeaR36
>>>>>>>>
>>>>>>>> Thanks & regards,
>>>>>>>> Ravindra.
>>>>>>>
>>>>>>>
>>>>
>>
>>
