Re: [julia-users] More with embedding with dynamic libraries (Linux)
sys.so assumes that libjulia has been opened as RTLD_GLOBAL, so it gets confused if it wasn't however, perhaps ironically, this is probably not an issue on windows, because it doesn't have an RTLD_GLOBAL flag. instead, we fake it and just annotate every symbol with the library it is expected to be found in. but there's really no reason to not to do this on linux too, since it should reduce dlsym lookup times during startup On Mon, Sep 15, 2014 at 3:35 PM, Jeff Waller truth...@gmail.com wrote: Hi all, Remember this problem with dlopen and openblas https://groups.google.com/forum/#!topic/julia-users/A8HwlmldVTM? That problem was resolved by modifying how embedded Julia loads (perhaps only on OS/X).Well now there is a similar problem on Linux. Same software. The exact problem is different because the dynamic linkers are different. It appears to me from LD_DEBUG that symbols in libjulia.so are not being resolved correctly when sys.so is initialized. Much of the functionality is missing, but for example tuple() still works... The relevant debug information is: 18788: find library=libjulia.so [0]; searching 18788: search path=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/x86_64: /usr/local/julia-aa5ffc6ac6/lib/julia/tls:/usr/local/julia-aa5ffc6ac6/lib/ julia/x86_64:/usr/local/julia-aa5ffc6ac6/lib/julia (RPATH from file /home/jeffw/src/nj-test2/node_modules/node-julia/build/Release/ nj.node) 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/x86_64/ libjulia.so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/ libjulia.so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/x86_64/ libjulia.so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so Followed by: 18788: calling init: /usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so Which returns with no errors Then when calling init on my .so, the following happens 18788: /usr/local/julia-aa5ffc6ac6/lib/../lib/julia/sys.so: error: symbol lookup error: undefined symbol: jl_diverror_exception (fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_ast_rettype (fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_prepare_ast (fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) followed by about 50 more unresolved symbols... But those symbols are in libjulia.so, The effect of this is later on is the following exception Error: Julia ccall: could not find function jl_prepare_ast Here's the output of ldd of the .so library . In linux @rpath is $ORIGIN, and I don't see that here, so maybe that's the culprit, or maybe a red herring linux-vdso.so.1 = (0x7fff765d9000) libjulia.so = /usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so ( 0x7f89c4cd1000) libstdc++.so.6 = /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ( 0x7f89c49c3000) libgcc_s.so.1 = /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x7f89c47ac000) libpthread.so.0 = /lib/x86_64-linux-gnu/libpthread.so.0 ( 0x7f89c458f000) libc.so.6 = /lib/x86_64-linux-gnu/libc.so.6 (0x7f89c41cf000) librt.so.1 = /lib/x86_64-linux-gnu/librt.so.1 (0x7f89c3fc6000) libdl.so.2 = /lib/x86_64-linux-gnu/libdl.so.2 (0x7f89c3dc2000) libz.so.1 = /lib/x86_64-linux-gnu/libz.so.1 (0x7f89c3bab000) libm.so.6 = /lib/x86_64-linux-gnu/libm.so.6 (0x7f89c38ae000) /lib64/ld-linux-x86-64.so.2 (0x7f89c5fb1000)
Re: [julia-users] More with embedding with dynamic libraries (Linux)
Oh I knew I could count on you guys. Thanks for the reply, of course my natural response is. What is RTLD_GLOBAL? Don't worry too much about that, I can probably look it up. I found a not desirable workaround by setting LD_PRELOAD to load libjulia.so first, but that causes a 2nd problem see here https://github.com/waTeim/node-julia/blob/master/README.md#linux. I'm up for whatever it takes to resolve this. am hopeful I will simply have to set RTLD_GLOBAL on libjulia ... whatever that is.
[julia-users] More with embedding with dynamic libraries (Linux)
Hi all, Remember this problem with dlopen and openblas https://groups.google.com/forum/#!topic/julia-users/A8HwlmldVTM? That problem was resolved by modifying how embedded Julia loads (perhaps only on OS/X).Well now there is a similar problem on Linux. Same software. The exact problem is different because the dynamic linkers are different. It appears to me from LD_DEBUG that symbols in libjulia.so are not being resolved correctly when sys.so is initialized. Much of the functionality is missing, but for example tuple() still works... The relevant debug information is: 18788: find library=libjulia.so [0]; searching 18788: search path=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/x86_64: /usr/local/julia-aa5ffc6ac6/lib/julia/tls:/usr/local/julia-aa5ffc6ac6/lib/ julia/x86_64:/usr/local/julia-aa5ffc6ac6/lib/julia (RPATH from file /home/jeffw/src/nj-test2/node_modules/node-julia/build/Release/nj.node) 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/x86_64/ libjulia.so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/libjulia. so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/x86_64/ libjulia.so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so Followed by: 18788: calling init: /usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so Which returns with no errors Then when calling init on my .so, the following happens 18788: /usr/local/julia-aa5ffc6ac6/lib/../lib/julia/sys.so: error: symbol lookup error: undefined symbol: jl_diverror_exception (fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_ast_rettype ( fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_prepare_ast (fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) followed by about 50 more unresolved symbols... But those symbols are in libjulia.so, The effect of this is later on is the following exception Error: Julia ccall: could not find function jl_prepare_ast Here's the output of ldd of the .so library . In linux @rpath is $ORIGIN, and I don't see that here, so maybe that's the culprit, or maybe a red herring linux-vdso.so.1 = (0x7fff765d9000) libjulia.so = /usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so ( 0x7f89c4cd1000) libstdc++.so.6 = /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ( 0x7f89c49c3000) libgcc_s.so.1 = /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x7f89c47ac000) libpthread.so.0 = /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f89c458f000 ) libc.so.6 = /lib/x86_64-linux-gnu/libc.so.6 (0x7f89c41cf000) librt.so.1 = /lib/x86_64-linux-gnu/librt.so.1 (0x7f89c3fc6000) libdl.so.2 = /lib/x86_64-linux-gnu/libdl.so.2 (0x7f89c3dc2000) libz.so.1 = /lib/x86_64-linux-gnu/libz.so.1 (0x7f89c3bab000) libm.so.6 = /lib/x86_64-linux-gnu/libm.so.6 (0x7f89c38ae000) /lib64/ld-linux-x86-64.so.2 (0x7f89c5fb1000)
