[julia-users] Re: Replacement for jl_gc_preserve() and jl_gc_unpreserve()?
Thanks for the heads up Palli. I was using "real-time" in the sense of a process running continuously with new data coming in, flowing through signal processing chains and being displayed and recorded, rather than being absolutely time critical. If a Julia gc causes a momentary delay in one or more of the channels, that should not be a problem. I agree that real-time isn't strictly about performance because the data is flowing through the system in a time limited way so all is ok so long as it is keeping up over the medium term. But in our system, the same code is also used offline when opening files, and in this case the user has to wait around. So far I have found signal processing components implemented with Julia scripts to be within 10% of their C++ equivalents in performance. On Wednesday, August 10, 2016 at 11:34:54 PM UTC+12, Páll Haraldsson wrote: > > On Wednesday, August 10, 2016 at 6:57:15 AM UTC, Kit Adams wrote: >> >> I am investigating the feasibility of embedding Julia in a C++ real-time >> signal processing framework, using Julia-0.4.6 (BTW, the performance is >> looking amazing). >> > > There are other thread[s] on Julia and real-time, I'll not repeat here. > You are ok with Julia for real-time work? Julia strictly isn't real-time, > you just have to be careful. > > I'm not looking into your embedding/GC issues as I'm not too familiar > with. Seems to me embedding doesn't fundamentally change that the GC isn't > real-time. And real-time isn't strictly about the performance. > > >> However, for this usage I need to retain Julia state variables across c++ >> function calls, so the stack based JL_GC_PUSH() and JL_GC_POP() are not >> sufficient. >> When I injected some jl_gc_collect() calls for testing purposes, to >> simulate having multiple Julia scripts running (from the same thread), I >> got crashes, which I was able to fix using e.g. jl_gc_preserve(mMyState); >> and appropriate matching jl_gc_unpreserve() calls. >> >> I see these functions have been removed from the latest Julia version. >> >> Is there an alternative that allows Julia values to be retained in a C++ >> app across gc calls? >> > > -- > Palli. > >
Re: [julia-users] Replacement for jl_gc_preserve() and jl_gc_unpreserve()?
Thanks for those ideas Bart. I think I will try stashing an index into the array in the smart pointer (along side the pointer) and use your idea of a stack of indices to the available null entries. I'm currently working on embedding v8 for comparison purposes, so I won't do this immediately. On Wednesday, August 10, 2016 at 11:06:23 PM UTC+12, Bart Janssens wrote: > > On Wed, Aug 10, 2016 at 11:46 AM Kit Adams <kit@gmail.com > > wrote: > >> Thank you for those links, they are a great help. >> >> Is there an "unprotect_from_gc(T* val)"? >> >> I am looking for a smart pointer a bit like v8's UniquePersistent<>. >> >> I guess I could make one that searched through the array for the value in >> order to remove it (in the smart pointer's dtor). >> >> > No, I didn't need unprotect so far. Iterating over the array is one way, > another option would be to keep a std::map along with the array to > immediately find the index. I think removing elements from the array is > cumbersome, it's probably best to set the value to null (by assigning a > jl_box_voidpointer(nullptr), setting a jl_value_t* to null directly feels > wrong) and depending on how many times objects are added / removed keep a > stack of the empty spots in the array so they can be reused. > >
Re: [julia-users] Replacement for jl_gc_preserve() and jl_gc_unpreserve()?
I guess I could try using std::remove() followed by jl_array_del_end() to remove entries. Cheers, Kit On Wednesday, August 10, 2016 at 9:45:55 PM UTC+12, Kit Adams wrote: > > Thank you for those links, they are a great help. > > Is there an "unprotect_from_gc(T* val)"? > > I am looking for a smart pointer a bit like v8's UniquePersistent<>. > > I guess I could make one that searched through the array for the value in > order to remove it (in the smart pointer's dtor). > > Thanks, > Kit > > > On Wednesday, August 10, 2016 at 8:10:34 PM UTC+12, Bart Janssens wrote: >> >> >> >> On Wed, Aug 10, 2016 at 9:11 AM Yichao Yu <yyc...@gmail.com> wrote: >> >>> On Wed, Aug 10, 2016 at 2:17 PM, Kit Adams <kit@gmail.com> wrote: >>> >>>> I am investigating the feasibility of embedding Julia in a C++ >>>> real-time signal processing framework, using Julia-0.4.6 (BTW, the >>>> performance is looking amazing). >>>> >>>> However, for this usage I need to retain Julia state variables across >>>> c++ function calls, so the stack based JL_GC_PUSH() and JL_GC_POP() are >>>> not >>>> sufficient. >>>> When I injected some jl_gc_collect() calls for testing purposes, to >>>> simulate having multiple Julia scripts running (from the same thread), I >>>> got crashes, which I was able to fix using e.g. jl_gc_preserve(mMyState); >>>> and appropriate matching jl_gc_unpreserve() calls. >>>> >>>> I see these functions have been removed from the latest Julia version. >>>> >>>> Is there an alternative that allows Julia values to be retained in a >>>> C++ app across gc calls? >>>> >>> >>> Copy from my reply on github >>> >>> > This never works in the way you think it did. For keeping a value >>> live, put it in a rooted global array. >>> >>> >>> >> I'm not saying this is the best way to implement Yichao's suggestion, but >> here is how it's done in CxxWrap.jl: >> >> https://github.com/barche/CxxWrap.jl/blob/master/deps/src/cxx_wrap/type_conversion.hpp#L27-L38 >> >> The array is allocated and rooted using jl_set_const here: >> >> https://github.com/barche/CxxWrap.jl/blob/master/deps/src/cxx_wrap/cxx_wrap.cpp#L14-L28 >> >> Cheers, >> >> Bart >> >
Re: [julia-users] Replacement for jl_gc_preserve() and jl_gc_unpreserve()?
Thank you for those links, they are a great help. Is there an "unprotect_from_gc(T* val)"? I am looking for a smart pointer a bit like v8's UniquePersistent<>. I guess I could make one that searched through the array for the value in order to remove it (in the smart pointer's dtor). Thanks, Kit On Wednesday, August 10, 2016 at 8:10:34 PM UTC+12, Bart Janssens wrote: > > > > On Wed, Aug 10, 2016 at 9:11 AM Yichao Yu <yyc...@gmail.com > > wrote: > >> On Wed, Aug 10, 2016 at 2:17 PM, Kit Adams <kit@gmail.com >> > wrote: >> >>> I am investigating the feasibility of embedding Julia in a C++ real-time >>> signal processing framework, using Julia-0.4.6 (BTW, the performance is >>> looking amazing). >>> >>> However, for this usage I need to retain Julia state variables across >>> c++ function calls, so the stack based JL_GC_PUSH() and JL_GC_POP() are not >>> sufficient. >>> When I injected some jl_gc_collect() calls for testing purposes, to >>> simulate having multiple Julia scripts running (from the same thread), I >>> got crashes, which I was able to fix using e.g. jl_gc_preserve(mMyState); >>> and appropriate matching jl_gc_unpreserve() calls. >>> >>> I see these functions have been removed from the latest Julia version. >>> >>> Is there an alternative that allows Julia values to be retained in a C++ >>> app across gc calls? >>> >> >> Copy from my reply on github >> >> > This never works in the way you think it did. For keeping a value live, >> put it in a rooted global array. >> >> >> > I'm not saying this is the best way to implement Yichao's suggestion, but > here is how it's done in CxxWrap.jl: > > https://github.com/barche/CxxWrap.jl/blob/master/deps/src/cxx_wrap/type_conversion.hpp#L27-L38 > > The array is allocated and rooted using jl_set_const here: > > https://github.com/barche/CxxWrap.jl/blob/master/deps/src/cxx_wrap/cxx_wrap.cpp#L14-L28 > > Cheers, > > Bart >
[julia-users] Replacement for jl_gc_preserve() and jl_gc_unpreserve()?
I am investigating the feasibility of embedding Julia in a C++ real-time signal processing framework, using Julia-0.4.6 (BTW, the performance is looking amazing). However, for this usage I need to retain Julia state variables across c++ function calls, so the stack based JL_GC_PUSH() and JL_GC_POP() are not sufficient. When I injected some jl_gc_collect() calls for testing purposes, to simulate having multiple Julia scripts running (from the same thread), I got crashes, which I was able to fix using e.g. jl_gc_preserve(mMyState); and appropriate matching jl_gc_unpreserve() calls. I see these functions have been removed from the latest Julia version. Is there an alternative that allows Julia values to be retained in a C++ app across gc calls?
[julia-users] Re: Embedding julia in VC++ app - crash in jl_init()
Many thanks for these helpful pointers, which led me to this: https://node-julia.readme.io/docs/the-windows-situation Linking against a libopenlibm.lib import library created manually from libopenlibm.dll using dumpbin and lib fixed the crash. Also, in looking to link against libjulia-debug, I noticed the lib/libjulia.dll.a and lib/libjulia-debug.dll.a import libraries supply with the julia install. Linking against these works in VS2012 and saves having to manually create an import library from libjulia.dll. I wonder if a libopenlibm.dll.a could be added to the lib folder in future releases to ease linking to julia from MSVC? Thanks again, Kit On Wednesday, August 3, 2016 at 2:27:32 AM UTC+12, Tony Kelman wrote: > > Can you try building against libjulia-debug or running this in a debugger? > You may need to statically link against openlibm.
[julia-users] Embedding julia in VC++ app - crash in jl_init()
Hi, I am investigating embedding julia in a Windows app built using Visual Studio 2012. I installed the julia-0.4.6 binaries, made an import library (libjulia.lib) from the libjulia.dll using dumpbin and lib. I linked a vc++ console app (based on examples/embedding.c) against libjulia.lib. I run this console app in the julia-0.4.6/bin folder. It crashes with the following stack: Exception: EXCEPTION_ACCESS_VIOLATION at 0x0 -- unknown function (ip: ) unknown function (ip: ) unknown function (ip: ) init at profile.jl:38 __init__ at profile.jl:46 jlcall___init___346 at (unknown line) jl_apply_generic at C:\Julia-0.4.6\bin\libjulia.dll (unknown line) jl_module_run_initializer at C:\Julia-0.4.6\bin\libjulia.dll (unknown line) jl_init_restored_modules at C:\Julia-0.4.6\bin\libjulia.dll (unknown line) jl_get_builtin_hooks at C:\Julia-0.4.6\bin\libjulia.dll (unknown line) julia_init at C:\Julia-0.4.6\bin\libjulia.dll (unknown line) jl_init at C:\Julia-0.4.6\bin\libjulia.dll (unknown line) wmain at g:\devquark\embedjulia\embedjulia\embedjulia.cpp:26 __tmainCRTStartup at f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c:533 wmainCRTStartup at f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c:377 BaseThreadInitThunk at C:\Windows\SYSTEM32\KERNEL32.DLL (unknown line) RtlUnicodeStringToInteger at C:\Windows\SYSTEM32\ntdll.dll (unknown line) RtlUnicodeStringToInteger at C:\Windows\SYSTEM32\ntdll.dll (unknown line) Any help would be greatly appreciated. Cheers, Kit
