John: Common Lisp and Julia have a lot in common. I didn't mean to suggest writing your software in Lisp, I meant that if ITA was able to run a hugely popular website involving a complicated optimization problem without triggering the GC, then you can do the same in Julia. Like others have suggested, you just preallocate everything (global const arrays), and make sure that every code path is run once (to force compilation) before the system goes online. @time will tell you if you've been successful at eliminating everything. You might run into issues with libraries allocating during their calls, and it might be easier all things considered in C, but it's certainly doable with enough efforts in Julia. I might be up for helping out, if you're interested.
On Thu, Jun 2, 2016 at 3:54 AM, Leger Jonathan <johnlege...@gmail.com> wrote: > Páll: don't worry about the project failing because of YOUUUUUU ;) in any > case we wanted to try Julia and see if we could get help/tips from the > community. > About the nogc I wonder if activating it will also prevent the core of > Julia to be garbage collected ? If yes for long run it's a bad idea to > disable it too long. > > For now the only options options are C/C++ and Julia, sorry no D or Lisp > :) Why would you not recommend C for this kind of tasks ? > And I said 1000 images/sec but the camera may be able to go up to 10 000 > images/sec so I think we can define it as hard real time. > > Thank you for all these ideas ! > > > > Le 01/06/2016 23:59, Páll Haraldsson a écrit : > > On Wednesday, June 1, 2016 at 9:40:54 AM UTC, John leger wrote: >> >> So for now the best is to build a toy that is equivalent in processing >> time to the original and see by myself what I'm able to get. >> We have many ideas, many theories due to the nature of the GC so the best >> is to try. >> >> Páll -> Thanks for the links >> > > No problem. > > While I did say it would be cool to now of Julia in space, I would hate > for the project to fail because of Julia (because of my advice). > > I endorse Julia for all kinds of uses, hard real-time (and building > operating systems) are where I have doubts. > > A. I thought a little more about making a macro @nogc to mark functions, > and it's probably not possible. You could I guess for one function, as the > macro has access to the AST of it. But what you really want to disallow, is > that function calling functions that are not similarly marked. I do not > know about metadata on functions and if a nogc-bit could be put in, but > even then, in theory couldn't that function be changed at runtime..? > > What you would want is that this nogc property is statically checked as I > guess D does, but Julia isn't separately compiled by default. Note there is > Julia2C, and see > > http://juliacomputing.com/blog/2016/02/09/static-julia.html > > for gory details on compiling Julia. > > I haven't looked, I guess Julia2C does not generate malloc and free, only > some malloc substitute in libjulia runtime. That substitute will allocate > and run the GC when needed. These are the calls you want to avoid in your > code and could maybe grep for.. There is a Lint.jl tool, but as memory > allocation isn't an error it would not flag it, maybe it could be an > option.. > > B. One idea I just had (in the shower..), if @nogc is used or just on > "gc_disable" (note it is deprecated*), it would disallow allocations (with > an exception if tried), not just postpone them, it would be much easier to > test if your code uses allocations or calls code that would. Still, you > would have to check all code-paths.. > > C. Ada, or the Spark-subset, might be the go-to language for hard > real-time. Rust seems also good, just not as tried. D could also be an > option with @nogc. And then there is C and especially C++ that I try do > avoid recommending. > > D. Do tell if you only need soft real-time, it makes the matter so much > simpler.. not just programming language choice.. > > * > help?> gc_enable > search: gc_enable > > gc_enable(on::Bool) > > Control whether garbage collection is enabled using a boolean argument > (true for enabled, false for disabled). Returns previous GC state. Disabling > garbage collection should be used only with extreme caution, as it can > cause memory use to grow without bound. > > > > >> >> Le mardi 31 mai 2016 18:44:17 UTC+2, Páll Haraldsson a écrit : >>> >>> On Monday, May 30, 2016 at 8:19:34 PM UTC, Tobias Knopp wrote: >>>> >>>> If you are prepared to make your code to not perform any heap >>>> allocations, I don't see a reason why there should be any issue. When I >>>> once worked on a very first multi-threading version of Julia I wrote >>>> exactly such functions that won't trigger gc since the later was not thread >>>> safe. This can be hard work but I would assume that its at least not more >>>> work than implementing the application in C/C++ (assuming that you have >>>> some Julia experience) >>>> >>> >>> I would really like to know why the work is hard, is it getting rid of >>> the allocations, or being sure there are no more hidden in your code? I >>> would also like to know then if you can do the same as in D language: >>> >>> http://wiki.dlang.org/Memory_Management >>> >>> The most reliable way to guarantee latency is to preallocate all data >>> that will be needed by the time critical portion. If no calls to allocate >>> memory are done, the GC will not run and so will not cause the maximum >>> latency to be exceeded. >>> >>> It is possible to create a real-time thread by detaching it from the >>> runtime, marking the thread function @nogc, and ensuring the real-time >>> thread does not hold any GC roots. GC objects can still be used in the >>> real-time thread, but they must be referenced from other threads to prevent >>> them from being collected." >>> >>> that is would it be possible to make a macro @nogc and mark functions in >>> a similar way? I'm not aware that such a macro is available, to disallow. >>> There is a macro, e.g. @time, that is not sufficient, that shows GC >>> actitivy, but knowing there was none could have been an accident; if you >>> run your code again and memory fills up you see different result. >>> >>> As with D, the GC in Julia is optional. The above @nogc, is really the >>> only thing different, that I can think of that is better with their >>> optional memory management. But I'm no expert on D, and I mey not have >>> looked too closely: >>> >>> https://dlang.org/spec/garbage.html >>> >>> >>>> Tobi >>>> >>>> Am Montag, 30. Mai 2016 12:00:13 UTC+2 schrieb John leger: >>>>> >>>>> Hi everyone, >>>>> >>>>> I am working in astronomy and we are thinking of using Julia for a >>>>> real time, high performance adaptive optics system on a solar telescope. >>>>> >>>>> This is how the system is supposed to work: >>>>> 1) the image is read from the camera >>>>> 2) some correction are applied >>>>> 3) the atmospheric turbulence is numerically estimated in order to >>>>> calculate the command to be sent to the deformable mirror >>>>> >>>>> The overall process should be executed in less than 1ms so that it can >>>>> be integrated to the chain (closed loop). >>>>> >>>>> Do you think it is possible to do all the computation in Julia or >>>>> would it be better to code some part in C/C++. What I fear the most is the >>>>> GC but in our case we can pre-allocate everything, so once we launch the >>>>> system there will not be any memory allocated during the experiment and it >>>>> will run for days. >>>>> >>>>> So, what do you think? Considering the current state of Julia will I >>>>> be able to get the performances I need. Will the garbage collector be an >>>>> hindrance ? >>>>> >>>>> Thank you. >>>>> >>>> >
