On Sun, 8 Jan 2012 21:46:29 +1000 David Seikel <[email protected]> said:
> On Sun, 8 Jan 2012 19:52:50 +0900 Carsten Haitzler (The Rasterman) > <[email protected]> wrote: > > > On Sun, 8 Jan 2012 18:54:12 +1000 David Seikel <[email protected]> > > said: > > > > > I just woke up, first email for the day. Not quite grasping why > > > this will be any different between edje Lua API and random host app > > > Lua API. They still both gotta be threadsafe when we thread > > > things. Random host app, being an EFL app, would have to be > > > threadsafe if EFL is threaded. > > > > incorrect. if efl is threaded host ap does not need to be threadsafe. > > that is my point. such a requirement would be onerous and wrong as > > the host doesnt know if edje is threaded or not. it shouldn't need to > > know. > > > > > The Host API stuff will go through the edje Lua API to do it's > > > thing, and the edje Lua API will have to be made threadsafe, so any > > > other thread issues will be the host apps plobrem. Just like apps > > > using the rest of EFL. > > > > thats not the issue - its that the call from lua (in edje) to host > > app happens directly as a function call, thus is lua is running as a > > thread, the call in host spac happens inside a thread created by > > edje. this is unacceptable. it must be MARSHALLED back to the > > mainloop - thus synchronous ipc calls. > > > > > Hell, the projects I'm working on now that need this host API stuff > > > have to be threadsafe anyway. One of the design goals is to be > > > able to run thousands of scripts at once safely and quickly, while > > > the rest of the world code is busy chugging along. That's why I'll > > > be experimenting with luaproc ansd LuaJIT. > > > > well this isnt really about your apps - but the general case - if its > > a general api made available, then the calls TO the host app must > > happen in the main loop context (thread) not another. all things must > > be marshalled back to the mainloop. thus why the messaging/signal api > > is much better here as it is a messaging api, not a call api. > > I understand now, I think. > > The host app itself may be multi threaded, so we do need to cater for > this. We also have to cater for those cases where it does not matter, > and those cases where a messaging API is a bad choice. We would have > to make sure that edje Lua itself is threadsafe first. if host app is threaded or not - all callbacks to the host should happen in the mainloop. anything else is pretty much wrong. (very very very rare exceptions, and this is not one as it is a general purpose communication path). > 0) > > Lua scripts themselves can be run on different threads, it all happens > via the Lua state, which is just a pointer to everything. luaproc is a > system of running lots of scripts in worker threads, where each worker > thread peels the next ready Lua script of the ready queue, runs it > until it yields, then slaps it on the bottom of the run queue. I'll be > experimenting with that soon. yes - but when lua calls back to a function provided by c - that function must run in the mainloop (or be lock-stepped with the mainloop - abysmal performance for doing lots of calls unbatched, same with synchronous messaging). > If I remember, edje Lua puts everything into the Lua state, with no > globals. So it should be threadsafe to start with. If not, it will > have to be made threadsafe anyway when edje goes threaded. > > 1) > > The host function might deal purely with some data structure that is > internal to the script. This means it's userdata (or perhaps just > plain Lua data) attached to the Lua state. This should be inherently > thread safe as per 0). This could be inefficient if dealt with via > messages. it still violates efl's position of always calling back to the mainloop. trust me - its HORRIBLE to call back from threads. it really is. don't do it. > 2) > > That userdata in 1) might involve things that happen in other threads > of the host app. Then it's up to the host app to deal with it's own > threading problems. Not our problem. totally app issue. > 3) > > The host function might deal with EFL stuff. This is a case where it > would be better to have more EFL API in the edje Lua, or use messages. > Messages is a usual method in EFL anyway. So what we are doing already > works for this case. that's our job to make al these extra helping apis then :) > 4) > > The host function might deal with it's own internal data structures. > This I think is the problem you see? If it's not a threadsafe host app, > then things can go horribly wrong when host functions are unexpectedly > called from some random edje thread. correct. > Wont the same problems happen when some host function EFL callback is > unexpectedly called from some random EFL thread? it never is currently. adding your externally provided lua functions will STOP us from being able to put lua execution into a thread. it will always have to be inline in the mainloop and that is a big limitation. messaging solves this problem. > So the same solutions could be used. Host API functions are really > just another callback system. The host registers it's callbacks with > edje Lua, edje Lua calls them back. Why is this different from the > rest of EFL using registered callbacks? because all the other callbacks come from code controlled by efl. lua funcs come from code (script - lua) not controlled by efl. > Only 4) is a problem, unless I've missed a use case? Please add more > if I did. thats the case i talk bout. without threaded execution of edje programs (lua, embryo or anything else) it's moot. on a mental todo list for the future is to allow these to be threaded - in fact to entirely thread them anyway. no options. the edje messages and signals were designed as async to allow for this to happen in the future with zero performance issues AND zero api and abi breaks. i also would love to put edje calculation into threads too and other thing edje does to not block the mainloop as much. (eg do initial check for existence of group and file in mainloop and then pass off rest of load and init into a thread). we do a lot of work loading edje groups/files and we can defer object creation for parts until much later too - thus we definitely can move load and setup of internal structs into threads without a problem. my problem is that by making lua extensible by functions provided from the app, these functions must, by efl convention, be called from the mainloop and thus you must implement them as synchronous ipc (if we add threads) thus you have to marshall/unmarshall params/returns down pipes. if you are going to do this you shoudl just do it all to begin with that way and simply have the write/read right next to eachother. this of course will add overhead to this. async messages are much more desirable as they allow both sides to not block. :) > BTW, I brought up my apps as it WILL be dealing with thread issues. So > it might make for a good test of my proposed addition. Without my > addition, people writing scripts for my apps will have to write two > scripts, one that deals with the GUI, and sends messages to the other > one. Considering some of my hard limits on this project (existing > stuff I have to work with, this is not being designed from scratch), > that's not acceptable for me. I'd have to come up with something else, > and not use edje Lua. > > -- > A big old stinking pile of genius that no one wants > coz there are too many silver coated monkeys in the world. -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- The Rasterman (Carsten Haitzler) [email protected] ------------------------------------------------------------------------------ Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox _______________________________________________ enlightenment-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
