On Wed, 13 Feb 2013 18:18:24 +1000 David Seikel <[email protected]> said:
> On Wed, 13 Feb 2013 16:14:08 +0900 Carsten Haitzler (The Rasterman) > <[email protected]> wrote: > > > On Sun, 3 Feb 2013 13:34:35 +1000 David Seikel <[email protected]> > > said: > > > > > One year later ... > > > > > > On Wed, 18 Jan 2012 21:26:55 +1000 David Seikel <[email protected]> > > > wrote: > > > > > > > On Wed, 18 Jan 2012 18:36:21 +0900 Carsten Haitzler (The > > > > Rasterman) <[email protected]> wrote: > > > > > > > > > get onto irc and i'll explain... i feel i'm repeating myself. > > > > > > > > After repeating ourselves on IRC once more, we came up with a > > > > workable plan that might make everyone happy. > > > > > > > > > > > > First step is to write edje functions for generically bundling up > > > > one or more Lua tables into a message, sending that message, then > > > > unpacking it again. This will be able to be done from edje, from > > > > C, and from Lua directly. Perhaps adding that as one more edje > > > > messege type. This is for sending Lua tables between threads. A > > > > later addition will be to send them through the 'net, probably as > > > > eet. > > > > > > > > Host apps can register arbitrary functions with edje Lua, using a > > > > more generic version of the code I already wrote for letting edje > > > > register lua functions. These host functions are by default not > > > > thread safe. Edje puts a wrapper function around the host app, > > > > and registers that wrapper function with Lua. The wrapper > > > > function, when called from Lua, does this - > > > > > > > > ecore_thread_main_loop_begin(); > > > > call the host app callback(); > > > > ecore_thread_main_loop_end(); > > > > > > > > The first alternative, which the host app must request, is for the > > > > wrapper function to use the table message functions to marshal the > > > > arguments, send it to the main thread, wait for the response (or > > > > do something else), then unmarshal the result before sending it > > > > back to Lua. > > > > > > > > The second alternative, which the host app must REALLY request, > > > > is for the host app to say "I'm REALLY going out of my way to be > > > > threadsafe, just call me direct". No edje wrapper function, BUT > > > > the host app still has to use edje to register this function. > > > > > > > > The first two might be done this way - > > > > > > > > host_cb = edje_lua2_functions_add(...); > > > > edje_lua2_function_marshal_set(host_cb, function); > > > > > > > > The last one could be - > > > > > > > > host_cb = edje_lua2_threadsafe_function_add(...); > > > > > > > > Note the difference between _functions_ and _function_. The first > > > > call registers an entire metatable full of functions, in the same > > > > way that edje does for it's functions. These are the default > > > > sort of functions. The second call references one of those > > > > previously registered functions, and makes it marshal arguments > > > > and results. The third one registers a single function, but it > > > > could be added to an existing metatable registered by the first > > > > function. > > > > > > > > Comments? > > > > > > No one actually commented on this. I suspect that most don't care, > > > and those that care figured raster and I had discussed it to death > > > and came up with the right solution. > > > > > > Right now is time for me to consider starting to implement this. > > > I'm looking at the design of the next step in my virtual world > > > projects, and I think this will come in handy. That's why I was > > > thinking about doing it before. Now it's time to put my money > > > where my mouth is. > > > > > > I'll think about it some more, with respect to the virtual world > > > stuff I want to implement now, and that I have already done. See > > > if it still fits into my plans, and how to actually go about it > > > all. Then I'll report here and get stuck into more Edje Lua code > > > writing. > > > > > > > BTW, since in my current project I'll be writing code to use > > > > luaproc for threading thousands of Lua scripts using worker > > > > threads and a queue, I can probably incorporate that into edje > > > > Lua to begin with. > > > > > > This part has been done in that project I mentioned. What I > > > actually ended up doing was to basically port luaproc to EFL. This > > > allowed me to throw away more than two thirds of luaproc source > > > code. Most of what was thrown away was just luaproc's own > > > implementations of stuff that EFL implemented anyway. The rest was > > > simplifying luaproc for my use case. > > > > > > If I remember correctly, Raster wanted to thread more parts of > > > Edje. I might be able to bring this work into Edje Lua such that > > > the Edje Lua scripts are threaded in a worker queue. When combined > > > with the above table marshalling additions, plus using a few more > > > things that have been added to EFL, I suspect I can get half of the > > > remaining code to vanish as well. It's not a lot of code now, 15 > > > KB. > > > > > > To top it all off, I'll likely be bringing my LuaJIT work to Edje > > > Lua as well. Not sure yet how to go about that, but I think it > > > would be preferable to have that as a compile time option. If > > > LuaJIT is available, compile for it, otherwise compile for ordinary > > > LUa. See my notes elsewhere for how much SPEEEEED LuaJIT brings. > > > B-) > > > > i like the idea of luajit. like. > > OK, I'll bring that in. > > > as for thready stuff - i'd like to see the lua stuff divorced of > > specifically having to be synchronous with respect to the app - thus > > the bundle tables up as ipc to me seems the best move imho. the only > > problem is if you do everything as ipc, a lot of stuff that is simple > > sync "get x" "set y" becomes pretty heavy or complex. ayns messages > > as tables sounds perfect at the higher levels. tho. > > The luaproc stuff I mentioned is specifically for running thousands of > Lua scripts spread across the CPUs, but it works fine for a handful as > well. It uses queues of Lua scripts, message passing, and worker > threads that peel off a ready to run Lua script, run it until it reaches > some sort of pause point, typically when it stops to wait for a > message, or sends one, then pauses that script and works on the next > one. So it naturally runs in threads outside the mainloop. > > I'm still thinking about my options for dealing with runaway scripts. > A few ways I can go with that. > > As we discussed before, I'm also very concerned about making simple > stuff too heavy. Performance is important to me. The virtual world > stuff I'm doing has to send messages to a part of the server to deal > with the in world objects, and get messages from that part saying what > the objects are doing. The part running the scripts has to interface > with that server part. So there are similar goals here, getting the > messages dealt with in a sorta real time way, without soaking up lots of > resources. > > > i'm wondering if perhaps it'd be good to have your lua "split". most > > of it runs in a thread totally async. you can create some "inline > > workers" that can process async message data inline in the mainloop > > and then do sync stuff like move/resize objects etc.? just thinking > > out loud here. this means "you have to be careful" and we offer async > > message based manipulation of objects from the thread anyway by > > default (and encourage that) - eg u can set a message "table" that is > > a table array of object id's PLUS properties to set (position, size, > > color, image file etc.) and edje has a built in evaluator that in the > > mainloop "atomically" implements the entire message blob > > (create/delete/modify a whole set of objects and properties). so you > > send just 1 message across and then can manipulate 80 objects in one > > go? the above split code with some lua in the mainloop would be a > > custom "hand written" evaluator done in lua. ? am i being too > > paranoid about not having lua block the mainloop accidentally (or > > maliciously)? > > Yes you are being too paranoid. lol > > One of my use cases involves running thousands of Lua scripts (and LSL > scripts compiled to Lua) written by random people, client and server > side. Including people experimenting with scripts in the client, where > it's fine if the script crashes, but bad if it brings down anything > else. I have worked in virtual worlds where part of my job is to track > down and deal with bad scripts (stupidity usually, but sometimes malice > involved). So I like to think I'm being paranoid enough already from > experience. B-) > > I've been trying to experiment with the eo stuff, so that I can try to > leverage it's power for Lua and other things. That might work for > sending a bunch of object manipulation messages in one go, but I've > not figured out eo enough yet. I don't think having this mainloop > worker being written in Lua is such a good idea for performance > reasons. If the messages end up being eo based messages, then we can > just dump them into eo from C. i dont think eo will help a lot here as u have to marshall stuff still as protocol... unless u mean marshalling eo op id's and params? > I'll keep playing with all of this, start bringing my LuaJIT & luaproc > work into edje, work on that table passing stuff we discussed long ago, > experimenting with eo and stuff, and eventually I'll come up with > something safe, efficient, and flexible. > > -- > A big old stinking pile of genius that no one wants > coz there are too many silver coated monkeys in the world. -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- The Rasterman (Carsten Haitzler) [email protected] ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ enlightenment-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
