On Mon, 5 Mar 2018 11:00:27 -0300 Gustavo Sverzut Barbieri <barbi...@gmail.com> said:
> On Sat, Mar 3, 2018 at 2:02 AM, Carsten Haitzler <ras...@rasterman.com> wrote: > > I just pushed: > > > > eb0b826776b60e0d97218242a5c285d146fb6f3b > > > > https://git.enlightenment.org/core/efl.git/commit/?id=1bdd9e4dd15fc27da43b50fd29bfb1b0b30ef6bd > > > > I wrote up a high level design document and description here for an idea of > > how it works: > > > > https://phab.enlightenment.org/w/efl-loops-threads/ > > > > I'm busy filling that document out a bit, but the core essentials are there. > > > > There are some details like loop having an exit eina value vs task having a > > simple int exit code (i'm sticking to this because it is what processes do, > > it's simple and its universally supported between processes, unlike an eina > > value). yes - it simplifies threads to only having int exit codes, but the > > simplicity of the design is what I'm going for. threads like executeables > > have full bytestream I/O for more complex data interchange. So there > > probably needs to be a bit of adjusting here and there to remove > > duplication. The Arguments event delivers arguments in an array, but the > > task object also stores arguments too permanently. Do we need to double-up > > the information? So a few small things like this. > > > > I've given this design a lot of thought and what I have here I think is > > clean and neat, tidy and fairly simple. It actually does work. I have > > tested it of course. But please have a look and let me know what you think. > > Are there some major defects in the design and idea? I know we can expand > > this in future with more controls (I have no pause/resume controls in the > > task interface but there should be. For processes use SIGSTOP/CONT and for > > threads a co-operative request on the control line). The internals could be > > better. I use pipes and this eats up a lot of FD's for the threads where I > > could use socketpairs instead. I have a separate control pipe in/out from > > I/O in/out where i could multiplex on the same socketpair. Currently we run > > out of FD's (after about 240 threads running at once because they eat up 8 > > FD's per thread. The Efl.Io code across the classes is a lot of copy & > > paste... And I don't have anything to change priority of a thread once it > > has run (no eina API for this - needs to be added). > > > > I'm not that happy though with Efl.Io and the sheer amount of code needed to > > deal with it. Even as a user of the API. > > > > Anyway - comments, thoughts, etc. etc. ? > > Well, I already mentioned this to you in irc, but replying here just > to make my point: > > I think the design is upside down, trying to make life easier at some > point resulted in messy at the other side. how? where? how is it messy? > okay, call it a task, work for both process and threads, the hope is > to facilitate "switch from threads to process, and vice versa", but > we're getting the worst part of each it seems. > > ie: most thread usage is about sending shared memory back and forth, > after all the benefit of threads is just that: shared memory. What we > got? argv + int, useless for real life threads. i did this because not only does this work between processes it can work for js and lua. lua can't share objects between threads. the best youcan do is a luastate per thread and that pretty much means treating threads as another process. the model of at the base terating threads like processes is more widely applicable than "pass share objects" because several languages/runtimes just can't do it at all. > solution? "stdio" for threads... you can send/receive I/O, done with > pipes, like it would be for processes. it's the same thing ecore_thread does but only one way (from thread to main loop) and only in a limited (only main loop can spawn threads". so this is a far more expanded model with data going the other way too. > I really, really don't like that design. To me threads should be > exposed as such, with pointer argument and result... if you wish to > make it easier, you could "box" it someway, like eina_value... or just i did the above because it's going to universally work with things like lua etc.. there needs to be some kind of hook to create a new luastate in the thread before executing, but then it'd work as-is. > let the traditional "void *pointer". But that's fundamental, say for > Python bindings to use threads, you want to send PyObject pointer to > the thread, that object will contain the information to callback the > user (the user's Python-callable). And for most usages as well, like > sending file info to be opened/decoded, etc. i can easily add a "set a void ptr on the thread" and make it gettable from the appthread. it wont work with lua though. it can't work with exe's. it's kind of not that useful for python etc. because it's just a void ptr. > thread I/O, while useful in some cases, I don't see why they should > always exist. They should be optional, and as such they should be > created explicitly. the control pipes do need to exist to handle joining the threads - knowing when they exit (and to request) but the stdio i can make optional with flags like i did with exe's. i might move those flags to the task class out of the exe class then. i don't think explicit creation is necessary or even good. some flags should do the trick just fine. > Last but not least, I'd expose the stack just like in the OS, to not > make confusion: > > - Application ("the binary") -> Processes -> Threads -> main loop > > you can present "getters" that make it easier to access (ie: main loop > "get" application, in addition to "get parent" which would return the > thread). i already mentioned this. we can't. app object would have to be a shared object. it then CANT have a non-shared child. it's a logical problem that a locked object can't then modify non-locked objects (parents modify children). but parents already have handles to child threads and can control them (co-operatively though efl takes care of being co-operative). > But mixing stuff "in the hope to make it easier" it not, it's just > making things more complicated... ALSO note that the developer that > would use this kind of API is "not the average developer", these don't > mess with such low level primitives. The developer that is likely to > use these primitives will say "what the fuck is this? I'm used to > processes and threads, these guys just messed it up". i disagree. i've seen the use of threads pretty much because "your annual bonus depends on you using more threads" and then madness that ends up as a result. making threads easier to access and communicate with and deal with is an important thing. hell - i want to make it easy for me too. ecore_thread is kind of horrible IMHO because of its major limitations in this regard. this fixes many of those aspects. > I'm in favor of interfaces for things that are the same, so if the > methods in process and threads share the same concept, behavior and > parameters, make them an interface... when switching from process x > threads one doesn't need to "sed" everything. However, definitely > constructors are NOT the same concept, behavior or parameters, thus > not part of the interface. to me threads and processes are the same with only a few minor differences: 1. a crash in a thread brings down everything, but in a process only brings itself down 2. threads by default share memory space (but accessing shared memory needs to be carefully controlled) 3. threads can't be sensibly killed off without side-effects for everyone else i can easily add a void * input and return in addition to args/exit code. i like the args/exit code and environment being universal because if you stick to this subset then things are portable between languages, runtimes and between threads and processes. i can add flags to explicitly enable i/o. what i haven't figured out yet or done is things like ecore_thread_async_call() (and a sync version) which of course isn't going to work for lua, but technically that's what the command pipe is for. i'll have to make it non-blocking though to use more than it is now just for exit and exited commands. so i'll add void ptrs for in/out in the thread class. i'll move i/o flags to task class from exe and use them to determine i/o then this covers your points i believe. in addition it still does args and so on like exe's. > > -- > Gustavo Sverzut Barbieri > -------------------------------------- > Mobile: +55 (16) 99354-9890 > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > enlightenment-devel mailing list > enlightenment-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- Carsten Haitzler - ras...@rasterman.com ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
