Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, 26 Oct 2016 11:28:30 -0700 Cedric BAIL said: > On Tue, Oct 25, 2016 at 5:17 PM, Carsten Haitzler > wrote: > > > > why are you doing a symlink if its JUST a quicklaunch binary change (that > > statically compiles all of efl into it)? > > As said it is a multiple stage update path. First just a change to > quicklaunch, which should work by itself. Then we can switch from real > libevas.so.1 to a symlink to that binary. This allow us a progressive > try without breaking everyone right away and making it possible for > people/packager on other OS to try it first. so let's not do this quicklaunch executable thing. as per exchange with vincent. windows cant loadlibrary an executable and have it WORK. it can find symbols, but not run (unless we do some linker level work it seems like resolve some libc symbols etc.). let's just merge .so's to fewer .so's (or fewer dll's) and get compatibility working without any quicklaunch craziness. let us see how much better things get then. we haven't measured it so we don't know. this is portable with symlinks on *nix OR "empty DLL's" on windows. my suggestion is we ALWAYS build the empty dll's and have a post install hook that deletes the files and replaces them with symnlinks on *nix only. this way we exercise the windows compatibility path ALWAYS on build+install and then add a post install step to optimize for *nix. we could even make the post install step explicit like make install make install-symlink > > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > > symlinks TO this quicklaunch binary ... to retain binary compatibility. you > > cant have the .so's AND this ql binary both have the same set of symbols. > > asking for trouble there. > > It shouldn't be a problem thanks to DSO on most system. System without > DSO may be an issue. my point is we can't maintain .so's AND a quicklaunch binary with libs statically linked in. its one or the other. you build an app normally even as a PIE exec, it will INSIST on linking to libevas.so and when it links libevas.so it must find symbols evas_* etc. from there. same for every library. if we have these .so's AND we have a quicklaunch lib with things compiled/linked inside we have multiple evas libs and evas symbols.. one from libevas.so and one from the ql lib. the only way to do this is replace ALL .so's with symlinks to the quicklaunch binary. so its this OR a set of .so's. you can't have both. let's not jump all that way (which cant work on windows as already mentioned. it can be a quicklaunch dll or a quicklaunch exe we can find symbols from but cant run anything from). let's just work on merging our libs and make it portable. let's just get this done. this is more than enough work for now and likely will buy us a lot of improvements on its own. let's measure them and see. > > but to USe this binaries HAVE to be PIe and this makes for a non-portable > > pain in the arse of creating apps. which is why i did .so's for quicklaunch > > before.. because it also works with dll's on windows. PIE does not. not > > that i could find. > > Most system have some form of PIE due to the need to implement ALSR. > For windows, let follow up the discussion with Vincent. yes. but we cant have a combined exe AND dll. you can find symbols and data but not run code according to vincent (unless we ALSO do some custom linker work ourselves which i think is going way way way too far). > > if top, htop, ps, killall, etc. don't work out of the box, it isby everyone > > (except yours) definition... a nightmare. i've messed with this before and > > it does not work across all the tools because of where they source command > > (and that can change based on config and cmdline options too). > > As said, this is just a matter of alias to be documented. I don't > think it is so much of a big deal. see simotek's response. it's not. it's a dealbreaker and will be for the vast vast vast majority of distributors and users. it means a quicklaunch/zygote kind of setup is going to be special and custom always for SOME systems where people are willing to pay this price. well until all ps/top like tools use the fields that allow process renaming to work. if we have this tool MY suggestion is to not even do PIE. do one big NON-RELOCATABLE executable (i know it kills ASLR for efl and anything linked into it, but you could force it to be PIE if you want this). this executable inits up to some specific point then CAN fork. maybe it forks 1 or 2 "spares" that sit and wait for commands. before they can accept commands though they do do some "app local" init like connect to x or wayland servers etc. and THEN sit and wait. when tols to run they dlopen the target and find the efl_main() func and launch from there by calling it. THIS works on windows. it works on *nix. BUT the app HAS to be a dll or .so to work. yes it needs special changes in building of the app, BUT you HAVE to force PIE (change app builds) to have this work
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Thu, 27 Oct 2016 10:39:47 +1030 Simon Lees said: > > > On 10/27/2016 04:58 AM, Cedric BAIL wrote: > > On Tue, Oct 25, 2016 at 5:17 PM, Carsten Haitzler > > wrote: > >> On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: > >>> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler > >>> wrote: > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL > said: i've been mulling this. i followed the feedback so far. basically > 2 main things: > > 1. this is intended to improve performance. especially startup times. > 2. this is highly problematic in terms of portability and is going to > create lots of code and build paths. > >>> > >>> It won't create lots of code and build paths. In fact it doesn't even > >>> need two build paths. It just needs to uses all .la files for building > >>> quicklaunch and can be rolled in, in parallel to current system. Just > >>> the link of the quicklaunch binary would be different. Next step would > >>> be to make a build option that setup symlink to that binary. Finally > >>> that option would be the default, likely only afffecting the install > >>> process. I don't think that create a lots of code (There should be no > >>> code change) and build paths change at all. > >> > >> why are you doing a symlink if its JUST a quicklaunch binary change (that > >> statically compiles all of efl into it)? > > > > As said it is a multiple stage update path. First just a change to > > quicklaunch, which should work by itself. Then we can switch from real > > libevas.so.1 to a symlink to that binary. This allow us a progressive > > try without breaking everyone right away and making it possible for > > people/packager on other OS to try it first. > > > >> you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > >> symlinks TO this quicklaunch binary ... to retain binary compatibility. > >> you cant have the .so's AND this ql binary both have the same set of > >> symbols. asking for trouble there. > > > > It shouldn't be a problem thanks to DSO on most system. System without > > DSO may be an issue. > > > >> but to USe this binaries HAVE to be PIe and this makes for a non-portable > >> pain in the arse of creating apps. which is why i did .so's for > >> quicklaunch before.. because it also works with dll's on windows. PIE does > >> not. not that i could find. > > > > Most system have some form of PIE due to the need to implement ALSR. > > For windows, let follow up the discussion with Vincent. > > > 3. this makes debugging and other things a nightmare (ps names are all > the same .. well depending on tool etc.) > >>> > >>> This is exagerated. Again just need to pass the right command line to > >>> ps/top and problem is solved. Can easily be documented and we can also > >>> provide a script that does it until we provide some nicer application > >>> there. > >> > >> i mean simple user debugging. someone runs ps or top and all they see is 20 > >> processes called "quicklaunch". because depending on the ps tool it may > >> use one proc field or another. so a user goes "quicklaunch process is > >> using N% cpu or N % ram". what real process is that? they don't know. > >> because procps tools will not do what you want and thus the average person > >> will become confused and find anything using efl like this to be horribly > >> hard to work with. > >> > >> if top, htop, ps, killall, etc. don't work out of the box, it isby everyone > >> (except yours) definition... a nightmare. i've messed with this before and > >> it does not work across all the tools because of where they source command > >> (and that can change based on config and cmdline options too). > > > > As said, this is just a matter of alias to be documented. I don't > > think it is so much of a big deal. > > > > As someone who helps with support a lot if top, htop, ps, killall and " > systemctl status user.slice" don't provide correct info out of the box > i'd consider it a deal breaker and broken and would probably look for > ways to patch around it at a distro level. bingo. thus why i dont think we could at all do a quicklaunch/zygote thing out of the box for everyone. i also see this as a deal breaker. if CURRENTLY 90%+ of ps/top/killall like tools used the same field AND thus showed the right cmd then ok - its a few that use a different field. this is NOT the case right now. it wont be unless these tools change. lots of them. -- - Codito, ergo sum - "I code, therefore I am" -- The Rasterman (Carsten Haitzler)ras...@rasterman.com -- The Command Line: Reinvented for Modern Developers Did the resurgence of CLI tooling catch you by surprise? Reconnect with the command line and become more productive. Learn the new .NET and ASP.NET CLI. Get your free copy! http://sdm.link/telerik ___ enlightenment-devel mailing list
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, 26 Oct 2016 07:14:05 +0200 Vincent Torri said: > On Wed, Oct 26, 2016 at 6:42 AM, Carsten Haitzler > wrote: > > On Wed, 26 Oct 2016 04:50:54 +0200 Vincent Torri > > said: > > > >> On Wed, Oct 26, 2016 at 2:17 AM, Carsten Haitzler > >> wrote: > >> > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL > >> > said: > >> > > >> >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler > >> >> wrote: > >> >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL > >> >> > said: > >> >> > > >> >> > i've been mulling this. i followed the feedback so far. basically 2 > >> >> > main things: > >> >> > > >> >> > 1. this is intended to improve performance. especially startup times. > >> >> > 2. this is highly problematic in terms of portability and is going to > >> >> > create lots of code and build paths. > >> >> > >> >> It won't create lots of code and build paths. In fact it doesn't even > >> >> need two build paths. It just needs to uses all .la files for building > >> >> quicklaunch and can be rolled in, in parallel to current system. Just > >> >> the link of the quicklaunch binary would be different. Next step would > >> >> be to make a build option that setup symlink to that binary. Finally > >> >> that option would be the default, likely only afffecting the install > >> >> process. I don't think that create a lots of code (There should be no > >> >> code change) and build paths change at all. > >> > > >> > why are you doing a symlink if its JUST a quicklaunch binary change (that > >> > statically compiles all of efl into it)? > >> > > >> > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > >> > symlinks TO this quicklaunch binary ... to retain binary compatibility. > >> > you cant have the .so's AND this ql binary both have the same set of > >> > symbols. asking for trouble there. > >> > > >> > but to USe this binaries HAVE to be PIe and this makes for a non-portable > >> > pain in the arse of creating apps. which is why i did .so's for > >> > quicklaunch before.. because it also works with dll's on windows. PIE > >> > does not. not that i could find. > >> > > >> >> > 3. this makes debugging and other things a nightmare (ps names are all > >> >> > the same .. well depending on tool etc.) > >> >> > >> >> This is exagerated. Again just need to pass the right command line to > >> >> ps/top and problem is solved. Can easily be documented and we can also > >> >> provide a script that does it until we provide some nicer application > >> >> there. > >> > > >> > i mean simple user debugging. someone runs ps or top and all they see is > >> > 20 processes called "quicklaunch". because depending on the ps tool it > >> > may use one proc field or another. so a user goes "quicklaunch process > >> > is using N% cpu or N % ram". what real process is that? they don't know. > >> > because procps tools will not do what you want and thus the average > >> > person will become confused and find anything using efl like this to be > >> > horribly hard to work with. > >> > > >> > if top, htop, ps, killall, etc. don't work out of the box, it isby > >> > everyone (except yours) definition... a nightmare. i've messed with this > >> > before and it does not work across all the tools because of where they > >> > source command (and that can change based on config and cmdline options > >> > too). > >> > > >> >> > 4. generally the issue of complexity in our build requires for > >> >> > portability etc. > >> >> > > >> >> > #4 isn't something we can solve right now, but has a better solution > >> >> > #if we go > >> >> > for a greater internal redesign. but not today. some other day. we > >> >> > have to live with what we have right now. > >> >> > >> >> Hum ? How ? I don't see how we can have good portability without a > >> >> complex build system. Would be interested to know more by what you > >> >> mean here ! > >> > > >> > we can isolate the portability layers in a much smaller area and always > >> > provide a portable interface wrapped on top of the real os one. currently > >> > our portability code is scattered all through efl. it's not in one > >> > place. by merging it into one location we'll be able to simplify it too > >> > so in the end we just have some "efl port sys" lib that provides an > >> > "always works" set of library calls and everything above that is 100% > >> > portable and never changes platform to platform. > >> > > >> >> > i am always for improvements. but this really makes this method a very > >> >> > very very hard ask to do this. i've thought about it and i think this > >> >> > is far TOO radical a change as it means different execution methods, > >> >> > library production, linking and so much more os by os. it's kind of > >> >> > insane though theoretically doable on *nix. i've checked and PIE > >> >> > seems to be urtterly unportable (windows can't do it based on my > >> >> > reading). the proposal would be workable ONLY if we could do it > >> >> > unversally so it wouldn't bitrot and it'd
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On 10/27/2016 04:58 AM, Cedric BAIL wrote: > On Tue, Oct 25, 2016 at 5:17 PM, Carsten Haitzler > wrote: >> On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: >>> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler >>> wrote: On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL said: i've been mulling this. i followed the feedback so far. basically 2 main things: 1. this is intended to improve performance. especially startup times. 2. this is highly problematic in terms of portability and is going to create lots of code and build paths. >>> >>> It won't create lots of code and build paths. In fact it doesn't even >>> need two build paths. It just needs to uses all .la files for building >>> quicklaunch and can be rolled in, in parallel to current system. Just >>> the link of the quicklaunch binary would be different. Next step would >>> be to make a build option that setup symlink to that binary. Finally >>> that option would be the default, likely only afffecting the install >>> process. I don't think that create a lots of code (There should be no >>> code change) and build paths change at all. >> >> why are you doing a symlink if its JUST a quicklaunch binary change (that >> statically compiles all of efl into it)? > > As said it is a multiple stage update path. First just a change to > quicklaunch, which should work by itself. Then we can switch from real > libevas.so.1 to a symlink to that binary. This allow us a progressive > try without breaking everyone right away and making it possible for > people/packager on other OS to try it first. > >> you are trying to replace all of libevas.so.1, libeina.so.1, etc. with >> symlinks >> TO this quicklaunch binary ... to retain binary compatibility. you cant have >> the .so's AND this ql binary both have the same set of symbols. asking for >> trouble there. > > It shouldn't be a problem thanks to DSO on most system. System without > DSO may be an issue. > >> but to USe this binaries HAVE to be PIe and this makes for a non-portable >> pain >> in the arse of creating apps. which is why i did .so's for quicklaunch >> before.. >> because it also works with dll's on windows. PIE does not. not that i could >> find. > > Most system have some form of PIE due to the need to implement ALSR. > For windows, let follow up the discussion with Vincent. > 3. this makes debugging and other things a nightmare (ps names are all the same .. well depending on tool etc.) >>> >>> This is exagerated. Again just need to pass the right command line to >>> ps/top and problem is solved. Can easily be documented and we can also >>> provide a script that does it until we provide some nicer application >>> there. >> >> i mean simple user debugging. someone runs ps or top and all they see is 20 >> processes called "quicklaunch". because depending on the ps tool it may use >> one >> proc field or another. so a user goes "quicklaunch process is using N% cpu >> or N >> % ram". what real process is that? they don't know. because procps tools will >> not do what you want and thus the average person will become confused and >> find >> anything using efl like this to be horribly hard to work with. >> >> if top, htop, ps, killall, etc. don't work out of the box, it isby everyone >> (except yours) definition... a nightmare. i've messed with this before and it >> does not work across all the tools because of where they source command (and >> that can change based on config and cmdline options too). > > As said, this is just a matter of alias to be documented. I don't > think it is so much of a big deal. > As someone who helps with support a lot if top, htop, ps, killall and " systemctl status user.slice" don't provide correct info out of the box i'd consider it a deal breaker and broken and would probably look for ways to patch around it at a distro level. -- Simon Lees (Simotek)http://simotek.net Emergency Update Team keybase.io/simotek SUSE LinuxAdeliade Australia, UTC+9:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B signature.asc Description: OpenPGP digital signature -- The Command Line: Reinvented for Modern Developers Did the resurgence of CLI tooling catch you by surprise? Reconnect with the command line and become more productive. Learn the new .NET and ASP.NET CLI. Get your free copy! http://sdm.link/telerik___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, Oct 26, 2016 at 11:40 AM, Vincent Torri wrote: > On Wed, Oct 26, 2016 at 7:42 PM, Cedric BAIL wrote: >> On Tue, Oct 25, 2016 at 10:14 PM, Vincent Torri >> wrote: >>> On Wed, Oct 26, 2016 at 6:42 AM, Carsten Haitzler >>> wrote: On Wed, 26 Oct 2016 04:50:54 +0200 Vincent Torri said: > On Wed, Oct 26, 2016 at 2:17 AM, Carsten Haitzler > wrote: > > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL > > said: > >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler > >> > >> wrote: > >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL > >> > said: > >> > > >> > i've been mulling this. i followed the feedback so far. basically 2 > >> > main > >> > things: > >> > > >> > 1. this is intended to improve performance. especially startup times. > >> > 2. this is highly problematic in terms of portability and is going to > >> > create lots of code and build paths. > >> > >> It won't create lots of code and build paths. In fact it doesn't even > >> need two build paths. It just needs to uses all .la files for building > >> quicklaunch and can be rolled in, in parallel to current system. Just > >> the link of the quicklaunch binary would be different. Next step would > >> be to make a build option that setup symlink to that binary. Finally > >> that option would be the default, likely only afffecting the install > >> process. I don't think that create a lots of code (There should be no > >> code change) and build paths change at all. > > > > why are you doing a symlink if its JUST a quicklaunch binary change > > (that > > statically compiles all of efl into it)? > > > > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > > symlinks TO this quicklaunch binary ... to retain binary compatibility. > > you > > cant have the .so's AND this ql binary both have the same set of > > symbols. > > asking for trouble there. > > > > but to USe this binaries HAVE to be PIe and this makes for a > > non-portable > > pain in the arse of creating apps. which is why i did .so's for > > quicklaunch > > before.. because it also works with dll's on windows. PIE does not. not > > that i could find. > > > >> > 3. this makes debugging and other things a nightmare (ps names are > >> > all > >> > the same .. well depending on tool etc.) > >> > >> This is exagerated. Again just need to pass the right command line to > >> ps/top and problem is solved. Can easily be documented and we can also > >> provide a script that does it until we provide some nicer application > >> there. > > > > i mean simple user debugging. someone runs ps or top and all they see > > is 20 > > processes called "quicklaunch". because depending on the ps tool it may > > use > > one proc field or another. so a user goes "quicklaunch process is using > > N% > > cpu or N % ram". what real process is that? they don't know. because > > procps > > tools will not do what you want and thus the average person will become > > confused and find anything using efl like this to be horribly hard to > > work > > with. > > > > if top, htop, ps, killall, etc. don't work out of the box, it isby > > everyone > > (except yours) definition... a nightmare. i've messed with this before > > and > > it does not work across all the tools because of where they source > > command > > (and that can change based on config and cmdline options too). > > > >> > 4. generally the issue of complexity in our build requires for > >> > portability etc. > >> > > >> > #4 isn't something we can solve right now, but has a better solution > >> > if > >> > #we go > >> > for a greater internal redesign. but not today. some other day. we > >> > have > >> > to live with what we have right now. > >> > >> Hum ? How ? I don't see how we can have good portability without a > >> complex build system. Would be interested to know more by what you > >> mean here ! > > > > we can isolate the portability layers in a much smaller area and always > > provide a portable interface wrapped on top of the real os one. > > currently > > our portability code is scattered all through efl. it's not in one > > place. by > > merging it into one location we'll be able to simplify it too so in the > > end > > we just have some "efl port sys" lib that provides an "always works" > > set of > > library calls and everything above that is 100% portable and never > > changes > > platform to platform. > > > >> > i am always for improvements. but this really makes this method a > >> > very > >> > very very hard ask to do this. i've thought about it and i think > >> > this
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, Oct 26, 2016 at 8:28 PM, Cedric BAIL wrote: > On Tue, Oct 25, 2016 at 5:17 PM, Carsten Haitzler > wrote: >> On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: >>> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler >>> wrote: >>> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL said: >>> > i've been mulling this. i followed the feedback so far. basically 2 main >>> > things: >>> > >>> > 1. this is intended to improve performance. especially startup times. >>> > 2. this is highly problematic in terms of portability and is going to >>> > create >>> > lots of code and build paths. >>> >>> It won't create lots of code and build paths. In fact it doesn't even >>> need two build paths. It just needs to uses all .la files for building >>> quicklaunch and can be rolled in, in parallel to current system. Just >>> the link of the quicklaunch binary would be different. Next step would >>> be to make a build option that setup symlink to that binary. Finally >>> that option would be the default, likely only afffecting the install >>> process. I don't think that create a lots of code (There should be no >>> code change) and build paths change at all. >> >> why are you doing a symlink if its JUST a quicklaunch binary change (that >> statically compiles all of efl into it)? > > As said it is a multiple stage update path. First just a change to > quicklaunch, which should work by itself. Then we can switch from real > libevas.so.1 to a symlink to that binary. This allow us a progressive > try without breaking everyone right away and making it possible for > people/packager on other OS to try it first. > >> you are trying to replace all of libevas.so.1, libeina.so.1, etc. with >> symlinks >> TO this quicklaunch binary ... to retain binary compatibility. you cant have >> the .so's AND this ql binary both have the same set of symbols. asking for >> trouble there. > > It shouldn't be a problem thanks to DSO on most system. System without > DSO may be an issue. > >> but to USe this binaries HAVE to be PIe and this makes for a non-portable >> pain >> in the arse of creating apps. which is why i did .so's for quicklaunch >> before.. >> because it also works with dll's on windows. PIE does not. not that i could >> find. > > Most system have some form of PIE due to the need to implement ALSR. > For windows, let follow up the discussion with Vincent. > >>> > 3. this makes debugging and other things a nightmare (ps names are all the >>> > same .. well depending on tool etc.) >>> >>> This is exagerated. Again just need to pass the right command line to >>> ps/top and problem is solved. Can easily be documented and we can also >>> provide a script that does it until we provide some nicer application >>> there. >> >> i mean simple user debugging. someone runs ps or top and all they see is 20 >> processes called "quicklaunch". because depending on the ps tool it may use >> one >> proc field or another. so a user goes "quicklaunch process is using N% cpu >> or N >> % ram". what real process is that? they don't know. because procps tools will >> not do what you want and thus the average person will become confused and >> find >> anything using efl like this to be horribly hard to work with. >> >> if top, htop, ps, killall, etc. don't work out of the box, it isby everyone >> (except yours) definition... a nightmare. i've messed with this before and it >> does not work across all the tools because of where they source command (and >> that can change based on config and cmdline options too). > > As said, this is just a matter of alias to be documented. I don't > think it is so much of a big deal. > >>> > 4. generally the issue of complexity in our build requires for portability >>> > etc. >>> > >>> > #4 isn't something we can solve right now, but has a better solution if we >>> > #go >>> > for a greater internal redesign. but not today. some other day. we have to >>> > live with what we have right now. >>> >>> Hum ? How ? I don't see how we can have good portability without a >>> complex build system. Would be interested to know more by what you >>> mean here ! >> >> we can isolate the portability layers in a much smaller area and always >> provide >> a portable interface wrapped on top of the real os one. currently our >> portability code is scattered all through efl. it's not in one place. by >> merging it into one location we'll be able to simplify it too so in the end >> we >> just have some "efl port sys" lib that provides an "always works" set of >> library calls and everything above that is 100% portable and never changes >> platform to platform. > > We kind of have that and kind of not. The issue is that depending on > how far from an Unix system an OS is the deeper the change are. > Windows require a base layer like Evil to handle things, most RTOS > would also require something like that, while Mac OS X doesn't but > will require alternate code for display, input, c&p and drag&drop. > Moving everything down t
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, Oct 26, 2016 at 7:42 PM, Cedric BAIL wrote: > On Tue, Oct 25, 2016 at 10:14 PM, Vincent Torri > wrote: >> On Wed, Oct 26, 2016 at 6:42 AM, Carsten Haitzler >> wrote: >>> On Wed, 26 Oct 2016 04:50:54 +0200 Vincent Torri >>> said: On Wed, Oct 26, 2016 at 2:17 AM, Carsten Haitzler wrote: > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL > said: >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler >> >> wrote: >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL >> > said: >> > >> > i've been mulling this. i followed the feedback so far. basically 2 >> > main >> > things: >> > >> > 1. this is intended to improve performance. especially startup times. >> > 2. this is highly problematic in terms of portability and is going to >> > create lots of code and build paths. >> >> It won't create lots of code and build paths. In fact it doesn't even >> need two build paths. It just needs to uses all .la files for building >> quicklaunch and can be rolled in, in parallel to current system. Just >> the link of the quicklaunch binary would be different. Next step would >> be to make a build option that setup symlink to that binary. Finally >> that option would be the default, likely only afffecting the install >> process. I don't think that create a lots of code (There should be no >> code change) and build paths change at all. > > why are you doing a symlink if its JUST a quicklaunch binary change (that > statically compiles all of efl into it)? > > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > symlinks TO this quicklaunch binary ... to retain binary compatibility. > you > cant have the .so's AND this ql binary both have the same set of symbols. > asking for trouble there. > > but to USe this binaries HAVE to be PIe and this makes for a non-portable > pain in the arse of creating apps. which is why i did .so's for > quicklaunch > before.. because it also works with dll's on windows. PIE does not. not > that i could find. > >> > 3. this makes debugging and other things a nightmare (ps names are all >> > the same .. well depending on tool etc.) >> >> This is exagerated. Again just need to pass the right command line to >> ps/top and problem is solved. Can easily be documented and we can also >> provide a script that does it until we provide some nicer application >> there. > > i mean simple user debugging. someone runs ps or top and all they see is > 20 > processes called "quicklaunch". because depending on the ps tool it may > use > one proc field or another. so a user goes "quicklaunch process is using > N% > cpu or N % ram". what real process is that? they don't know. because > procps > tools will not do what you want and thus the average person will become > confused and find anything using efl like this to be horribly hard to > work > with. > > if top, htop, ps, killall, etc. don't work out of the box, it isby > everyone > (except yours) definition... a nightmare. i've messed with this before > and > it does not work across all the tools because of where they source > command > (and that can change based on config and cmdline options too). > >> > 4. generally the issue of complexity in our build requires for >> > portability etc. >> > >> > #4 isn't something we can solve right now, but has a better solution >> > if >> > #we go >> > for a greater internal redesign. but not today. some other day. we >> > have >> > to live with what we have right now. >> >> Hum ? How ? I don't see how we can have good portability without a >> complex build system. Would be interested to know more by what you >> mean here ! > > we can isolate the portability layers in a much smaller area and always > provide a portable interface wrapped on top of the real os one. currently > our portability code is scattered all through efl. it's not in one > place. by > merging it into one location we'll be able to simplify it too so in the > end > we just have some "efl port sys" lib that provides an "always works" set > of > library calls and everything above that is 100% portable and never > changes > platform to platform. > >> > i am always for improvements. but this really makes this method a very >> > very very hard ask to do this. i've thought about it and i think this >> > is >> > far TOO radical a change as it means different execution methods, >> > library >> > production, linking and so much more os by os. it's kind of insane >> > though >> > theoretically doable
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Tue, Oct 25, 2016 at 5:17 PM, Carsten Haitzler wrote: > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler >> wrote: >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL said: >> > i've been mulling this. i followed the feedback so far. basically 2 main >> > things: >> > >> > 1. this is intended to improve performance. especially startup times. >> > 2. this is highly problematic in terms of portability and is going to >> > create >> > lots of code and build paths. >> >> It won't create lots of code and build paths. In fact it doesn't even >> need two build paths. It just needs to uses all .la files for building >> quicklaunch and can be rolled in, in parallel to current system. Just >> the link of the quicklaunch binary would be different. Next step would >> be to make a build option that setup symlink to that binary. Finally >> that option would be the default, likely only afffecting the install >> process. I don't think that create a lots of code (There should be no >> code change) and build paths change at all. > > why are you doing a symlink if its JUST a quicklaunch binary change (that > statically compiles all of efl into it)? As said it is a multiple stage update path. First just a change to quicklaunch, which should work by itself. Then we can switch from real libevas.so.1 to a symlink to that binary. This allow us a progressive try without breaking everyone right away and making it possible for people/packager on other OS to try it first. > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > symlinks > TO this quicklaunch binary ... to retain binary compatibility. you cant have > the .so's AND this ql binary both have the same set of symbols. asking for > trouble there. It shouldn't be a problem thanks to DSO on most system. System without DSO may be an issue. > but to USe this binaries HAVE to be PIe and this makes for a non-portable pain > in the arse of creating apps. which is why i did .so's for quicklaunch > before.. > because it also works with dll's on windows. PIE does not. not that i could > find. Most system have some form of PIE due to the need to implement ALSR. For windows, let follow up the discussion with Vincent. >> > 3. this makes debugging and other things a nightmare (ps names are all the >> > same .. well depending on tool etc.) >> >> This is exagerated. Again just need to pass the right command line to >> ps/top and problem is solved. Can easily be documented and we can also >> provide a script that does it until we provide some nicer application >> there. > > i mean simple user debugging. someone runs ps or top and all they see is 20 > processes called "quicklaunch". because depending on the ps tool it may use > one > proc field or another. so a user goes "quicklaunch process is using N% cpu or > N > % ram". what real process is that? they don't know. because procps tools will > not do what you want and thus the average person will become confused and find > anything using efl like this to be horribly hard to work with. > > if top, htop, ps, killall, etc. don't work out of the box, it isby everyone > (except yours) definition... a nightmare. i've messed with this before and it > does not work across all the tools because of where they source command (and > that can change based on config and cmdline options too). As said, this is just a matter of alias to be documented. I don't think it is so much of a big deal. >> > 4. generally the issue of complexity in our build requires for portability >> > etc. >> > >> > #4 isn't something we can solve right now, but has a better solution if we >> > #go >> > for a greater internal redesign. but not today. some other day. we have to >> > live with what we have right now. >> >> Hum ? How ? I don't see how we can have good portability without a >> complex build system. Would be interested to know more by what you >> mean here ! > > we can isolate the portability layers in a much smaller area and always > provide > a portable interface wrapped on top of the real os one. currently our > portability code is scattered all through efl. it's not in one place. by > merging it into one location we'll be able to simplify it too so in the end we > just have some "efl port sys" lib that provides an "always works" set of > library calls and everything above that is 100% portable and never changes > platform to platform. We kind of have that and kind of not. The issue is that depending on how far from an Unix system an OS is the deeper the change are. Windows require a base layer like Evil to handle things, most RTOS would also require something like that, while Mac OS X doesn't but will require alternate code for display, input, c&p and drag&drop. Moving everything down to a very low level means that even for Unix we need to abstract all this things into a lower layer. I am not to sure of the benefit and that it would work. Can we make xlib call looks lik
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Tue, Oct 25, 2016 at 10:14 PM, Vincent Torri wrote: > On Wed, Oct 26, 2016 at 6:42 AM, Carsten Haitzler > wrote: >> On Wed, 26 Oct 2016 04:50:54 +0200 Vincent Torri >> said: >>> On Wed, Oct 26, 2016 at 2:17 AM, Carsten Haitzler >>> wrote: >>> > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: >>> >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler >>> >> wrote: >>> >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL >>> >> > said: >>> >> > >>> >> > i've been mulling this. i followed the feedback so far. basically 2 >>> >> > main >>> >> > things: >>> >> > >>> >> > 1. this is intended to improve performance. especially startup times. >>> >> > 2. this is highly problematic in terms of portability and is going to >>> >> > create lots of code and build paths. >>> >> >>> >> It won't create lots of code and build paths. In fact it doesn't even >>> >> need two build paths. It just needs to uses all .la files for building >>> >> quicklaunch and can be rolled in, in parallel to current system. Just >>> >> the link of the quicklaunch binary would be different. Next step would >>> >> be to make a build option that setup symlink to that binary. Finally >>> >> that option would be the default, likely only afffecting the install >>> >> process. I don't think that create a lots of code (There should be no >>> >> code change) and build paths change at all. >>> > >>> > why are you doing a symlink if its JUST a quicklaunch binary change (that >>> > statically compiles all of efl into it)? >>> > >>> > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with >>> > symlinks TO this quicklaunch binary ... to retain binary compatibility. >>> > you >>> > cant have the .so's AND this ql binary both have the same set of symbols. >>> > asking for trouble there. >>> > >>> > but to USe this binaries HAVE to be PIe and this makes for a non-portable >>> > pain in the arse of creating apps. which is why i did .so's for >>> > quicklaunch >>> > before.. because it also works with dll's on windows. PIE does not. not >>> > that i could find. >>> > >>> >> > 3. this makes debugging and other things a nightmare (ps names are all >>> >> > the same .. well depending on tool etc.) >>> >> >>> >> This is exagerated. Again just need to pass the right command line to >>> >> ps/top and problem is solved. Can easily be documented and we can also >>> >> provide a script that does it until we provide some nicer application >>> >> there. >>> > >>> > i mean simple user debugging. someone runs ps or top and all they see is >>> > 20 >>> > processes called "quicklaunch". because depending on the ps tool it may >>> > use >>> > one proc field or another. so a user goes "quicklaunch process is using N% >>> > cpu or N % ram". what real process is that? they don't know. because >>> > procps >>> > tools will not do what you want and thus the average person will become >>> > confused and find anything using efl like this to be horribly hard to work >>> > with. >>> > >>> > if top, htop, ps, killall, etc. don't work out of the box, it isby >>> > everyone >>> > (except yours) definition... a nightmare. i've messed with this before and >>> > it does not work across all the tools because of where they source command >>> > (and that can change based on config and cmdline options too). >>> > >>> >> > 4. generally the issue of complexity in our build requires for >>> >> > portability etc. >>> >> > >>> >> > #4 isn't something we can solve right now, but has a better solution if >>> >> > #we go >>> >> > for a greater internal redesign. but not today. some other day. we have >>> >> > to live with what we have right now. >>> >> >>> >> Hum ? How ? I don't see how we can have good portability without a >>> >> complex build system. Would be interested to know more by what you >>> >> mean here ! >>> > >>> > we can isolate the portability layers in a much smaller area and always >>> > provide a portable interface wrapped on top of the real os one. currently >>> > our portability code is scattered all through efl. it's not in one place. >>> > by >>> > merging it into one location we'll be able to simplify it too so in the >>> > end >>> > we just have some "efl port sys" lib that provides an "always works" set >>> > of >>> > library calls and everything above that is 100% portable and never changes >>> > platform to platform. >>> > >>> >> > i am always for improvements. but this really makes this method a very >>> >> > very very hard ask to do this. i've thought about it and i think this >>> >> > is >>> >> > far TOO radical a change as it means different execution methods, >>> >> > library >>> >> > production, linking and so much more os by os. it's kind of insane >>> >> > though >>> >> > theoretically doable on *nix. i've checked and PIE seems to be urtterly >>> >> > unportable (windows can't do it based on my reading). the proposal >>> >> > would >>> >> > be workable ONLY if we could do it unversally so it wouldn't bitrot and >>> >> > it'd
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, Oct 26, 2016 at 6:42 AM, Carsten Haitzler wrote: > On Wed, 26 Oct 2016 04:50:54 +0200 Vincent Torri > said: > >> On Wed, Oct 26, 2016 at 2:17 AM, Carsten Haitzler >> wrote: >> > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: >> > >> >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler >> >> wrote: >> >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL >> >> > said: >> >> > >> >> > i've been mulling this. i followed the feedback so far. basically 2 main >> >> > things: >> >> > >> >> > 1. this is intended to improve performance. especially startup times. >> >> > 2. this is highly problematic in terms of portability and is going to >> >> > create lots of code and build paths. >> >> >> >> It won't create lots of code and build paths. In fact it doesn't even >> >> need two build paths. It just needs to uses all .la files for building >> >> quicklaunch and can be rolled in, in parallel to current system. Just >> >> the link of the quicklaunch binary would be different. Next step would >> >> be to make a build option that setup symlink to that binary. Finally >> >> that option would be the default, likely only afffecting the install >> >> process. I don't think that create a lots of code (There should be no >> >> code change) and build paths change at all. >> > >> > why are you doing a symlink if its JUST a quicklaunch binary change (that >> > statically compiles all of efl into it)? >> > >> > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with >> > symlinks TO this quicklaunch binary ... to retain binary compatibility. you >> > cant have the .so's AND this ql binary both have the same set of symbols. >> > asking for trouble there. >> > >> > but to USe this binaries HAVE to be PIe and this makes for a non-portable >> > pain in the arse of creating apps. which is why i did .so's for quicklaunch >> > before.. because it also works with dll's on windows. PIE does not. not >> > that i could find. >> > >> >> > 3. this makes debugging and other things a nightmare (ps names are all >> >> > the same .. well depending on tool etc.) >> >> >> >> This is exagerated. Again just need to pass the right command line to >> >> ps/top and problem is solved. Can easily be documented and we can also >> >> provide a script that does it until we provide some nicer application >> >> there. >> > >> > i mean simple user debugging. someone runs ps or top and all they see is 20 >> > processes called "quicklaunch". because depending on the ps tool it may use >> > one proc field or another. so a user goes "quicklaunch process is using N% >> > cpu or N % ram". what real process is that? they don't know. because procps >> > tools will not do what you want and thus the average person will become >> > confused and find anything using efl like this to be horribly hard to work >> > with. >> > >> > if top, htop, ps, killall, etc. don't work out of the box, it isby everyone >> > (except yours) definition... a nightmare. i've messed with this before and >> > it does not work across all the tools because of where they source command >> > (and that can change based on config and cmdline options too). >> > >> >> > 4. generally the issue of complexity in our build requires for >> >> > portability etc. >> >> > >> >> > #4 isn't something we can solve right now, but has a better solution if >> >> > #we go >> >> > for a greater internal redesign. but not today. some other day. we have >> >> > to live with what we have right now. >> >> >> >> Hum ? How ? I don't see how we can have good portability without a >> >> complex build system. Would be interested to know more by what you >> >> mean here ! >> > >> > we can isolate the portability layers in a much smaller area and always >> > provide a portable interface wrapped on top of the real os one. currently >> > our portability code is scattered all through efl. it's not in one place. >> > by >> > merging it into one location we'll be able to simplify it too so in the end >> > we just have some "efl port sys" lib that provides an "always works" set of >> > library calls and everything above that is 100% portable and never changes >> > platform to platform. >> > >> >> > i am always for improvements. but this really makes this method a very >> >> > very very hard ask to do this. i've thought about it and i think this is >> >> > far TOO radical a change as it means different execution methods, >> >> > library >> >> > production, linking and so much more os by os. it's kind of insane >> >> > though >> >> > theoretically doable on *nix. i've checked and PIE seems to be urtterly >> >> > unportable (windows can't do it based on my reading). the proposal would >> >> > be workable ONLY if we could do it unversally so it wouldn't bitrot and >> >> > it'd work. this is actually why quicklaunch did .so's instead of PIE >> >> > because that WORKS portably >> >> >> >> Windows is anyway another platform with a lot of specificity. >> >> Quicklaunch will never work there as there is
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, 26 Oct 2016 04:50:54 +0200 Vincent Torri said: > On Wed, Oct 26, 2016 at 2:17 AM, Carsten Haitzler > wrote: > > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: > > > >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler > >> wrote: > >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL > >> > said: > >> > > >> > i've been mulling this. i followed the feedback so far. basically 2 main > >> > things: > >> > > >> > 1. this is intended to improve performance. especially startup times. > >> > 2. this is highly problematic in terms of portability and is going to > >> > create lots of code and build paths. > >> > >> It won't create lots of code and build paths. In fact it doesn't even > >> need two build paths. It just needs to uses all .la files for building > >> quicklaunch and can be rolled in, in parallel to current system. Just > >> the link of the quicklaunch binary would be different. Next step would > >> be to make a build option that setup symlink to that binary. Finally > >> that option would be the default, likely only afffecting the install > >> process. I don't think that create a lots of code (There should be no > >> code change) and build paths change at all. > > > > why are you doing a symlink if its JUST a quicklaunch binary change (that > > statically compiles all of efl into it)? > > > > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > > symlinks TO this quicklaunch binary ... to retain binary compatibility. you > > cant have the .so's AND this ql binary both have the same set of symbols. > > asking for trouble there. > > > > but to USe this binaries HAVE to be PIe and this makes for a non-portable > > pain in the arse of creating apps. which is why i did .so's for quicklaunch > > before.. because it also works with dll's on windows. PIE does not. not > > that i could find. > > > >> > 3. this makes debugging and other things a nightmare (ps names are all > >> > the same .. well depending on tool etc.) > >> > >> This is exagerated. Again just need to pass the right command line to > >> ps/top and problem is solved. Can easily be documented and we can also > >> provide a script that does it until we provide some nicer application > >> there. > > > > i mean simple user debugging. someone runs ps or top and all they see is 20 > > processes called "quicklaunch". because depending on the ps tool it may use > > one proc field or another. so a user goes "quicklaunch process is using N% > > cpu or N % ram". what real process is that? they don't know. because procps > > tools will not do what you want and thus the average person will become > > confused and find anything using efl like this to be horribly hard to work > > with. > > > > if top, htop, ps, killall, etc. don't work out of the box, it isby everyone > > (except yours) definition... a nightmare. i've messed with this before and > > it does not work across all the tools because of where they source command > > (and that can change based on config and cmdline options too). > > > >> > 4. generally the issue of complexity in our build requires for > >> > portability etc. > >> > > >> > #4 isn't something we can solve right now, but has a better solution if > >> > #we go > >> > for a greater internal redesign. but not today. some other day. we have > >> > to live with what we have right now. > >> > >> Hum ? How ? I don't see how we can have good portability without a > >> complex build system. Would be interested to know more by what you > >> mean here ! > > > > we can isolate the portability layers in a much smaller area and always > > provide a portable interface wrapped on top of the real os one. currently > > our portability code is scattered all through efl. it's not in one place. by > > merging it into one location we'll be able to simplify it too so in the end > > we just have some "efl port sys" lib that provides an "always works" set of > > library calls and everything above that is 100% portable and never changes > > platform to platform. > > > >> > i am always for improvements. but this really makes this method a very > >> > very very hard ask to do this. i've thought about it and i think this is > >> > far TOO radical a change as it means different execution methods, library > >> > production, linking and so much more os by os. it's kind of insane though > >> > theoretically doable on *nix. i've checked and PIE seems to be urtterly > >> > unportable (windows can't do it based on my reading). the proposal would > >> > be workable ONLY if we could do it unversally so it wouldn't bitrot and > >> > it'd work. this is actually why quicklaunch did .so's instead of PIE > >> > because that WORKS portably > >> > >> Windows is anyway another platform with a lot of specificity. > >> Quicklaunch will never work there as there is no equivalent to fork > >> anyway. Emulating what PIE allow is also mostly impossible as > > > > actually it does work. fork works. it's emulated and expensive, bit it > > WORKS.
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Wed, Oct 26, 2016 at 2:17 AM, Carsten Haitzler wrote: > On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: > >> On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler >> wrote: >> > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL said: >> > >> > i've been mulling this. i followed the feedback so far. basically 2 main >> > things: >> > >> > 1. this is intended to improve performance. especially startup times. >> > 2. this is highly problematic in terms of portability and is going to >> > create >> > lots of code and build paths. >> >> It won't create lots of code and build paths. In fact it doesn't even >> need two build paths. It just needs to uses all .la files for building >> quicklaunch and can be rolled in, in parallel to current system. Just >> the link of the quicklaunch binary would be different. Next step would >> be to make a build option that setup symlink to that binary. Finally >> that option would be the default, likely only afffecting the install >> process. I don't think that create a lots of code (There should be no >> code change) and build paths change at all. > > why are you doing a symlink if its JUST a quicklaunch binary change (that > statically compiles all of efl into it)? > > you are trying to replace all of libevas.so.1, libeina.so.1, etc. with > symlinks > TO this quicklaunch binary ... to retain binary compatibility. you cant have > the .so's AND this ql binary both have the same set of symbols. asking for > trouble there. > > but to USe this binaries HAVE to be PIe and this makes for a non-portable pain > in the arse of creating apps. which is why i did .so's for quicklaunch > before.. > because it also works with dll's on windows. PIE does not. not that i could > find. > >> > 3. this makes debugging and other things a nightmare (ps names are all the >> > same .. well depending on tool etc.) >> >> This is exagerated. Again just need to pass the right command line to >> ps/top and problem is solved. Can easily be documented and we can also >> provide a script that does it until we provide some nicer application >> there. > > i mean simple user debugging. someone runs ps or top and all they see is 20 > processes called "quicklaunch". because depending on the ps tool it may use > one > proc field or another. so a user goes "quicklaunch process is using N% cpu or > N > % ram". what real process is that? they don't know. because procps tools will > not do what you want and thus the average person will become confused and find > anything using efl like this to be horribly hard to work with. > > if top, htop, ps, killall, etc. don't work out of the box, it isby everyone > (except yours) definition... a nightmare. i've messed with this before and it > does not work across all the tools because of where they source command (and > that can change based on config and cmdline options too). > >> > 4. generally the issue of complexity in our build requires for portability >> > etc. >> > >> > #4 isn't something we can solve right now, but has a better solution if we >> > #go >> > for a greater internal redesign. but not today. some other day. we have to >> > live with what we have right now. >> >> Hum ? How ? I don't see how we can have good portability without a >> complex build system. Would be interested to know more by what you >> mean here ! > > we can isolate the portability layers in a much smaller area and always > provide > a portable interface wrapped on top of the real os one. currently our > portability code is scattered all through efl. it's not in one place. by > merging it into one location we'll be able to simplify it too so in the end we > just have some "efl port sys" lib that provides an "always works" set of > library calls and everything above that is 100% portable and never changes > platform to platform. > >> > i am always for improvements. but this really makes this method a very very >> > very hard ask to do this. i've thought about it and i think this is far TOO >> > radical a change as it means different execution methods, library >> > production, linking and so much more os by os. it's kind of insane though >> > theoretically doable on *nix. i've checked and PIE seems to be urtterly >> > unportable (windows can't do it based on my reading). the proposal would be >> > workable ONLY if we could do it unversally so it wouldn't bitrot and it'd >> > work. this is actually why quicklaunch did .so's instead of PIE because >> > that WORKS portably >> >> Windows is anyway another platform with a lot of specificity. >> Quicklaunch will never work there as there is no equivalent to fork >> anyway. Emulating what PIE allow is also mostly impossible as > > actually it does work. fork works. it's emulated and expensive, bit it WORKS. > it means you have an already "partially launched" app around. if we had a fork > POOL sitting around "ready to go" and they get ipc'd the command to run... > it'd > work the same on windows and on *nix. it'd be faster on both becaus
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Tue, 25 Oct 2016 14:32:29 -0700 Cedric BAIL said: > On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler > wrote: > > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL said: > > > > i've been mulling this. i followed the feedback so far. basically 2 main > > things: > > > > 1. this is intended to improve performance. especially startup times. > > 2. this is highly problematic in terms of portability and is going to create > > lots of code and build paths. > > It won't create lots of code and build paths. In fact it doesn't even > need two build paths. It just needs to uses all .la files for building > quicklaunch and can be rolled in, in parallel to current system. Just > the link of the quicklaunch binary would be different. Next step would > be to make a build option that setup symlink to that binary. Finally > that option would be the default, likely only afffecting the install > process. I don't think that create a lots of code (There should be no > code change) and build paths change at all. why are you doing a symlink if its JUST a quicklaunch binary change (that statically compiles all of efl into it)? you are trying to replace all of libevas.so.1, libeina.so.1, etc. with symlinks TO this quicklaunch binary ... to retain binary compatibility. you cant have the .so's AND this ql binary both have the same set of symbols. asking for trouble there. but to USe this binaries HAVE to be PIe and this makes for a non-portable pain in the arse of creating apps. which is why i did .so's for quicklaunch before.. because it also works with dll's on windows. PIE does not. not that i could find. > > 3. this makes debugging and other things a nightmare (ps names are all the > > same .. well depending on tool etc.) > > This is exagerated. Again just need to pass the right command line to > ps/top and problem is solved. Can easily be documented and we can also > provide a script that does it until we provide some nicer application > there. i mean simple user debugging. someone runs ps or top and all they see is 20 processes called "quicklaunch". because depending on the ps tool it may use one proc field or another. so a user goes "quicklaunch process is using N% cpu or N % ram". what real process is that? they don't know. because procps tools will not do what you want and thus the average person will become confused and find anything using efl like this to be horribly hard to work with. if top, htop, ps, killall, etc. don't work out of the box, it isby everyone (except yours) definition... a nightmare. i've messed with this before and it does not work across all the tools because of where they source command (and that can change based on config and cmdline options too). > > 4. generally the issue of complexity in our build requires for portability > > etc. > > > > #4 isn't something we can solve right now, but has a better solution if we > > #go > > for a greater internal redesign. but not today. some other day. we have to > > live with what we have right now. > > Hum ? How ? I don't see how we can have good portability without a > complex build system. Would be interested to know more by what you > mean here ! we can isolate the portability layers in a much smaller area and always provide a portable interface wrapped on top of the real os one. currently our portability code is scattered all through efl. it's not in one place. by merging it into one location we'll be able to simplify it too so in the end we just have some "efl port sys" lib that provides an "always works" set of library calls and everything above that is 100% portable and never changes platform to platform. > > i am always for improvements. but this really makes this method a very very > > very hard ask to do this. i've thought about it and i think this is far TOO > > radical a change as it means different execution methods, library > > production, linking and so much more os by os. it's kind of insane though > > theoretically doable on *nix. i've checked and PIE seems to be urtterly > > unportable (windows can't do it based on my reading). the proposal would be > > workable ONLY if we could do it unversally so it wouldn't bitrot and it'd > > work. this is actually why quicklaunch did .so's instead of PIE because > > that WORKS portably > > Windows is anyway another platform with a lot of specificity. > Quicklaunch will never work there as there is no equivalent to fork > anyway. Emulating what PIE allow is also mostly impossible as actually it does work. fork works. it's emulated and expensive, bit it WORKS. it means you have an already "partially launched" app around. if we had a fork POOL sitting around "ready to go" and they get ipc'd the command to run... it'd work the same on windows and on *nix. it'd be faster on both because when an app runs, the init/liking phase is already done by a process that is sitting as a hot spare ready to go. so ql dlopen()'s the apps' DLL (as now it has to be a dll and not an exe) and the app i
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Mon, Oct 24, 2016 at 10:31 PM, Carsten Haitzler wrote: > On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL said: > > i've been mulling this. i followed the feedback so far. basically 2 main > things: > > 1. this is intended to improve performance. especially startup times. > 2. this is highly problematic in terms of portability and is going to create > lots of code and build paths. It won't create lots of code and build paths. In fact it doesn't even need two build paths. It just needs to uses all .la files for building quicklaunch and can be rolled in, in parallel to current system. Just the link of the quicklaunch binary would be different. Next step would be to make a build option that setup symlink to that binary. Finally that option would be the default, likely only afffecting the install process. I don't think that create a lots of code (There should be no code change) and build paths change at all. > 3. this makes debugging and other things a nightmare (ps names are all the > same .. well depending on tool etc.) This is exagerated. Again just need to pass the right command line to ps/top and problem is solved. Can easily be documented and we can also provide a script that does it until we provide some nicer application there. > 4. generally the issue of complexity in our build requires for portability > etc. > > #4 isn't something we can solve right now, but has a better solution if we go > for a greater internal redesign. but not today. some other day. we have to > live > with what we have right now. Hum ? How ? I don't see how we can have good portability without a complex build system. Would be interested to know more by what you mean here ! > i am always for improvements. but this really makes this method a very very > very hard ask to do this. i've thought about it and i think this is far TOO > radical a change as it means different execution methods, library production, > linking and so much more os by os. it's kind of insane though theoretically > doable on *nix. i've checked and PIE seems to be urtterly unportable (windows > can't do it based on my reading). the proposal would be workable ONLY if we > could do it unversally so it wouldn't bitrot and it'd work. this is actually > why quicklaunch did .so's instead of PIE because that WORKS portably Windows is anyway another platform with a lot of specificity. Quicklaunch will never work there as there is no equivalent to fork anyway. Emulating what PIE allow is also mostly impossible as initialization of a library is different as what a binary require (Even with ALSR). Anyway we already have different code path/build differrence for Windows and Linux and this one is not a new one. As in, it won't increase the difference much. As for the change, they will be the same for all Unix. It is a compilation option on just the binary plus a static linking of all efl to it (basically an ifdef with a lot a +=). So I disagree here with your assesment of the complexity introduced in our build system. The main change is going to be on install which will be made by doing symlink to that executable. > so here is what i think is sane: > > 1. merge libraries aggressively. this reduces linking time BETWEEN efl libs > and > thus an efl lib .so when loaded has the libs that were merged already linked > together. i propose we try and get down to maybe 3 or 4 .so's this will close > the gap. > > how do we make this portable? well on *nix we can install symlinks from our > old .so major versions TO the new merged ones. but what on windows? the best i > can think of for windows is produce "empty" dll's that ONLY contain like a > dummy function and the correct linking to other efl libs they depend on eg if > we merged eina, eo, ecore, efl into "efl" then eina depends on efl, eo, depend > on efl and ecore depends on efl. thus link any one of these and you got your > symbols from before plus more. this solution ALSO works on *nix ... so we > could > use it as a start with symlinks being an optimization of it. yes it means 2 > install paths unfortunately. we COULD do it as a post-install that "rm's" > the .so's we installed and replaces them by symlinks which makes this kind of > sane to keep around. Wouldn't that not work due to DSO ? No idea if that would work with Windows. Also, I am not to sure it would improve things that much as you still need to load a library, agreed an empty one, but still need to load one. It will also be a lot more work than my proposed change as it require to shuffle all the code around and create a set of new empty libraries. > 2. move quicklaunch further down to core (efl core) so its not only up in elm > land. this would make it a core capability. we STILL should load .so's/dylibs > or dll's if there, and try PIE binaries... but we should work here on windows > too. make it a core feature low down the stack We are already doing PIE binary. As stated above Windows can never have quicklaunch in any form (No fork, and li
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Fri, 21 Oct 2016 15:14:20 -0700 Cedric BAIL said: i've been mulling this. i followed the feedback so far. basically 2 main things: 1. this is intended to improve performance. especially startup times. 2. this is highly problematic in terms of portability and is going to create lots of code and build paths. 3. this makes debugging and other things a nightmare (ps names are all the same .. well depending on tool etc.) 4. generally the issue of complexity in our build requires for portability etc. #4 isn't something we can solve right now, but has a better solution if we go for a greater internal redesign. but not today. some other day. we have to live with what we have right now. i am always for improvements. but this really makes this method a very very very hard ask to do this. i've thought about it and i think this is far TOO radical a change as it means different execution methods, library production, linking and so much more os by os. it's kind of insane though theoretically doable on *nix. i've checked and PIE seems to be urtterly unportable (windows can't do it based on my reading). the proposal would be workable ONLY if we could do it unversally so it wouldn't bitrot and it'd work. this is actually why quicklaunch did .so's instead of PIE because that WORKS portably so here is what i think is sane: 1. merge libraries aggressively. this reduces linking time BETWEEN efl libs and thus an efl lib .so when loaded has the libs that were merged already linked together. i propose we try and get down to maybe 3 or 4 .so's this will close the gap. how do we make this portable? well on *nix we can install symlinks from our old .so major versions TO the new merged ones. but what on windows? the best i can think of for windows is produce "empty" dll's that ONLY contain like a dummy function and the correct linking to other efl libs they depend on eg if we merged eina, eo, ecore, efl into "efl" then eina depends on efl, eo, depend on efl and ecore depends on efl. thus link any one of these and you got your symbols from before plus more. this solution ALSO works on *nix ... so we could use it as a start with symlinks being an optimization of it. yes it means 2 install paths unfortunately. we COULD do it as a post-install that "rm's" the .so's we installed and replaces them by symlinks which makes this kind of sane to keep around. 2. move quicklaunch further down to core (efl core) so its not only up in elm land. this would make it a core capability. we STILL should load .so's/dylibs or dll's if there, and try PIE binaries... but we should work here on windows too. make it a core feature low down the stack 3. use quicklaunch mode for e. since efl will be there then this will work. this does not MEAN we have to use it for all apps nor does it mean we MUST use it for e... but this would help these above should get almost all the benefit of the PIE thing with the least of the downsides. we still have ps name issues though. :( we need to dig more to try our best to make this "ok" but this means we still work normally. as long as the "quicklaunch" binary uses very few efl symbols, loading efl libs and linking will be a much lower cost as we do less intra-lib linking and minimal app <-> lib linking. in fact we could make a single libefl.so as a build option if symlinks and dependency .so's as above work. this should be almost the same cost as the pie binary proposal but is portable. > Hello, > > Before I embark on a crazy idea that may just break everything, I > would like to get some feedback. So one of the problem we are facing > with EFL is that with all those separated library we end up with some > serious time during application startup dedicated to linking. A work > around has been quicklaunch, but it is mostly unused. Also it does > only improve things once it has started, not much during startup time > where you still have to do all the linking. > > With PIE and -rdynamic that we use today on quicklaunch binary, we use > dlopen to get symbol from it and load this binary as if they were > library... So maybe we can actually use that same mechanism to > actually statically link efl with the quicklaunch server binary and > make all efl .so library just a symlink to that binary. Now, > enlightenment could actually try to rely on quicklaunch to start > itself and start other application. This should speed up boot time and > application start up time. The reason to make it the default way of > doing things is to make sure that it is maintained and work. > > Now some of the drawback. First it is a trick, meaning most people > that will try to dig in will get confused at first on what is going on > and we need to document it. Building EFL is going to become even more > complex (I don't know yet how to generate the proper symlink), let's > enjoy more of the autofoo dark art. All binary will happear as if they > are just a fork of enlightenment and you will need to use some flags > to ps/top and friend
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Mon, Oct 24, 2016 at 1:11 AM, Stefan Schmidt wrote: > Hello. > > On 22/10/16 00:53, Cedric BAIL wrote: >> On Fri, Oct 21, 2016 at 3:31 PM, Stefan Schmidt >> wrote: >>> On 22/10/16 00:14, Cedric BAIL wrote: Before I embark on a crazy idea that may just break everything, I would like to get some feedback. So one of the problem we are facing with EFL is that with all those separated library we end up with some serious time during application startup dedicated to linking. A work around has been quicklaunch, but it is mostly unused. Also it does only improve things once it has started, not much during startup time where you still have to do all the linking. With PIE and -rdynamic that we use today on quicklaunch binary, we use dlopen to get symbol from it and load this binary as if they were library... So maybe we can actually use that same mechanism to actually statically link efl with the quicklaunch server binary and make all efl .so library just a symlink to that binary. Now, enlightenment could actually try to rely on quicklaunch to start itself and start other application. This should speed up boot time and application start up time. The reason to make it the default way of doing things is to make sure that it is maintained and work. >>> >>> Are there actual numbers about speed improvement? Are we really needed >>> this speedup so badly? As you can already see from this two sentences >>> I'm not really in favor of it. See below for some arguments. >> >> Yes, it does, but really depends on your hardware. > > And by making it the default your are forcing this quite ugly change to > all EFL users while only certain hardware setups would really see a > significant benefit. Well, the pay off on startup is paying a lot more on the lower end of the spectrum for sure, but the memory saving will pay off for everyone. Statically linking efl and initializing them once for the entire system, means that a lot of the memory consumption issue we see, like the eo one, would not be a problem anymore. This also would include loading the system theme once and avoid duplicating that data across the system. Overall everyone benefit, just the biggest visible change are obviously on the lower end if you want to notice it (EFL application on my laptop would never come close to the memory consumption of Firefox !). >> For example, on >> Raspberry Pi out of the 4s it takes to startup elemines, 1s can be >> saved by quicklaunch. We are talking of a 25% improvement here. It >> will also likely push us to improve further quicklaunch >> infrastructure. > > 4 seconds?? Full 4 seconds to get started? I have a hard time to believe > that. Maybe some of the system configuration is badly? Really slow SD > card? And anyway, what does elimines do the other 3 seconds of startup? Yes, Raspberry Pi is slow. SD is slow. The other hardware we have in mind is max three times faster. And yes, we are mostly targeting the lower end of the spectrum. Also no elemines doesn't do the other 3 seconds. It spend part of its time reading configuration, decompressing the theme, loading jpeg and connecting to X. Basically 80 to 90% of the time is spend in efl itself. > Does other efl apps have the same problem? Terminology? Rage? > You are asking for a drastic change with quite some downsides here so I > hope it is not just based on data from one app in one hardware > configuration. :) Bigger application like Terminology take much longer to start. I have no profiled them to tell exactly where the times goes, but for sure the first 3s are identical. Terminology has a special mode to have one process for all session and any further start is blazingly fast. Which is the situation we want for all application without specific code change per application. > We should be able to preconnect to X/Wayland and save >> for the new process that is starting. We should also be able to pre >> load the theme which would also save a lot of memory as those data >> would be shared among all process. Elementary should be able to pre >> load profile too and save that to every process. So first step is a >> 25% win for sure. Further down the line, I bet we can save up to 60% >> of our startup time. Basically some people are trying to push E/EFL >> even further into smaller embeded device and they are requesting our >> assistance. > > You are asking for it as default! Not as some way to make EFL better > suited for specific scenarios. A big difference in my eyes. I understand > the not being tested problem well enough (I deal with not normally > tested code paths myself during all the QA work), but that should not > mean we make something very specific our default. Default should be > generic. e are not enabling architecture based compiler optimizations > either but leave that to the user. I am afraid it will bit rot to much. Also I think it would deliver enough improvement for ever
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
Hello. On 22/10/16 00:53, Cedric BAIL wrote: > On Fri, Oct 21, 2016 at 3:31 PM, Stefan Schmidt > wrote: >> On 22/10/16 00:14, Cedric BAIL wrote: >>> Before I embark on a crazy idea that may just break everything, I >>> would like to get some feedback. So one of the problem we are facing >>> with EFL is that with all those separated library we end up with some >>> serious time during application startup dedicated to linking. A work >>> around has been quicklaunch, but it is mostly unused. Also it does >>> only improve things once it has started, not much during startup time >>> where you still have to do all the linking. >>> >>> With PIE and -rdynamic that we use today on quicklaunch binary, we use >>> dlopen to get symbol from it and load this binary as if they were >>> library... So maybe we can actually use that same mechanism to >>> actually statically link efl with the quicklaunch server binary and >>> make all efl .so library just a symlink to that binary. Now, >>> enlightenment could actually try to rely on quicklaunch to start >>> itself and start other application. This should speed up boot time and >>> application start up time. The reason to make it the default way of >>> doing things is to make sure that it is maintained and work. >> >> Are there actual numbers about speed improvement? Are we really needed >> this speedup so badly? As you can already see from this two sentences >> I'm not really in favor of it. See below for some arguments. > > Yes, it does, but really depends on your hardware. And by making it the default your are forcing this quite ugly change to all EFL users while only certain hardware setups would really see a significant benefit. For example, on > Raspberry Pi out of the 4s it takes to startup elemines, 1s can be > saved by quicklaunch. We are talking of a 25% improvement here. It > will also likely push us to improve further quicklaunch > infrastructure. 4 seconds?? Full 4 seconds to get started? I have a hard time to believe that. Maybe some of the system configuration is badly? Really slow SD card? And anyway, what does elimines do the other 3 seconds of startup? Does other efl apps have the same problem? Terminology? Rage? You are asking for a drastic change with quite some downsides here so I hope it is not just based on data from one app in one hardware configuration. :) We should be able to preconnect to X/Wayland and save > for the new process that is starting. We should also be able to pre > load the theme which would also save a lot of memory as those data > would be shared among all process. Elementary should be able to pre > load profile too and save that to every process. So first step is a > 25% win for sure. Further down the line, I bet we can save up to 60% > of our startup time. Basically some people are trying to push E/EFL > even further into smaller embeded device and they are requesting our > assistance. You are asking for it as default! Not as some way to make EFL better suited for specific scenarios. A big difference in my eyes. I understand the not being tested problem well enough (I deal with not normally tested code paths myself during all the QA work), but that should not mean we make something very specific our default. Default should be generic. e are not enabling architecture based compiler optimizations either but leave that to the user. >>> Now some of the drawback. First it is a trick, meaning most people >>> that will try to dig in will get confused at first on what is going on >>> and we need to document it. Building EFL is going to become even more >>> complex (I don't know yet how to generate the proper symlink), let's >>> enjoy more of the autofoo dark art. >> >> To be honest here, this one alone is a killer in my opinion. What we >> have is already complex enough. I often enough find myself in situation >> where I bang my head against the wall until I get the system to do what >> I want. > > Yes, that is not going to improve this way for sure. I your proposal we would just need to live with this? >>> All binary will happear as if they >>> are just a fork of enlightenment and you will need to use some flags >>> to ps/top and friends to ask them to read /proc to get the new command >>> line otherwise all binary have the same name. >> >> And that one is the nail in the coffin in my opinion. >> >>> I think that quicklaunch will require also to be improved by being >>> made more robust/with a larger set of feature (nicely opposing goal). >>> Eventually it should handle application restart for example (What >>> enlightenment_start does today)... which annoy me a bit. Also we would >>> likely be on a path to reimplement systemd --user here. Something I >>> have tried to avoid for some time. >>> >>> Ok, does anyone see some serious blocker to this idea ? Impact on >>> packager to much ? Something I missed ? Something that need to be >>> clarified ? >> >> I have a hard time to see that this drastic change is real
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On 10/22/2016 09:23 AM, Cedric BAIL wrote: > On Fri, Oct 21, 2016 at 3:31 PM, Stefan Schmidt > wrote: >> On 22/10/16 00:14, Cedric BAIL wrote: >>> Before I embark on a crazy idea that may just break everything, I >>> would like to get some feedback. So one of the problem we are facing >>> with EFL is that with all those separated library we end up with some >>> serious time during application startup dedicated to linking. A work >>> around has been quicklaunch, but it is mostly unused. Also it does >>> only improve things once it has started, not much during startup time >>> where you still have to do all the linking. >>> >>> With PIE and -rdynamic that we use today on quicklaunch binary, we use >>> dlopen to get symbol from it and load this binary as if they were >>> library... So maybe we can actually use that same mechanism to >>> actually statically link efl with the quicklaunch server binary and >>> make all efl .so library just a symlink to that binary. Now, >>> enlightenment could actually try to rely on quicklaunch to start >>> itself and start other application. This should speed up boot time and >>> application start up time. The reason to make it the default way of >>> doing things is to make sure that it is maintained and work. >> >> Are there actual numbers about speed improvement? Are we really needed >> this speedup so badly? As you can already see from this two sentences >> I'm not really in favor of it. See below for some arguments. > > Yes, it does, but really depends on your hardware. For example, on > Raspberry Pi out of the 4s it takes to startup elemines, 1s can be > saved by quicklaunch. We are talking of a 25% improvement here. It > will also likely push us to improve further quicklaunch > infrastructure. We should be able to preconnect to X/Wayland and save > for the new process that is starting. We should also be able to pre > load the theme which would also save a lot of memory as those data > would be shared among all process. Elementary should be able to pre > load profile too and save that to every process. So first step is a > 25% win for sure. Further down the line, I bet we can save up to 60% > of our startup time. Basically some people are trying to push E/EFL > even further into smaller embeded device and they are requesting our > assistance. > >>> Now some of the drawback. First it is a trick, meaning most people >>> that will try to dig in will get confused at first on what is going on >>> and we need to document it. Building EFL is going to become even more >>> complex (I don't know yet how to generate the proper symlink), let's >>> enjoy more of the autofoo dark art. >> >> To be honest here, this one alone is a killer in my opinion. What we >> have is already complex enough. I often enough find myself in situation >> where I bang my head against the wall until I get the system to do what >> I want. > > Yes, that is not going to improve this way for sure. > >>> All binary will happear as if they >>> are just a fork of enlightenment and you will need to use some flags >>> to ps/top and friends to ask them to read /proc to get the new command >>> line otherwise all binary have the same name. >> >> And that one is the nail in the coffin in my opinion. >> >>> I think that quicklaunch will require also to be improved by being >>> made more robust/with a larger set of feature (nicely opposing goal). >>> Eventually it should handle application restart for example (What >>> enlightenment_start does today)... which annoy me a bit. Also we would >>> likely be on a path to reimplement systemd --user here. Something I >>> have tried to avoid for some time. >>> >>> Ok, does anyone see some serious blocker to this idea ? Impact on >>> packager to much ? Something I missed ? Something that need to be >>> clarified ? >> >> I have a hard time to see that this drastic change is really worth some >> improved launch time. Especially when you keep in mind that we are >> planning to merge some of our libs anyway thus reducing the amount of >> linking. > > I do think we will never save as much nor open ourself to all the > potential optimization that quicklaunch provide b merging a few > library. I don't have comparison there, but from above number you can > guess that it will be below 25%. Also this approach is I think nicer > as we do not need to forcefully merge library into weird name that > have less sense and endup with endless debate, like what does efl_core > integrate and so on. The only merge I would still go with is > efl/eo/ecore as they are clearly linked strongly by behavior now. > I think that given the long term plan has been to merge libraries, its only really worth while if the speedup vs the merged libraries is significant and if it really is only systems like rpi's that are having enough issues for it to be worth it maybe it should be a configure option for people building on those systems so general purpose distro's stay as is, the downside of that though is
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
On Fri, Oct 21, 2016 at 3:31 PM, Stefan Schmidt wrote: > On 22/10/16 00:14, Cedric BAIL wrote: >> Before I embark on a crazy idea that may just break everything, I >> would like to get some feedback. So one of the problem we are facing >> with EFL is that with all those separated library we end up with some >> serious time during application startup dedicated to linking. A work >> around has been quicklaunch, but it is mostly unused. Also it does >> only improve things once it has started, not much during startup time >> where you still have to do all the linking. >> >> With PIE and -rdynamic that we use today on quicklaunch binary, we use >> dlopen to get symbol from it and load this binary as if they were >> library... So maybe we can actually use that same mechanism to >> actually statically link efl with the quicklaunch server binary and >> make all efl .so library just a symlink to that binary. Now, >> enlightenment could actually try to rely on quicklaunch to start >> itself and start other application. This should speed up boot time and >> application start up time. The reason to make it the default way of >> doing things is to make sure that it is maintained and work. > > Are there actual numbers about speed improvement? Are we really needed > this speedup so badly? As you can already see from this two sentences > I'm not really in favor of it. See below for some arguments. Yes, it does, but really depends on your hardware. For example, on Raspberry Pi out of the 4s it takes to startup elemines, 1s can be saved by quicklaunch. We are talking of a 25% improvement here. It will also likely push us to improve further quicklaunch infrastructure. We should be able to preconnect to X/Wayland and save for the new process that is starting. We should also be able to pre load the theme which would also save a lot of memory as those data would be shared among all process. Elementary should be able to pre load profile too and save that to every process. So first step is a 25% win for sure. Further down the line, I bet we can save up to 60% of our startup time. Basically some people are trying to push E/EFL even further into smaller embeded device and they are requesting our assistance. >> Now some of the drawback. First it is a trick, meaning most people >> that will try to dig in will get confused at first on what is going on >> and we need to document it. Building EFL is going to become even more >> complex (I don't know yet how to generate the proper symlink), let's >> enjoy more of the autofoo dark art. > > To be honest here, this one alone is a killer in my opinion. What we > have is already complex enough. I often enough find myself in situation > where I bang my head against the wall until I get the system to do what > I want. Yes, that is not going to improve this way for sure. >> All binary will happear as if they >> are just a fork of enlightenment and you will need to use some flags >> to ps/top and friends to ask them to read /proc to get the new command >> line otherwise all binary have the same name. > > And that one is the nail in the coffin in my opinion. > >> I think that quicklaunch will require also to be improved by being >> made more robust/with a larger set of feature (nicely opposing goal). >> Eventually it should handle application restart for example (What >> enlightenment_start does today)... which annoy me a bit. Also we would >> likely be on a path to reimplement systemd --user here. Something I >> have tried to avoid for some time. >> >> Ok, does anyone see some serious blocker to this idea ? Impact on >> packager to much ? Something I missed ? Something that need to be >> clarified ? > > I have a hard time to see that this drastic change is really worth some > improved launch time. Especially when you keep in mind that we are > planning to merge some of our libs anyway thus reducing the amount of > linking. I do think we will never save as much nor open ourself to all the potential optimization that quicklaunch provide b merging a few library. I don't have comparison there, but from above number you can guess that it will be below 25%. Also this approach is I think nicer as we do not need to forcefully merge library into weird name that have less sense and endup with endless debate, like what does efl_core integrate and so on. The only merge I would still go with is efl/eo/ecore as they are clearly linked strongly by behavior now. -- Cedric BAIL -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
I don't know how portable that would be??? I don't think /proc is available by default on most non-Linux Unix-like OS I use... On Fri, Oct 21, 2016 at 11:31 PM, Stefan Schmidt wrote: > Hello. > > On 22/10/16 00:14, Cedric BAIL wrote: > > Hello, > > > > Before I embark on a crazy idea that may just break everything, I > > would like to get some feedback. So one of the problem we are facing > > with EFL is that with all those separated library we end up with some > > serious time during application startup dedicated to linking. A work > > around has been quicklaunch, but it is mostly unused. Also it does > > only improve things once it has started, not much during startup time > > where you still have to do all the linking. > > > > With PIE and -rdynamic that we use today on quicklaunch binary, we use > > dlopen to get symbol from it and load this binary as if they were > > library... So maybe we can actually use that same mechanism to > > actually statically link efl with the quicklaunch server binary and > > make all efl .so library just a symlink to that binary. Now, > > enlightenment could actually try to rely on quicklaunch to start > > itself and start other application. This should speed up boot time and > > application start up time. The reason to make it the default way of > > doing things is to make sure that it is maintained and work. > > Are there actual numbers about speed improvement? Are we really needed > this speedup so badly? As you can already see from this two sentences > I'm not really in favor of it. See below for some arguments. > > > Now some of the drawback. First it is a trick, meaning most people > > that will try to dig in will get confused at first on what is going on > > and we need to document it. Building EFL is going to become even more > > complex (I don't know yet how to generate the proper symlink), let's > > enjoy more of the autofoo dark art. > > To be honest here, this one alone is a killer in my opinion. What we > have is already complex enough. I often enough find myself in situation > where I bang my head against the wall until I get the system to do what > I want. > > All binary will happear as if they > > are just a fork of enlightenment and you will need to use some flags > > to ps/top and friends to ask them to read /proc to get the new command > > line otherwise all binary have the same name. > > And that one is the nail in the coffin in my opinion. > > > I think that quicklaunch will require also to be improved by being > > made more robust/with a larger set of feature (nicely opposing goal). > > Eventually it should handle application restart for example (What > > enlightenment_start does today)... which annoy me a bit. Also we would > > likely be on a path to reimplement systemd --user here. Something I > > have tried to avoid for some time. > > > > Ok, does anyone see some serious blocker to this idea ? Impact on > > packager to much ? Something I missed ? Something that need to be > > clarified ? > > I have a hard time to see that this drastic change is really worth some > improved launch time. Especially when you keep in mind that we are > planning to merge some of our libs anyway thus reducing the amount of > linking. > > regards > Stefan Schmidt > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > ___ > enlightenment-devel mailing list > enlightenment-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed
Hello. On 22/10/16 00:14, Cedric BAIL wrote: > Hello, > > Before I embark on a crazy idea that may just break everything, I > would like to get some feedback. So one of the problem we are facing > with EFL is that with all those separated library we end up with some > serious time during application startup dedicated to linking. A work > around has been quicklaunch, but it is mostly unused. Also it does > only improve things once it has started, not much during startup time > where you still have to do all the linking. > > With PIE and -rdynamic that we use today on quicklaunch binary, we use > dlopen to get symbol from it and load this binary as if they were > library... So maybe we can actually use that same mechanism to > actually statically link efl with the quicklaunch server binary and > make all efl .so library just a symlink to that binary. Now, > enlightenment could actually try to rely on quicklaunch to start > itself and start other application. This should speed up boot time and > application start up time. The reason to make it the default way of > doing things is to make sure that it is maintained and work. Are there actual numbers about speed improvement? Are we really needed this speedup so badly? As you can already see from this two sentences I'm not really in favor of it. See below for some arguments. > Now some of the drawback. First it is a trick, meaning most people > that will try to dig in will get confused at first on what is going on > and we need to document it. Building EFL is going to become even more > complex (I don't know yet how to generate the proper symlink), let's > enjoy more of the autofoo dark art. To be honest here, this one alone is a killer in my opinion. What we have is already complex enough. I often enough find myself in situation where I bang my head against the wall until I get the system to do what I want. All binary will happear as if they > are just a fork of enlightenment and you will need to use some flags > to ps/top and friends to ask them to read /proc to get the new command > line otherwise all binary have the same name. And that one is the nail in the coffin in my opinion. > I think that quicklaunch will require also to be improved by being > made more robust/with a larger set of feature (nicely opposing goal). > Eventually it should handle application restart for example (What > enlightenment_start does today)... which annoy me a bit. Also we would > likely be on a path to reimplement systemd --user here. Something I > have tried to avoid for some time. > > Ok, does anyone see some serious blocker to this idea ? Impact on > packager to much ? Something I missed ? Something that need to be > clarified ? I have a hard time to see that this drastic change is really worth some improved launch time. Especially when you keep in mind that we are planning to merge some of our libs anyway thus reducing the amount of linking. regards Stefan Schmidt -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
[E-devel] [RFC] One binary to rule them all, packagers opinion needed
Hello, Before I embark on a crazy idea that may just break everything, I would like to get some feedback. So one of the problem we are facing with EFL is that with all those separated library we end up with some serious time during application startup dedicated to linking. A work around has been quicklaunch, but it is mostly unused. Also it does only improve things once it has started, not much during startup time where you still have to do all the linking. With PIE and -rdynamic that we use today on quicklaunch binary, we use dlopen to get symbol from it and load this binary as if they were library... So maybe we can actually use that same mechanism to actually statically link efl with the quicklaunch server binary and make all efl .so library just a symlink to that binary. Now, enlightenment could actually try to rely on quicklaunch to start itself and start other application. This should speed up boot time and application start up time. The reason to make it the default way of doing things is to make sure that it is maintained and work. Now some of the drawback. First it is a trick, meaning most people that will try to dig in will get confused at first on what is going on and we need to document it. Building EFL is going to become even more complex (I don't know yet how to generate the proper symlink), let's enjoy more of the autofoo dark art. All binary will happear as if they are just a fork of enlightenment and you will need to use some flags to ps/top and friends to ask them to read /proc to get the new command line otherwise all binary have the same name. I think that quicklaunch will require also to be improved by being made more robust/with a larger set of feature (nicely opposing goal). Eventually it should handle application restart for example (What enlightenment_start does today)... which annoy me a bit. Also we would likely be on a path to reimplement systemd --user here. Something I have tried to avoid for some time. Ok, does anyone see some serious blocker to this idea ? Impact on packager to much ? Something I missed ? Something that need to be clarified ? Have fun, -- Cedric BAIL -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
