Re: prelink performance gains
Am 18.10.2013 22:58, schrieb Robert Relyea: On 10/17/2013 06:48 AM, Jan Kratochvil wrote: rpm uses prelink -y so it already works in most cases and the rare cases can be fixed in prelink. The problem is its maintainer Jakub has more significant work to do nowadays. I use it as well, but it causes all sorts of problems (particularly in selinux restricted apps) because it's really unfriendly for a library to exec a random program and open a pipe. One of the things that would have to be done would be either 1) provide a library call that can supply the unlinked data, or 2) provide infrastructure in prelink that can reliably update the integrity check files in a way that doesn't race the changed libraries (and in a way that makes sure only prelink changed the libraries, not someone else). Both of these are easy to get wrong and *that* is the point why crap which plays with integrity has to be banned from defaults as long it's gains are not *really large* keep things as simple as possible is the thumb of rules in case of integrity, error proof and security - and *nothing* ever will change this fact if someone does not care about securiyt and integrity: *fine* it is his choice *but* do not make such wrong decisions any longer the default also for people who do not care because they do not know better and not becuase they do not care as their own decisions defaults need to be sane, *rock solid* and questionable optimizings have to be a opt-in for people who know hat tey are doing or at least thnik so there is no but and if and no place for niceness - period *personally* i do not care about this whole discussion because prelink is banned for a long time and any package wichich requires it is banned too because i consider it as crap - *but* i do not only care about my personal environment - otherwise i would not need to waste any second on public mailing lists signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 17.10.2013 15:48, schrieb Jan Kratochvil: On Thu, 17 Oct 2013 00:16:35 +0200, Robert Relyea wrote: prelink throws rocks at a lot of packages that have to check the integrity of the shared libraries they are using. It provides no real useful way of assisting in those tasks, It provides 'prelink -y' only for exactly that purpose. There is a bug in -y; but it does not work in some (rare) cases. https://bugzilla.redhat.com/show_bug.cgi?id=666143 Workaround of that bug is one line of code, it just has not been accepted yet. and we can't meaningfully measure or observe the performance gains. You will need to strongly show the latter, because the cost it forces on other packages is unbearable. Here is another measurement. I do not agree with the initial post's approach as (1) It flushes disk cache. That has no meaning for prelink measurement, it just adds more fuzziness to the results and it is even unreal representation of real world use cases. (2) It runs big end-user GUI application. what is the proposed usecase for prelink all the years This adds various interactions with X and the applications has its own heavy startup cost, it all also adds fuzziness to the results. (3) When we look at global GNU/Linux market its end user deployment (*Office) is not relevant, server side execution matters. = It all seems to me as intentionally chosen just to prove prelink gain is not measurable. you think the startup performance matters on servers? come on - nobody is starting and stopping servcices all day long *espcecially* on servers you *do not want* prelink because intrusion detection and the fact that honestly *all* long running processes aka services must be PIE and so are not prelined at all so, and for the ones violating package guidelines because not PIE you do *not* want prelink because on a server typically you install updates, restart the service or reboot in rare cases, have no ASLR and that prelink comes at night for the minimal ASLR does not help because the binaries are already running and *no* do *not* propose now restart services blindly after prelink so what about are you talking in this whole thread? signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/17/2013 06:48 AM, Jan Kratochvil wrote: On Thu, 17 Oct 2013 00:16:35 +0200, Robert Relyea wrote: prelink throws rocks at a lot of packages that have to check the integrity of the shared libraries they are using. It provides no real useful way of assisting in those tasks, It provides 'prelink -y' only for exactly that purpose. There is a bug in -y; but it does not work in some (rare) cases. https://bugzilla.redhat.com/show_bug.cgi?id=666143 Workaround of that bug is one line of code, it just has not been accepted yet. I do not know the FIPS prelink issues to be able to talk more about it. 2. FIPS isn't the only place you need to do sofware integrity checks. (see rpm). rpm uses prelink -y so it already works in most cases and the rare cases can be fixed in prelink. The problem is its maintainer Jakub has more significant work to do nowadays. I use it as well, but it causes all sorts of problems (particularly in selinux restricted apps) because it's really unfriendly for a library to exec a random program and open a pipe. One of the things that would have to be done would be either 1) provide a library call that can supply the unlinked data, or 2) provide infrastructure in prelink that can reliably update the integrity check files in a way that doesn't race the changed libraries (and in a way that makes sure only prelink changed the libraries, not someone else). Both of these are easy to get wrong. smime.p7s Description: S/MIME Cryptographic Signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/13 at 03:21pm, Daniel P. Berrange wrote: On Tue, Oct 15, 2013 at 10:14:13AM -0400, Paul Wouters wrote: Please take prelink behind the barn and shoot it. Thanks. ... How can we have this discussion? We have had reports of numbers showing no real gain. We know it affects ASLR negatively and complicates FIPS engineering. Why haven't we at least reached a compromise where prelink is _optional_ and not installed per default? It seems currently to be part of the fedora live install (and the rhel workstation installs) This mail is just a pre-cursor to proposing it be removed. When prelink was added by default, the justification was the performance benefits to startup. To justify removing it, we just need to collect data to show that those performance benefits no longer exist, with current hardware and software combination in Fedora. That is what this email thread is seeking to confirm. So assuming no one comes forward to show some incredible benefit(s) of prelink, a proposal to kill it by default will surely follow. I have created a FESCo ticket titled Don't enable prelink by default in Fedora. Please see https://fedorahosted.org/fesco/ticket/1183 URL. -- Dhiru -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, 17 Oct 2013 00:16:35 +0200, Robert Relyea wrote: prelink throws rocks at a lot of packages that have to check the integrity of the shared libraries they are using. It provides no real useful way of assisting in those tasks, It provides 'prelink -y' only for exactly that purpose. There is a bug in -y; but it does not work in some (rare) cases. https://bugzilla.redhat.com/show_bug.cgi?id=666143 Workaround of that bug is one line of code, it just has not been accepted yet. and we can't meaningfully measure or observe the performance gains. You will need to strongly show the latter, because the cost it forces on other packages is unbearable. Here is another measurement. I do not agree with the initial post's approach as (1) It flushes disk cache. That has no meaning for prelink measurement, it just adds more fuzziness to the results and it is even unreal representation of real world use cases. (2) It runs big end-user GUI application. This adds various interactions with X and the applications has its own heavy startup cost, it all also adds fuzziness to the results. (3) When we look at global GNU/Linux market its end user deployment (*Office) is not relevant, server side execution matters. = It all seems to me as intentionally chosen just to prove prelink gain is not measurable. Therefore I made IMO a more real world measurement with: time gdb/configure Particularly: for i in `seq 1 20`;do git clean -dfx /dev/null;sync;sleep 6;(time ./configure /dev/null) 21|grep ^real|tee -a /tmp/times;done To make clear why such test matters. Running configure scripts has become the major builds bottleneck in recent days as it cannot be parallelized on multicore and multinode machines. For GDB it takes now 56% (of 1m37.380s) of the whole compilation time. And be sure developers are running configure by orders of magnitude more times than *Office (or even LaTeX). without prelink (in seconds): 54.832 54.099 55.122 54.704 54.228 54.728 54.240 53.914 54.878 54.308 54.461 53.859 54.311 53.964 53.958 54.712 54.498 53.924 54.988 54.502 mean 54.4115 | standard deviation 0.39093 with prelink (in seconds): 53.392 52.569 52.549 53.005 52.452 52.719 53.278 52.534 52.439 52.737 52.571 52.656 53.142 52.671 52.555 52.973 52.483 52.369 53.246 53.128 mean 52.7734 | standard deviation 0.31998 mean gain 1.6381 == 3.01% Unfortunately I have to admit that does not mean much. According to Observer Effect and Measurement Bias in Performance Analysis http://www.inf.usi.ch/faculty/hauswirth/publications/CU-CS-1042-08.pdf completely unrelated environment conditions (like object files reorder) cause up to 5% performance measurement bias; please read the PDF for the reasons. - FIPS foot-bullets [..] 1. Some packages supports FIPS on the fly. I do not know the FIPS prelink issues to be able to talk more about it. 2. FIPS isn't the only place you need to do sofware integrity checks. (see rpm). rpm uses prelink -y so it already works in most cases and the rare cases can be fixed in prelink. The problem is its maintainer Jakub has more significant work to do nowadays. If prelink provided noticeable improvement, It depends whether 3% (of configure time); or rather 1.69% of total build time matters or not. If it is relevant for example more expensive i7-4771 is just 3% faster than i7-4770. I would say we put the resources into prelink to make it play more nicely with these integrity systems, but since it doesn't, the more rational approach is have it go away. According to the numbers above my conclusion is different. I agree there remains some work on prelink itself and some packages around to make prelink relevant again for the predominant notebook market nowadays. prelink performance gains [summary] https://lists.fedoraproject.org/pipermail/devel/2013-October/190309.html Thanks, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/17/13 at 03:48pm, Jan Kratochvil wrote: Here is another measurement. I do not agree with the initial post's approach as (1) It flushes disk cache. That has no meaning for prelink measurement, it just adds more fuzziness to the results and it is even unreal representation of real world use cases. (2) It runs big end-user GUI application. This adds various interactions with X and the applications has its own heavy startup cost, it all also adds fuzziness to the results. (3) When we look at global GNU/Linux market its end user deployment (*Office) is not relevant, server side execution matters. = It all seems to me as intentionally chosen just to prove prelink gain is not measurable. Therefore I made IMO a more real world measurement with: time gdb/configure ... kidding Hopefully, you are not building fresh GCC (and other big applications) on your *production* servers. If yes, we need to have a private talk ;) /kidding To make clear why such test matters. Running configure scripts has become the major builds bottleneck in recent days as it cannot be parallelized on multicore and multinode machines. For GDB it takes now 56% (of 1m37.380s) of the whole compilation time. And be sure developers are running configure by orders of magnitude more times than *Office (or even LaTeX). I too face this problem on a regular basis. I guess that a lot of folks will agree that configure just sucks in modern times. Here is my workaround, yum install ccache. It works great for the projects I work on. -- Dhiru -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, 17 Oct 2013, Jan Kratochvil wrote: Workaround of that bug is one line of code, it just has not been accepted yet. And this is the core of the problem. No one has been spending 5 minutes on fixing prelink, yet people have described hours and days of effort wasted because of prelink. If the people who deem prelink useful can't ensure prelink does not cause damage to people with no interest in prelink, the package should go. Therefore I made IMO a more real world measurement with: time gdb/configure Unfortunately I have to admit that does not mean much. I do not know the FIPS prelink issues to be able to talk more about it. rpm uses prelink -y so it already works in most cases and the rare cases can be fixed in prelink. The problem is its maintainer Jakub has more significant work to do nowadays. I agree there remains some work on prelink itself and some packages around to make prelink relevant again I don't mean to pick a fight with you Jan, but you are the only person actively defending prelink right now. When even you reach the above conclusions and cannot put in the time, and the maintainer isn't looking at filed bugs for over a year, the only real answer is to turn prelink into a dead.package for now. Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, Oct 17, 2013 at 7:22 AM, Paul Wouters pwout...@redhat.com wrote: On Thu, 17 Oct 2013, Jan Kratochvil wrote: I agree there remains some work on prelink itself and some packages around to make prelink relevant again I don't mean to pick a fight with you Jan, but you are the only person actively defending prelink right now. When even you reach the above conclusions and cannot put in the time, and the maintainer isn't looking at filed bugs for over a year, the only real answer is to turn prelink into a dead.package for now. There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, 17 Oct 2013 16:28:07 +0200, Josh Boyer wrote: On Thu, Oct 17, 2013 at 7:22 AM, Paul Wouters pwout...@redhat.com wrote: On Thu, 17 Oct 2013, Jan Kratochvil wrote: I agree there remains some work on prelink itself and some packages around to make prelink relevant again I don't mean to pick a fight with you Jan, but you are the only person actively defending prelink right now. When even you reach the above conclusions and cannot put in the time, and the maintainer isn't looking at filed bugs for over a year, the only real answer is to turn prelink into a dead.package for now. There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. This is exactly my opinion which I have already expressed several times in this thread. prelink is useful on some systems (including mine) but I agree it currently does more harm than good for average/default Fedora user. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, Oct 17, 2013 at 07:28:07AM -0700, Josh Boyer wrote: On Thu, Oct 17, 2013 at 7:22 AM, Paul Wouters pwout...@redhat.com wrote: On Thu, 17 Oct 2013, Jan Kratochvil wrote: I agree there remains some work on prelink itself and some packages around to make prelink relevant again I don't mean to pick a fight with you Jan, but you are the only person actively defending prelink right now. When even you reach the above conclusions and cannot put in the time, and the maintainer isn't looking at filed bugs for over a year, the only real answer is to turn prelink into a dead.package for now. There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. Agreed, there's no reason to kill it entirely. Let people opt-in if they wish to install it later understand the cost/benefit tradeoff. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
2013/10/17 Josh Boyer jwbo...@fedoraproject.org On Thu, Oct 17, 2013 at 7:22 AM, Paul Wouters pwout...@redhat.com wrote: On Thu, 17 Oct 2013, Jan Kratochvil wrote: I agree there remains some work on prelink itself and some packages around to make prelink relevant again I don't mean to pick a fight with you Jan, but you are the only person actively defending prelink right now. When even you reach the above conclusions and cannot put in the time, and the maintainer isn't looking at filed bugs for over a year, the only real answer is to turn prelink into a dead.package for now. There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. And if we get this fixed https://bugzilla.redhat.com/show_bug.cgi?id=841434 those who are using prelink can remove it and end up with a sane system -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, 17 Oct 2013, Daniel P. Berrange wrote: There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. Agreed, there's no reason to kill it entirely. Let people opt-in if they wish to install it later understand the cost/benefit tradeoff. How do we make it go away on the installs it currently affects badly? Do we add Conflict: to some packages (eg FIPS capable ones) ? Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, Oct 17, 2013 at 10:54:44AM -0400, Paul Wouters wrote: On Thu, 17 Oct 2013, Daniel P. Berrange wrote: There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. Agreed, there's no reason to kill it entirely. Let people opt-in if they wish to install it later understand the cost/benefit tradeoff. How do we make it go away on the installs it currently affects badly? Do we add Conflict: to some packages (eg FIPS capable ones) ? I don't think the downsides are critical enough to play games to force its removal on existing installs being upgraded. Suffice to just not install it on newly provisioned installs. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Hi, On 10/17/2013 04:54 PM, Paul Wouters wrote: On Thu, 17 Oct 2013, Daniel P. Berrange wrote: There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. Agreed, there's no reason to kill it entirely. Let people opt-in if they wish to install it later understand the cost/benefit tradeoff. How do we make it go away on the installs it currently affects badly? Do we add Conflict: to some packages (eg FIPS capable ones) ? We could change the default /etc/sysconfig/prelink to default to no prelinking, then for people with an unmodified /etc/sysconfig/prelink, this will become the new /etc/sysconfig/prelink and the first time the cronjob runs after the update it will unprelink all binaries (and I hope it is smart enough to not run any more after that). Regards, Hans -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, 17 Oct 2013, Hans de Goede wrote: We could change the default /etc/sysconfig/prelink to default to no prelinking, then for people with an unmodified /etc/sysconfig/prelink, this will become the new /etc/sysconfig/prelink and the first time the cronjob runs after the update it will unprelink all binaries (and I hope it is smart enough to not run any more after that). Ah yes. that works. Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, Oct 17, 2013 at 10:54:44AM -0400, Paul Wouters wrote: There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. Agreed, there's no reason to kill it entirely. Let people opt-in if they wish to install it later understand the cost/benefit tradeoff. How do we make it go away on the installs it currently affects badly? Do we add Conflict: to some packages (eg FIPS capable ones) ? If we wanted to do this, changing 'PRELINKING=no' in /etc/sysconfig/prelink in a package update would do it -- the config file is marked as 'noreplace', but if the file hasn't been edited locally, it would be overwritten with the updated one. Then, at the next cron run, prelink would undo the prelinking. I'm not sure if this is the route we should take, or if installation of the package should default to it being active; just putting this out there as an option. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Once upon a time, Josh Boyer jwbo...@fedoraproject.org said: There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. Well, in this case, I think it should be killed. Having prelink in the distribution implies that it is expected that it should, which means that all the other packages that have to support/work-around/etc. prelink still have to have all those hacks. Maintainers would still be expected to fix problems and such. It creates a burden on other packages, just by being in the distribution. If it doesn't appear to provide a significant benefit, and there's no expectation of support (for some meaning of support), it should go away. IMHO this is one of the differences between a distribution and a random collection of packages. -- Chris Adams li...@cmadams.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, Oct 17, 2013 at 10:36:42AM -0500, Chris Adams wrote: Well, in this case, I think it should be killed. Having prelink in the distribution implies that it is expected that it should, which means that all the other packages that have to support/work-around/etc. prelink still have to have all those hacks. Maintainers would still be expected to fix problems and such. It creates a burden on other packages, just by being in the distribution. I don't think that is necessarily the case. Or, at least, I think that it shouldn't be the case even if it is currently. We've got thousands of packages in the distribution, and requiring this level of burden for other packages from any package which passes review is a path to madness. If it doesn't appear to provide a significant benefit, and there's no expectation of support (for some meaning of support), it should go away. IMHO this is one of the differences between a distribution and a random collection of packages. We need both (although I'd chose a more positive adjective than 'random'), and a way to draw the line. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Thu, Oct 17, 2013 at 5:54 PM, Matthew Miller mat...@fedoraproject.org wrote: If it doesn't appear to provide a significant benefit, and there's no expectation of support (for some meaning of support), it should go away. IMHO this is one of the differences between a distribution and a random collection of packages. We need both (although I'd chose a more positive adjective than 'random'), and a way to draw the line. Users may need both; we are supposedly producing a distribution, i.e. not the other thing. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Chris Adams wrote: Once upon a time, Josh Boyer jwbo...@fedoraproject.org said: There's no reason to kill the package entirely. Some people still want to use it despite the current issues. So just don't install it by default. Reducing everything down to absolutes isn't helpful. Well, in this case, I think it should be killed. The bar for banning software from the distribution is *way* higher than what it takes to remove from installing it by default. Baby steps. -- Rex -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 23:51:18 +0200, Dridi Boukelmoune wrote: $ rpm -V libreoffice-core prelink: /tmp/#prelink#.TZlaPL: Recorded 92 dependencies, now seeing -1 Repeating for the third time in this thread: This is a known prelink Bug and you can find the single line fix/workaround there: -y has false mismatches https://bugzilla.redhat.com/show_bug.cgi?id=666143 The 'rpm -V' output should be also fixed by full prelink of the system: touch /var/lib/prelink/force; /etc/cron.daily/prelink Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/16/2013 12:51 AM, Dridi Boukelmoune wrote: Hi, This is one of these passionate threads I enjoy reading because I usually learn something interesting, and I also enjoy people being passionate about stuff. But this time I'm a bit disappointed about the defaults for Fedora. I'm a developer, and I work with production-like systems (and in some cases production systems) on my day job, so integrity of the system is something very important to me. I was startled when I first read that prelink touches binaries. For I'm too lazy to mount /usr as read-only (actually too lazy to mount it read/write for each yum upgrade), I naively expected binaries would be safe with default settings (assuming no attack). I've run the following commands: $ rpm -V varnish S.5T. c /etc/varnish/varnish.params $ rpm -V firefox $ rpm -V libreoffice-core prelink: /tmp/#prelink#.TZlaPL: Recorded 92 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/gengal.bin prelink: /tmp/#prelink#.3AZudQ: Recorded 87 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libavmedialo.so prelink: /tmp/#prelink#.9xDUuT: Recorded 16 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libbasegfxlo.so [...] Obviously, I'm ok with varnish being touched, I've changed something in the configuration. I'm also relieved that firefox's clean, because I use it heavily on a day-to-day basis. But this is rather disturbing to see prelink on rpm's output. Does it mean that rpm *itself* has been touched by prelink ? This sounds critical to me, how can I know that my rpmdb hasn't been corrupted ? Yup, prelink screws up rpm -V pretty badly. Rpm does do know about prelink and does 'prelink -u' to verify the unprelinked binary matches the original digest but all too often prelink -u fails, in which case there's nothing rpm can do about it. As in the above cases and perhaps the more common one: [root@localhost ~]# rpm -Va prelink: /usr/lib/udev/udev-configure-printer: at least one of file's dependencies has changed since prelinking S.?../usr/lib/udev/udev-configure-printer .M.../var/lib/nfs/rpc_pipefs .M.../var/log/journal prelink: /usr/bin/eog: at least one of file's dependencies has changed since prelinking S.?../usr/bin/eog [...] This might not be *that* much of an issue on a more stable distro, but given the constant churn of Fedora there's not a day when something hasn't changed, causing rpm -Va (and other security tools) to come up with indeterminate results until the prelink cronjob runs. And then more stuff gets updated etc... Of course an attacker that would gain root access to the system could probably alter the rpmdb to hide what changed on the filesystem, but that's not my point. Silently changing rpmdb to match what changed on the filesystem is that easy as the file digests are buried within headers guarded with digital signatures and further digests over the contents. You basically need to replace the entire package, at which point it will no longer have a Fedora signature and is quite a clear indication that something is not right. - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Wed, Oct 16, 2013 at 7:57 AM, Panu Matilainen pmati...@laiskiainen.org wrote: On 10/16/2013 12:51 AM, Dridi Boukelmoune wrote: Hi, This is one of these passionate threads I enjoy reading because I usually learn something interesting, and I also enjoy people being passionate about stuff. But this time I'm a bit disappointed about the defaults for Fedora. I'm a developer, and I work with production-like systems (and in some cases production systems) on my day job, so integrity of the system is something very important to me. I was startled when I first read that prelink touches binaries. For I'm too lazy to mount /usr as read-only (actually too lazy to mount it read/write for each yum upgrade), I naively expected binaries would be safe with default settings (assuming no attack). I've run the following commands: $ rpm -V varnish S.5T. c /etc/varnish/varnish.params $ rpm -V firefox $ rpm -V libreoffice-core prelink: /tmp/#prelink#.TZlaPL: Recorded 92 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/gengal.bin prelink: /tmp/#prelink#.3AZudQ: Recorded 87 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libavmedialo.so prelink: /tmp/#prelink#.9xDUuT: Recorded 16 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libbasegfxlo.so [...] Obviously, I'm ok with varnish being touched, I've changed something in the configuration. I'm also relieved that firefox's clean, because I use it heavily on a day-to-day basis. But this is rather disturbing to see prelink on rpm's output. Does it mean that rpm *itself* has been touched by prelink ? This sounds critical to me, how can I know that my rpmdb hasn't been corrupted ? Yup, prelink screws up rpm -V pretty badly. Rpm does do know about prelink and does 'prelink -u' to verify the unprelinked binary matches the original digest but all too often prelink -u fails, in which case there's nothing rpm can do about it. As in the above cases and perhaps the more common one: [root@localhost ~]# rpm -Va prelink: /usr/lib/udev/udev-configure-printer: at least one of file's dependencies has changed since prelinking S.?../usr/lib/udev/udev-configure-printer .M.../var/lib/nfs/rpc_pipefs .M.../var/log/journal prelink: /usr/bin/eog: at least one of file's dependencies has changed since prelinking S.?../usr/bin/eog [...] This might not be *that* much of an issue on a more stable distro, but given the constant churn of Fedora there's not a day when something hasn't changed, causing rpm -Va (and other security tools) to come up with indeterminate results until the prelink cronjob runs. And then more stuff gets updated etc... Of course an attacker that would gain root access to the system could probably alter the rpmdb to hide what changed on the filesystem, but that's not my point. Silently changing rpmdb to match what changed on the filesystem is that easy s/is/isn't/ ? as the file digests are buried within headers guarded with digital signatures and further digests over the contents. You basically need to replace the entire package, at which point it will no longer have a Fedora signature and is quite a clear indication that something is not right. I said probably because I didn't know. I'd assume a tight security on such a critical package. But I have to confess I don't check that all my packages are signed that often. And rpm -V doesn't warn me about home-made packages not being signed. So a package losing its signature in the database could easily stay unnoticed (unless there are other mechanisms). - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Wed, Oct 16, 2013 at 7:45 AM, Jan Kratochvil jan.kratoch...@redhat.com wrote: On Tue, 15 Oct 2013 23:51:18 +0200, Dridi Boukelmoune wrote: $ rpm -V libreoffice-core prelink: /tmp/#prelink#.TZlaPL: Recorded 92 dependencies, now seeing -1 Repeating for the third time in this thread: This is a known prelink Bug and you can find the single line fix/workaround there: -y has false mismatches https://bugzilla.redhat.com/show_bug.cgi?id=666143 I understand, but what bothers me isn't the prelink bug but prelink itself being installed by default (for what it does regardless of the bug). The 'rpm -V' output should be also fixed by full prelink of the system: touch /var/lib/prelink/force; /etc/cron.daily/prelink Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains [summary]
On Wed, 16 Oct 2013 09:20:02 +0200, Dridi Boukelmoune wrote: I understand, but what bothers me isn't the prelink bug but prelink itself being installed by default (for what it does regardless of the bug). What exactly bothers you? It (generally) speeds up programs startup. As a summary I see prelink has some bugs: * -y has false mismatches: https://bugzilla.redhat.com/show_bug.cgi?id=666143 * %preun does not unprelink: https://bugzilla.redhat.com/show_bug.cgi?id=841434 * It has a bug that in some cases it slows down the startup (seen on mplayer). https://lists.fedoraproject.org/pipermail/devel/2013-October/190274.html * It possibly should disable itself to run on very low memory systems as there can be running multiple copies of prelinked/unprelinked binaries. Plus prelink is affected by bugs in other packages: * cron: It should not run cron.daily(+weekly...) if the user is not idle or if the system is running on battery * systemd should restart daemons which are updated. For example openssh-server restarts itself in %postuninstall but not all packages do so. * tripwire/rkhunter/...: System verificators should use 'prelink -y'. * FIPS: It should unprelink the whole system if it needs it that way. There is remaining security request to have all binaries PIE. IIUC it is for the case of untrusted files stored at local filesystem accessed by for example gzip where exploits could be reduced by having for example even gzip PIE. I do not find it worth the small performance hit but people may disagree. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/2013 10:01 PM, Jan Kratochvil wrote: I have tested a possibly real world script: rm *;sync;(time for i in `seq 0 89`;do mplayer /dev/null -nosound -vo png -ss $i -endpos 0 video.mp4;mv 0001.png $i.png;done) 21|grep ^real|tee -a /tmp/times without prelink: 6.220s 6.212s 6.218s 7.463s 6.277s 6.216s 6.203s 6.255s 6.209s 6.205s with prelink: 6.560s 6.496s 6.501s 6.530s 6.583s 6.616s 6.511s 6.498s 6.489s 6.915s Uhm, I'm not actually sure which one is better. Prelinking appears slower, but seems to offer more consistent performance. If I interpret the result of R's t-test correctly, the difference is hardly significant (the p-value is rather high): without_prelink - c(6.220, 6.212, 6.218, 7.463, 6.277, 6.216, 6.203, 6.255, 6.209, 6.205) with_prelink - c(6.560, 6.496, 6.501, 6.530, 6.583, 6.616, 6.511, 6.498, 6.489, 6.915) t.test(without_prelink, with_prelink) Welch Two Sample t-test data: without_prelink and with_prelink t = -1.7004, df = 10.905, p-value = 0.1174 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -0.50988354 0.06568354 sample estimates: mean of x mean of y 6.34786.5699 -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/16/2013 10:17 AM, Dridi Boukelmoune wrote: On Wed, Oct 16, 2013 at 7:57 AM, Panu Matilainen pmati...@laiskiainen.org wrote: On 10/16/2013 12:51 AM, Dridi Boukelmoune wrote: Hi, This is one of these passionate threads I enjoy reading because I usually learn something interesting, and I also enjoy people being passionate about stuff. But this time I'm a bit disappointed about the defaults for Fedora. I'm a developer, and I work with production-like systems (and in some cases production systems) on my day job, so integrity of the system is something very important to me. I was startled when I first read that prelink touches binaries. For I'm too lazy to mount /usr as read-only (actually too lazy to mount it read/write for each yum upgrade), I naively expected binaries would be safe with default settings (assuming no attack). I've run the following commands: $ rpm -V varnish S.5T. c /etc/varnish/varnish.params $ rpm -V firefox $ rpm -V libreoffice-core prelink: /tmp/#prelink#.TZlaPL: Recorded 92 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/gengal.bin prelink: /tmp/#prelink#.3AZudQ: Recorded 87 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libavmedialo.so prelink: /tmp/#prelink#.9xDUuT: Recorded 16 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libbasegfxlo.so [...] Obviously, I'm ok with varnish being touched, I've changed something in the configuration. I'm also relieved that firefox's clean, because I use it heavily on a day-to-day basis. But this is rather disturbing to see prelink on rpm's output. Does it mean that rpm *itself* has been touched by prelink ? This sounds critical to me, how can I know that my rpmdb hasn't been corrupted ? Yup, prelink screws up rpm -V pretty badly. Rpm does do know about prelink and does 'prelink -u' to verify the unprelinked binary matches the original digest but all too often prelink -u fails, in which case there's nothing rpm can do about it. As in the above cases and perhaps the more common one: [root@localhost ~]# rpm -Va prelink: /usr/lib/udev/udev-configure-printer: at least one of file's dependencies has changed since prelinking S.?../usr/lib/udev/udev-configure-printer .M.../var/lib/nfs/rpc_pipefs .M.../var/log/journal prelink: /usr/bin/eog: at least one of file's dependencies has changed since prelinking S.?../usr/bin/eog [...] This might not be *that* much of an issue on a more stable distro, but given the constant churn of Fedora there's not a day when something hasn't changed, causing rpm -Va (and other security tools) to come up with indeterminate results until the prelink cronjob runs. And then more stuff gets updated etc... Of course an attacker that would gain root access to the system could probably alter the rpmdb to hide what changed on the filesystem, but that's not my point. Silently changing rpmdb to match what changed on the filesystem is that easy s/is/isn't/ ? Whoops, indeed :) as the file digests are buried within headers guarded with digital signatures and further digests over the contents. You basically need to replace the entire package, at which point it will no longer have a Fedora signature and is quite a clear indication that something is not right. I said probably because I didn't know. I'd assume a tight security on such a critical package. But I have to confess I don't check that all my packages are signed that often. And rpm -V doesn't warn me about home-made packages not being signed. So a package losing its signature in the database could easily stay unnoticed (unless there are other mechanisms). Yup, there are plenty of shortcomings in this area, rpm doesn't make it particularly easy (much less automated) to notice such issues. Such change does leave a detectable trace however so its *possible* to notice. - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Resending since the last attempt bounced. On Wed, Oct 16, 2013 at 03:21:31PM +0530, Siddhesh Poyarekar wrote: On Tue, Oct 15, 2013 at 10:27:36PM +0200, Reindl Harald wrote: Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. This isn't totally invalid. I assume that some shell scripts with tight loops are the only thing that actually benefits from prelinking today. People write those, unfortunately. it is - they are *not* loading a lot of dynmaic linked libraries It is spawning a program a bunch of times, which results in program load time becoming all the more relevant. Shell scripts that spawn multiple programs in tight loops are not exactly uncommon, so please don't pretend that they don't exist. [harry@srv-rhsoft:~]$ ldd /usr/bin/bash linux-vdso.so.1 = (0x7fffc9764000) libtinfo.so.5 = /lib64/libtinfo.so.5 (0x7f99b21aa000) libdl.so.2 = /lib64/libdl.so.2 (0x7f99b1fa6000) libc.so.6 = /lib64/libc.so.6 (0x7f99b1be4000) /lib64/ld-linux-x86-64.so.2 (0x7f99b23ee000) I'm attaching a deliberately badly written script which should be fairly representative, alas. I can' benchmark it right now because the system isn't idle, but if someone else wants to have a go at it, be my guest. if you *only* can measure it if your system is *idle* than you have what we called maske dby noise already in this thread and that is *not* significant You haven't understood the point of the statement - a benchmark measurement is reliable only if the 'before' and 'after' measurements are done in identical conditions. In that context, an idle system is the ideal pre-condition. That point has nothing to do with whether the performance improvement is evident when the system is under any amount of load. Siddhesh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/2013 10:04 PM, Florian Weimer wrote: On 10/15/2013 09:10 PM, Chris Adams wrote: Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. This isn't totally invalid. I assume that some shell scripts with tight loops are the only thing that actually benefits from prelinking today. People write those, unfortunately. I'm attaching a deliberately badly written script which should be fairly representative, alas. I can' benchmark it right now because the system isn't idle, but if someone else wants to have a go at it, be my guest. I've no run this script (on Fedora 19 x86_64) with an input file calibrated to run in roughly ten seconds (with prelink), both with and without prelink, each 30 times. R reports this for the wall-clock time: t.test(prelink1noff$V1, noprelink1noff$V1) Welch Two Sample t-test data: prelink1noff$V1 and noprelink1noff$V1 t = -50.8453, df = 57.923, p-value 2.2e-16 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -0.9337006 -0.8629660 sample estimates: mean of x mean of y 10.05300 10.95133 This suggests that there is a statistically significant difference in favor of prelinking, of about 0.9 seconds for a 10 second run. So even in that totally artificial case, we gain very little, considering the trouble that prelink is. -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Wed, 16 Oct 2013 11:56:44 +0200, Florian Weimer wrote: So even in that totally artificial case, we gain very little, considering the trouble that prelink is. After all the discussion I have listed the current known issues: prelink performance gains [summary] https://lists.fedoraproject.org/pipermail/devel/2013-October/190309.html While I even agree currently prelink may cause more harm than good for regular users do you see any issue there which is not fixable? Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Wed, Oct 16, 2013 at 02:42:37PM +0200, Jan Kratochvil wrote: On Wed, 16 Oct 2013 11:56:44 +0200, Florian Weimer wrote: So even in that totally artificial case, we gain very little, considering the trouble that prelink is. After all the discussion I have listed the current known issues: prelink performance gains [summary] https://lists.fedoraproject.org/pipermail/devel/2013-October/190309.html While I even agree currently prelink may cause more harm than good for regular users do you see any issue there which is not fixable? Given how little value prelink adds, I have to question whether it is even worth spending effort on any of those items, vs working on other areas of Fedora which would have greater end user benefit. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Wed, 16 Oct 2013 14:45:00 +0200, Reindl Harald wrote: why waste time and energy to fix things with little to no benefit IIRC compiler team spends 1.5 year to get 1% of performane gain. Here you have almost ready feature with up to ... questionable but it is in a range of percents in some real world cases. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains [summary]
On Wed, Oct 16, 2013 at 9:44 AM, Jan Kratochvil jan.kratoch...@redhat.com wrote: What exactly bothers you? It (generally) speeds up programs startup. As a summary I see prelink has some bugs: * -y has false mismatches: https://bugzilla.redhat.com/show_bug.cgi?id=666143 * %preun does not unprelink: https://bugzilla.redhat.com/show_bug.cgi?id=841434 * It has a bug that in some cases it slows down the startup (seen on mplayer). https://lists.fedoraproject.org/pipermail/devel/2013-October/190274.html * It possibly should disable itself to run on very low memory systems as there can be running multiple copies of prelinked/unprelinked binaries. Plus prelink is affected by bugs in other packages: * cron: It should not run cron.daily(+weekly...) if the user is not idle or if the system is running on battery * systemd should restart daemons which are updated. For example openssh-server restarts itself in %postuninstall but not all packages do so. * tripwire/rkhunter/...: System verificators should use 'prelink -y'. * FIPS: It should unprelink the whole system if it needs it that way. I'm not bothered about prelink requiring extra work, you're right that it is a toolchain optimization work like any other in that regard; but knowing about these bugs and not having enough people interested in fixing them does bother me. There is remaining security request to have all binaries PIE. IIUC it is for the case of untrusted files stored at local filesystem accessed by for example gzip where exploits could be reduced by having for example even gzip PIE. I do not find it worth the small performance hit but people may disagree. FESCo has discussed using PIE for all binaries on x86_64 fairly recently (https://fedorahosted.org/fesco/ticket/1113), and rejected it due to the performance impact; AFAICS removing prelink would _not_ change the rationale used to make that decision. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Wed, 2013-10-16 at 14:58 +0200, Jan Kratochvil wrote: On Wed, 16 Oct 2013 14:45:00 +0200, Reindl Harald wrote: why waste time and energy to fix things with little to no benefit IIRC compiler team spends 1.5 year to get 1% of performane gain. Here you have almost ready feature with up to ... questionable but it is in a range of percents in some real world cases. Jan it seem to me you fail to understand one key difference of a 1% gain in gcc compared to a 1% gain in prelink. prelink affects exclusively startup time, this means it matters only if you keep starting up a program *many* times in a short period of time. This behavior happens only in badly written shell scripts. A 1% gain in gcc means that on average a program will be 1% faster during its lifetime, not just be a smidget faster at startup, so the reasons to work on a 15 speedup on gcc dwarf by importance anything prelink does and is not even worth comparing IMO. They are really apples and oranges. What most people question is whether it makes sense to sepnd so much time on fixing all prelink shortcomings and issues it causes to normal systems when the gains are imperceptible to most common usage scenarios. Personally I think the very minor startup gains prelink offer are not worth the hassle. But let me add this. I am not against the *concept* of prelinking if it didn't have so many annoying side issues. For example if prelink were changed from actually modifying the binary to instead create an extended attribute attached to the binary that contains information to speed up load time, then it would be in a much better position. It wouldn't taint binaries, for example. Anyway atm I think the default should be to switch prelink off going forward, the downsides are too big, perhaps *if* someone steps up and fix all the bugs you pointed out in the bug you opened, then we should reconsider. Simo. -- Simo Sorce * Red Hat, Inc * New York -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 16.10.2013 14:42, schrieb Jan Kratochvil: On Wed, 16 Oct 2013 11:56:44 +0200, Florian Weimer wrote: So even in that totally artificial case, we gain very little, considering the trouble that prelink is. After all the discussion I have listed the current known issues: prelink performance gains [summary] https://lists.fedoraproject.org/pipermail/devel/2013-October/190309.html While I even agree currently prelink may cause more harm than good for regular users do you see any issue there which is not fixable? the real question remains: why waste time and energy to fix things with little to no benefit instead take out the complexity at all which also makes future development easier and future regressions impossible? signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 16.10.2013 14:58, schrieb Jan Kratochvil: On Wed, 16 Oct 2013 14:45:00 +0200, Reindl Harald wrote: why waste time and energy to fix things with little to no benefit IIRC compiler team spends 1.5 year to get 1% of performane gain. Here you have almost ready feature with up to ... questionable but it is in a range of percents in some real world cases. you really do not understand the difference between faster *running* code and some micro-seconds at *startup* of the application? you really do not understand the difference between a *one* time compile-optimizig for the binary shipped to *all* users between the prelinking-overhand on the user side? signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains [summary]
Am 16.10.2013 09:44, schrieb Jan Kratochvil: On Wed, 16 Oct 2013 09:20:02 +0200, Dridi Boukelmoune wrote: I understand, but what bothers me isn't the prelink bug but prelink itself being installed by default (for what it does regardless of the bug). What exactly bothers you? It (generally) speeds up programs startup. As a summary I see prelink has some bugs: * -y has false mismatches: https://bugzilla.redhat.com/show_bug.cgi?id=666143 yes * %preun does not unprelink: https://bugzilla.redhat.com/show_bug.cgi?id=841434 yes * It has a bug that in some cases it slows down the startup (seen on mplayer). https://lists.fedoraproject.org/pipermail/devel/2013-October/190274.html maybe * It possibly should disable itself to run on very low memory systems as there can be running multiple copies of prelinked/unprelinked binaries. it should be disabled everywhere after setup and only opt-in Plus prelink is affected by bugs in other packages: * cron: It should not run cron.daily(+weekly...) if the user is not idle or if the system is running on battery if the user is not idle oh yeah, i am idle while coming back from the kitchen, that does not mean that i am 5 seconds after it has started also be idle in general: if i notice someting that much that it should only run if the machine is idle it is *pretty clear* the any gains this crap in theory has are outbeated by the cron runs systemd should restart daemons which are updated no or at least there should be an opt-out on production systems you do *not* want this on workstations it may not bother For example openssh-server restarts itself in %postuninstall but not all packages do so oh yeah, i remeber a multiple times interrupted dist-upgrade via SSH until someone realized that it is a very bad idea to kill the current session too *WHAT* has this to do with the problem that the daemon is restarted after update, not randomized and *later* prelink steps in while the process is already running - my Firefox is running sometimes for days * tripwire/rkhunter/...: System verificators should use 'prelink -y' the whole prelink idea is broken - period * FIPS: It should unprelink the whole system if it needs it that way. the whole system shoul dnot be prelinked as default - period There is remaining security request to have all binaries PIE. IIUC it is for the case of untrusted files stored at local filesystem accessed by for example gzip where exploits could be reduced by having for example even gzip PIE. I do not find it worth the small performance hit but people may disagree the history of the last 3-5 years if someone is *really* following security lists shows pretty clear that *any* binary was exploitable in the one or other way and so it is unacceptable to make distinctions while for a very low price the whole distribution could be hardended the first step to do so would be ban prelink signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 16.10.2013 15:20, schrieb Jan Kratochvil: On Wed, 16 Oct 2013 15:11:13 +0200, Reindl Harald wrote: you really do not understand the difference between faster *running* code and some micro-seconds at *startup* of the application? You do not understand overall system performance is increased when _every_ startup of a program 100 times a second is faster? and you do not understand that i switched from prelinked system to non prelink-system and carefully looked at the user expierience by opening large applications, login in my desktop or boot the system well, you can tell me calculations of 200 ms here and there multiplied over 24 hours but this does not make sense because you do not start and stop large applications all day long nor do you realize 200 ms now and 200 ms 5 minutes later by starting another application you really do not understand the difference between a *one* time compile-optimizig for the binary shipped to *all* users between the prelinking-overhand on the user side? You still do not understand the nightly prelink run is absolutely zero cost? you still do not understand that it is only a nightly in your small world but not for the majority of users where the damned cron job starts at times with workload because the machine do not run 24 hours a day and the theory it starts only when there is no load remains theory: teh cronjob does *not* and *can not* know if i start 5 seconds later a heavy task *additionally* for machines running 24 hours a day prelink is completly useless crap because you do not close all your daily work application if you don't shutdown the machine and since the application is running at the next morning there is *no* startup which could be boosted at all signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Wed, Oct 16, 2013 at 09:57:37AM +0300, Panu Matilainen wrote: Yup, prelink screws up rpm -V pretty badly. To me, this line _alone_ should end the conversation. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 5:19 AM, Dhiru Kholia dhiru.kho...@gmail.com wrote: Hi, During the development of unSPEC [1] benchmarking suite, I made some interesting observations regarding prelink. - Here are some measurements (for LibreOffice [2] loading time in seconds) done using the unSPEC benchmarking suite. These numbers are repeatable and you are encouraged to try unSPEC to do independent validation of these numbers. - hkario (modern SSD based system, cache flushed): (1.816, 1.811, 1.797, 1.827 with prelink), (2.034, 2.042, 2.027, 2.016 without prelink) - hkario (modern SSD based system, cache intact): (2.155, 2.121, 2.101, 2.299 with prelink), (2.311, 2.052, 2.047, 2.037 without prelink) - halfie (T430s): (10.725, 10.095, 10.378, 10.568 with prelink), (8.901, 8.993, 9.075, 9.448, 9.489 without prelink) - danpb (T530): I see basically no measurable difference in times with or without prelink - quite a lot of variation, but all in same ballpark, (8.374, 7.849, 8.457, 7.673, 7.608, 8.031, 8.350, 8.183, 7.381 with prelink), (7.366, 8.009, 7.500, 7.949, 8.208, 8.351, 7.849, without prelink). - For building kernels (using the kernel-bench [3] component of unSPEC suite), prelink saved = 250 ms over the non-prelink environment (which took 1m19.138s). hkario even reports worse performance numbers for the prelink environment. Additionally, we have specialized softwares like ccache and distcc to solve long-compilation-time problems. I wouldn't expect building kernels to be a great thing to use to measure performance benefits of prelink. A kernel build is basically just calling gcc and ld over and over, and those two things themselves have relatively few libraries involved. So your numbers match what I would expect in this case, but I don't think it's really and accurate testcase. Prelink isn't intended to reduce compilation times. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/13 at 05:30am, Josh Boyer wrote: On Tue, Oct 15, 2013 at 5:19 AM, Dhiru Kholia dhiru.kho...@gmail.com wrote: During the development of unSPEC [1] benchmarking suite, I made some interesting observations regarding prelink. ... - For building kernels (using the kernel-bench [3] component of unSPEC suite), prelink saved = 250 ms over the non-prelink environment (which took 1m19.138s). hkario even reports worse performance numbers for the prelink environment. Additionally, we have specialized softwares like ccache and distcc to solve long-compilation-time problems. I wouldn't expect building kernels to be a great thing to use to measure performance benefits of prelink. A kernel build is basically just calling gcc and ld over and over, and those two things themselves have relatively few libraries involved. So your numbers match what I would expect in this case, but I don't think it's really and accurate testcase. Prelink isn't intended to reduce compilation times. Hi Josh, Good points. Please see http://lwn.net/Articles/341244/ page. In particular, Note, also small but frequently used apps benefit. I run gcc etc a lot and like every single saved cycle. I just wanted to quantify these kinds of use-cases too. Does this make sense? -- Dhiru -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 5:55 AM, Dhiru Kholia dhiru.kho...@gmail.com wrote: On 10/15/13 at 05:30am, Josh Boyer wrote: On Tue, Oct 15, 2013 at 5:19 AM, Dhiru Kholia dhiru.kho...@gmail.com wrote: During the development of unSPEC [1] benchmarking suite, I made some interesting observations regarding prelink. ... - For building kernels (using the kernel-bench [3] component of unSPEC suite), prelink saved = 250 ms over the non-prelink environment (which took 1m19.138s). hkario even reports worse performance numbers for the prelink environment. Additionally, we have specialized softwares like ccache and distcc to solve long-compilation-time problems. I wouldn't expect building kernels to be a great thing to use to measure performance benefits of prelink. A kernel build is basically just calling gcc and ld over and over, and those two things themselves have relatively few libraries involved. So your numbers match what I would expect in this case, but I don't think it's really and accurate testcase. Prelink isn't intended to reduce compilation times. Hi Josh, Good points. Please see http://lwn.net/Articles/341244/ page. In particular, Note, also small but frequently used apps benefit. I run gcc etc a lot and like every single saved cycle. I just wanted to quantify these kinds of use-cases too. Does this make sense? Sure, if you take the save every cycle I can approach. It's all relative to your point of view. I was just noting that a common Fedora user wouldn't necessarily expect prelink to help with compilation times. Even a crazy kernel developer like me doesn't expect that. I can be very impatient when it comes to dealing with machines, but even I can't notice 250ms ;). josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013, Dhiru Kholia wrote: In short, we could not distinguish the performance gains of prelink over the background noise in many (or even most) cases. So, I was wondering if you are aware of any use-cases where prelink provides measurable benefits. In would be awesome if you could run unSPEC on your systems and report back the numbers. Please take prelink behind the barn and shoot it. Thanks. Every year people point out how obsoleted it is, yet we can't get seem to rid of it. The amount of time I've wasted on prelink in combination with FIPS is something another decade of prelink speed gains isn't going to compensate me for. I think /etc/prelink.conf.d along with the cron job and its sysconfig override is an overengineered solution (and ugly because any blacklist package has to own the directory of prelink too) How can we have this discussion? We have had reports of numbers showing no real gain. We know it affects ASLR negatively and complicates FIPS engineering. Why haven't we at least reached a compromise where prelink is _optional_ and not installed per default? It seems currently to be part of the fedora live install (and the rhel workstation installs) Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 10:14:13AM -0400, Paul Wouters wrote: On Tue, 15 Oct 2013, Dhiru Kholia wrote: In short, we could not distinguish the performance gains of prelink over the background noise in many (or even most) cases. So, I was wondering if you are aware of any use-cases where prelink provides measurable benefits. In would be awesome if you could run unSPEC on your systems and report back the numbers. Please take prelink behind the barn and shoot it. Thanks. Every year people point out how obsoleted it is, yet we can't get seem to rid of it. The amount of time I've wasted on prelink in combination with FIPS is something another decade of prelink speed gains isn't going to compensate me for. I think /etc/prelink.conf.d along with the cron job and its sysconfig override is an overengineered solution (and ugly because any blacklist package has to own the directory of prelink too) How can we have this discussion? We have had reports of numbers showing no real gain. We know it affects ASLR negatively and complicates FIPS engineering. Why haven't we at least reached a compromise where prelink is _optional_ and not installed per default? It seems currently to be part of the fedora live install (and the rhel workstation installs) This mail is just a pre-cursor to proposing it be removed. When prelink was added by default, the justification was the performance benefits to startup. To justify removing it, we just need to collect data to show that those performance benefits no longer exist, with current hardware and software combination in Fedora. That is what this email thread is seeking to confirm. So assuming no one comes forward to show some incredible benefit(s) of prelink, a proposal to kill it by default will surely follow. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 16:21:01 +0200, Daniel P. Berrange wrote: To justify removing it, we just need to collect data to show that those performance benefits no longer exist, with current hardware and software combination in Fedora. That is what this email thread is seeking to confirm. There is missing some statistics behind it such as deviation computation. And 4 tests/numbers seem to be too few to compute any real statistics. Some numbers in the initial mail even show as if prelink is slowing down the startup. Technically I do not understand how it can happen and IMHO rather the numbers are bogus. If there is a proven performance degradation it would be good to know the reason. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 04:54:16PM +0200, Jan Kratochvil wrote: To justify removing it, we just need to collect data to show that those performance benefits no longer exist, with current hardware and software combination in Fedora. That is what this email thread is seeking to confirm. There is missing some statistics behind it such as deviation computation. And 4 tests/numbers seem to be too few to compute any real statistics. But, since prelink presents other problems on its own, I think the bar is pretty low here. Even though prelink is currently in, the burden of proof should be on demonstrating its continued usefulness, and the threshold for that should be be higher than marginal noise. When in doubt, let's go for smaller and more simple. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 16:59:59 +0200, Daniel P. Berrange wrote: I wouldn't read that as saying that prelink is slowing down startup, rather that the benefit of prelink is so small as to be indistinguishable from the background noise. That's the problem we even disagree how to read the numbers. There aren't enough numbers (+their stats). Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 17:04:05 +0200, Matthew Miller wrote: But, since prelink presents other problems on its own, Prelinked system is a good test for tools like GDB, elfutils and others they can properly handle the displacements of sections/segments. This is something that ELF does not forbid so tools no longer compatible with non-zero file address base may have problems in future if someone wants to use them that way for example on some embedded system. But another opinion can be s/he should fix them her/himself if s/he needs such a feature. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 05:11:52PM +0200, Jan Kratochvil wrote: On Tue, 15 Oct 2013 16:59:59 +0200, Daniel P. Berrange wrote: I wouldn't read that as saying that prelink is slowing down startup, rather that the benefit of prelink is so small as to be indistinguishable from the background noise. That's the problem we even disagree how to read the numbers. There aren't enough numbers (+their stats). Well if no one can come up with a way to produce numbers that clearly demonstrate that prelink is useful we should ditch it, regardless due to the complexity / downsides it causes. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 04:54:16PM +0200, Jan Kratochvil wrote: On Tue, 15 Oct 2013 16:21:01 +0200, Daniel P. Berrange wrote: To justify removing it, we just need to collect data to show that those performance benefits no longer exist, with current hardware and software combination in Fedora. That is what this email thread is seeking to confirm. There is missing some statistics behind it such as deviation computation. And 4 tests/numbers seem to be too few to compute any real statistics. Some numbers in the initial mail even show as if prelink is slowing down the startup. Technically I do not understand how it can happen and IMHO rather the numbers are bogus. If there is a proven performance degradation it would be good to know the reason. I wouldn't read that as saying that prelink is slowing down startup, rather that the benefit of prelink is so small as to be indistinguishable from the background noise. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
the prelink default should be banned from the distribution because this is always the excuse not activate hardening-flags for packages because PIE binaries are not prelinked and people still insist it brings great performance gains which is mostly not true Am 15.10.2013 14:19, schrieb Dhiru Kholia: During the development of unSPEC [1] benchmarking suite, I made some interesting observations regarding prelink. - Here are some measurements (for LibreOffice [2] loading time in seconds) done using the unSPEC benchmarking suite. These numbers are repeatable and you are encouraged to try unSPEC to do independent validation of these numbers. - hkario (modern SSD based system, cache flushed): (1.816, 1.811, 1.797, 1.827 with prelink), (2.034, 2.042, 2.027, 2.016 without prelink) - hkario (modern SSD based system, cache intact): (2.155, 2.121, 2.101, 2.299 with prelink), (2.311, 2.052, 2.047, 2.037 without prelink) - halfie (T430s): (10.725, 10.095, 10.378, 10.568 with prelink), (8.901, 8.993, 9.075, 9.448, 9.489 without prelink) - danpb (T530): I see basically no measurable difference in times with or without prelink - quite a lot of variation, but all in same ballpark, (8.374, 7.849, 8.457, 7.673, 7.608, 8.031, 8.350, 8.183, 7.381 with prelink), (7.366, 8.009, 7.500, 7.949, 8.208, 8.351, 7.849, without prelink). - For building kernels (using the kernel-bench [3] component of unSPEC suite), prelink saved = 250 ms over the non-prelink environment (which took 1m19.138s). hkario even reports worse performance numbers for the prelink environment. Additionally, we have specialized softwares like ccache and distcc to solve long-compilation-time problems. In short, we could not distinguish the performance gains of prelink over the background noise in many (or even most) cases. So, I was wondering if you are aware of any use-cases where prelink provides measurable benefits. In would be awesome if you could run unSPEC on your systems and report back the numbers. unSPEC is easy to use and doesn't take much time (or steps) to run. For more information, please see the following links. References: [1] https://github.com/kholia/unSPEC [2] https://github.com/kholia/unSPEC/tree/master/LibreOffice [3] https://github.com/kholia/unSPEC/tree/master/kernel-bench signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/13 at 05:11pm, Jan Kratochvil wrote: On Tue, 15 Oct 2013 16:59:59 +0200, Daniel P. Berrange wrote: I wouldn't read that as saying that prelink is slowing down startup, rather that the benefit of prelink is so small as to be indistinguishable from the background noise. That's the problem we even disagree how to read the numbers. There aren't enough numbers (+their stats). Jan, Yes, the numbers I posted have zero scientific value and are simply crude measurements. In spite of this fact, I believe that they are enough to demonstrate that prelink is not resulting in any big gains anymore. Even if these numbers don't demonstrate that, they should *at least* make us doubt prelink's relevance. I am going to quote Matthew on this, Even though prelink is currently in, the burden of proof should be on demonstrating its continued usefulness, and the threshold for that should be be higher than marginal noise. When in doubt, let's go for smaller and more simple. unSPEC is super easy to run and you are encouraged to get all the numbers and stats you need. -- Dhiru -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 18:27:23 +0200, Dhiru Kholia wrote: In spite of this fact, I believe that they are enough to demonstrate that prelink is not resulting in any big gains anymore. Nobody says prelink brings _big_ gains. It is just a negligible performance and negligible battery optimization nowadays. I just do not understand why to give up on that negligible optimization when it brings no disadvantages. The disagreement here is whether it brings some disadvantages or not. So the discussion should rather be if the average (default) user faces the claimed disadvantages or not (*), and therefore whether prelink should be installed by default or not. (*) I find there for example a problem that nightly prelink will run even if you are currently running on battery. But that affects more cron scripts and it is a cron bug, not prelink bug. unSPEC is super easy to run and you are encouraged to get all the numbers and stats you need. I intentionally do not post the numbers but it seems to me prelink does bring negligible performance (and battery) gain for LibreOffice on my system. It is just difficult to measure if you do not trust LD_DEBUG=statistics. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 19:32, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 18:27:23 +0200, Dhiru Kholia wrote: In spite of this fact, I believe that they are enough to demonstrate that prelink is not resulting in any big gains anymore. Nobody says prelink brings _big_ gains. It is just a negligible performance and negligible battery optimization nowadays. I just do not understand why to give up on that negligible optimization when it brings no disadvantages. The disagreement here is whether it brings some disadvantages or not. So the discussion should rather be if the average (default) user faces the claimed disadvantages or not (*), and therefore whether prelink should be installed by default or not. * look at the amount of updates and how they hit prelinked libraries until prelink ran again * look at the lsof | grep DEL | grep /usr output caused by prelink * look at the wasted cycles of running prelink itself and compare to the gain in the past on notebooks i hated prelink and god bless the maintainer which removed the prelink-require out of rkhunter which was pervert most of the time i noticed the weak performance while prelink ran between that i got alarmed all the time by rkhunter-notifies *because* i should prelink this and that file hence - at the end of the day prelink itself consumed more CPU and did more harm as it ever could have gained performance signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 19:42:25 +0200, Reindl Harald wrote: * look at the amount of updates and how they hit prelinked libraries until prelink ran again * look at the lsof | grep DEL | grep /usr output caused by prelink Sorry I do not see what disadvantage is it? * look at the wasted cycles of running prelink itself and compare to the gain the cycles of prelink happen during night when nobody waits for anything. The gain happens when user waits until the program starts. in the past on notebooks i hated prelink and god bless the maintainer which removed the prelink-require out of rkhunter which was pervert most of the time i noticed the weak performance while prelink ran between that i got alarmed all the time by rkhunter-notifies *because* i should prelink this and that file Sorry I do not see how is rkhunter related and what is the disadvantage. If you mean that prelink ever runs while you sit at the computer then it is a bug in cron, not in prelink. /etc/cron.daily/mlocate has a similar problem. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 2013-10-15 at 19:32 +0200, Jan Kratochvil wrote: On Tue, 15 Oct 2013 18:27:23 +0200, Dhiru Kholia wrote: In spite of this fact, I believe that they are enough to demonstrate that prelink is not resulting in any big gains anymore. Nobody says prelink brings _big_ gains. It is just a negligible performance and negligible battery optimization nowadays. I just do not understand why to give up on that negligible optimization when it brings no disadvantages. Prelink does big disadvantages, otherwise nobody would care. One is about security, as it negates randomization of addresses, modification of binaries in itself is pretty perverse to gain just imperceptible performance gains. Many tools need to juggle the fact these binaries have been changed, and make checkers more complex and prone to faults. In general prelink makes things more complex for negligible gains, its worth is highly questionable. The disagreement here is whether it brings some disadvantages or not. I just hope you are not saying that there is a doubt there are disadvantages. The real question here is whether advantages supersede disadvantges, and given the only advantage seem to be performance and it is lost in noise, I hardly see how the advantages are enough to justify using it these days. Simo. -- Simo Sorce * Red Hat, Inc * New York -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: On Tue, 15 Oct 2013 19:42:25 +0200, Reindl Harald wrote: * look at the amount of updates and how they hit prelinked libraries until prelink ran again * look at the lsof | grep DEL | grep /usr output caused by prelink Sorry I do not see what disadvantage is it? If you install updates, reboot, and log in, you are running non-prelinked binaries/libraries. If you don't log out (just lock screen or suspend for example), when you next use the system after prelink has run, new programs will use the prelinked bins/libs. Now you are wasting a chunk of RAM, as it can't be shared between non-prelinked and prelinked bins/libs. -- Chris Adams li...@cmadams.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 19:50:44 +0200, Simo Sorce wrote: Many tools need to juggle the fact these binaries have been changed, and make checkers more complex and prone to faults. So let's build the whole system with -O0 and we can throw away most of compilers and half of debuggers, which are all needlessly complex and prone to faults due to -O2. Do we want to build simple system or good system? In general prelink makes things more complex for negligible gains, its worth is highly questionable. I am aware of it, I have spent a lot of time making tools prelink compatible. But even compilers have very complex parts for negligible gains. I just hope you are not saying that there is a doubt there are disadvantages. I really have not yet seen any valid one. The real question here is whether advantages supersede disadvantges, and given the only advantage seem to be performance and it is lost in noise, I would not say it is lost in noise but let's say it is not big. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 19:54:21 +0200, Chris Adams wrote: Now you are wasting a chunk of RAM, as it can't be shared between non-prelinked and prelinked bins/libs. OK, yes. I believe with RAM prices and therefore RAM sizes nowadays you will still have overall better system performance with prelink. I understand you may find a machine low on RAM where the overall performance with prelink will be lower. That is a special case, not an average user. You can uninstall prelink on such underpowered machine. I have no numbers to back my performance guess above. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 10:56 AM, Jan Kratochvil jan.kratoch...@redhat.com wrote: On Tue, 15 Oct 2013 19:50:44 +0200, Simo Sorce wrote: Many tools need to juggle the fact these binaries have been changed, and make checkers more complex and prone to faults. So let's build the whole system with -O0 and we can throw away most of compilers and half of debuggers, which are all needlessly complex and prone to faults due to -O2. Do we want to build simple system or good system? In general prelink makes things more complex for negligible gains, its worth is highly questionable. I am aware of it, I have spent a lot of time making tools prelink compatible. But even compilers have very complex parts for negligible gains. I just hope you are not saying that there is a doubt there are disadvantages. I really have not yet seen any valid one. The real question here is whether advantages supersede disadvantges, and given the only advantage seem to be performance and it is lost in noise, I would not say it is lost in noise but let's say it is not big. The definition of negligible (the word you keep using to describe the performance benefits is: so small or unimportant as to be not worth considering; insignificant. That is exactly the same as lost in noise. Perhaps you want to use some word other than negligible to describe the performance benefits, because right now your argumentation is very confusing. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 7:50 PM, Simo Sorce s...@redhat.com wrote: Prelink does big disadvantages, otherwise nobody would care. One is about security, as it negates randomization of addresses, Most of the security benefit is, AFAICS, not negated by prelink (see https://lists.fedoraproject.org/pipermail/devel/2013-April/181376.html and following). Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 19:54:15 +0200, Reindl Harald wrote: have fun in distinct between prelink caused needs-restarting or real This is a bug of update system it does not know if an updated service needs restarting or not. your notebooks are running 24 hours a day? really? OT: Yes, really. I use it with remote display+keyboard as a workstation. and try what happens if there where some updates and prelink did not run before the next rkhunter startip - you get alarm mails This is a known prelink Bug: -y has false mismatches https://bugzilla.redhat.com/show_bug.cgi?id=666143 Just the evil Fedora Bugs autoclose has already closed it but it is still valid. I have provided even a single line fix there. there are people wich shut down or suspend machines when they are not in use how do you imagine the cronjob run while they are not in use for this *majority* of users? These users really should uninstall prelink as they cannot use it. BTW all my servers run 24x7. Regards, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 2013-10-15 at 19:56 +0200, Jan Kratochvil wrote: On Tue, 15 Oct 2013 19:50:44 +0200, Simo Sorce wrote: Many tools need to juggle the fact these binaries have been changed, and make checkers more complex and prone to faults. So let's build the whole system with -O0 and we can throw away most of compilers and half of debuggers, which are all needlessly complex and prone to faults due to -O2. Do we want to build simple system or good system? This comparison makes no sense, and you know it. If you want to negate others arguments so not to have to address them suit yourself. Simo. -- Simo Sorce * Red Hat, Inc * New York -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013, Jan Kratochvil wrote: I just do not understand why to give up on that negligible optimization when it brings no disadvantages. Because you did not my previous email? - complexity - complicated prelink blacklists - complicated cron job exclusion with sysconfig - FIPS foot-bullets - reduced alsr Other people added: - battery drain (i dont care if its cron or not, without prelink no drain) - sluggish systems when prelink is updating - additional ram use when logged in for a long time So far you seem to say those are not prelink bugs. Just the FIPS issue for me (speaking as a member of the Red Hat Security Group) is enough to say that if the gains are too small to measure, that it's time for Ocam's Razor and not make our systems needlesly complex. Furthermore, in the past I've indicated that we should have support for systems booted in FIPS mode with fips=1, where though libraries and programs that could not be prelinked should be unprelinked, as the sysadmin specifically told us (via fips=1) that they value security over speed gains) prelink has served us in the past. It's time to let it go. Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 20:04, schrieb Miloslav Trmač: On Tue, Oct 15, 2013 at 7:50 PM, Simo Sorce s...@redhat.com wrote: Prelink does big disadvantages, otherwise nobody would care. One is about security, as it negates randomization of addresses, Most of the security benefit is, AFAICS, not negated by prelink (see https://lists.fedoraproject.org/pipermail/devel/2013-April/181376.html and following). try to understand what you read there * update installed * prelink doe *one time* randomization until the next update/full prelink there is no randomization because prelink a lot of binaries are not PIE code * update * reboot * applications are running * prelink does the one time randomization * the apps are still running and not randomized at all signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 19:49, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 19:42:25 +0200, Reindl Harald wrote: * look at the amount of updates and how they hit prelinked libraries until prelink ran again * look at the lsof | grep DEL | grep /usr output caused by prelink Sorry I do not see what disadvantage is it? have fun in distinct between prelink caused needs-restarting or real * look at the wasted cycles of running prelink itself and compare to the gain the cycles of prelink happen during night when nobody waits for anything. The gain happens when user waits until the program starts. your notebooks are running 24 hours a day? really? in the past on notebooks i hated prelink and god bless the maintainer which removed the prelink-require out of rkhunter which was pervert most of the time i noticed the weak performance while prelink ran between that i got alarmed all the time by rkhunter-notifies *because* i should prelink this and that file Sorry I do not see how is rkhunter related and what is the disadvantage. well, read what rkhunter does it verifies file integrity wonderful idea have a software default touching the binaries and try what happens if there where some updates and prelink did not run before the next rkhunter startip - you get alarm mails If you mean that prelink ever runs while you sit at the computer then it is a bug in cron, not in prelink. /etc/cron.daily/mlocate has a similar problem there are people wich shut down or suspend machines when they are not in use how do you imagine the cronjob run while they are not in use for this *majority* of users? signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 19:54, schrieb Chris Adams: Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: On Tue, 15 Oct 2013 19:42:25 +0200, Reindl Harald wrote: * look at the amount of updates and how they hit prelinked libraries until prelink ran again * look at the lsof | grep DEL | grep /usr output caused by prelink Sorry I do not see what disadvantage is it? If you install updates, reboot, and log in, you are running non-prelinked binaries/libraries. If you don't log out (just lock screen or suspend for example), when you next use the system after prelink has run, new programs will use the prelinked bins/libs. Now you are wasting a chunk of RAM, as it can't be shared between non-prelinked and prelinked bins/libs *and* because prelink is the reason *not* build a lot of packages with position independent code because it can't be prelinked and the excuse is that prelink itself does a little randomization in the case you reboot after updates and login you have pretty well *disabled ASLR at all* until re-login after prelink did it's job from security point of view prelink is nothing else than a nightmare and it stands in the way get the whole distribution hardened for what gain? for one where you can't make a distinct between prelink optimization our typical noise on a multi-threaded operating system? signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 20:07, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 19:54:15 +0200, Reindl Harald wrote: have fun in distinct between prelink caused needs-restarting or real This is a bug of update system it does not know if an updated service needs restarting or not. you can always point with your finger somewhere else the better way is solve the root cause your notebooks are running 24 hours a day? really? OT: Yes, really. I use it with remote display+keyboard as a workstation. as i often enough heard here: the world is not turning around you and try what happens if there where some updates and prelink did not run before the next rkhunter startip - you get alarm mails This is a known prelink Bug: -y has false mismatches https://bugzilla.redhat.com/show_bug.cgi?id=666143 Just the evil Fedora Bugs autoclose has already closed it but it is still valid. I have provided even a single line fix there. you can always point with your finger somewhere else the better way is solve the root cause there are people wich shut down or suspend machines when they are not in use how do you imagine the cronjob run while they are not in use for this *majority* of users? These users really should uninstall prelink as they cannot use it if the majority should uninstall something because they can't use it it must not be active in a default setup BTW all my servers run 24x7 and *there* you do *not* need prelink because the theoretical gain in faster startup does *not matter* so, and now come on and list some *real* benefits, stop to point with your fingers how many people should fix these and and that to work better with prelink, take the wasted time into account if you consider the prelink-benefits and if you can't find *really* good arguments stop to defeat prelink in the default install and agree that it has to be removed from it only the time wasted multiple times a year with discussions and fixing this and that to work with prelink makes it's benefits for gain 200 ms here and there non existent signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 19:56, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 19:50:44 +0200, Simo Sorce wrote: Many tools need to juggle the fact these binaries have been changed, and make checkers more complex and prone to faults. So let's build the whole system with -O0 and we can throw away most of compilers and half of debuggers, which are all needlessly complex and prone to faults due to -O2. Do we want to build simple system or good system? don't get me wrong but *please* take some security education because with -O0 https://fedoraproject.org/wiki/Security_Features?rd=Security/Features would not work *and* prelink works against ASLR In general prelink makes things more complex for negligible gains, its worth is highly questionable. I am aware of it, I have spent a lot of time making tools prelink compatible. But even compilers have very complex parts for negligible gains. and pointing with the finger somewhere and saying there is someting bad makes bad things better? not in the reality! I just hope you are not saying that there is a doubt there are disadvantages. I really have not yet seen any valid one. no - you refused to understand them The real question here is whether advantages supersede disadvantges, and given the only advantage seem to be performance and it is lost in noise, I would not say it is lost in noise but let's say it is not big if benchmarks sometimes are faster without and sometimes with prelink the only conlcusion is that it got lost in the noise signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 20:24:06 +0200, Reindl Harald wrote: Am 15.10.2013 20:07, schrieb Jan Kratochvil: This is a bug of update system it does not know if an updated service needs restarting or not. you can always point with your finger somewhere else the better way is solve the root cause The root cause is really the update system. Why should I look for what daemon to restart by hand? It should happen automatically, it is even a security hole if nightly yum updates a daemon but its old version remains running. IIRC it is some systemd feature in development but I cannot find it now. You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. This is a known prelink Bug: -y has false mismatches https://bugzilla.redhat.com/show_bug.cgi?id=666143 Just the evil Fedora Bugs autoclose has already closed it but it is still valid. I have provided even a single line fix there. you can always point with your finger somewhere else the better way is solve the root cause You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. there are people wich shut down or suspend machines when they are not in use how do you imagine the cronjob run while they are not in use for this *majority* of users? These users really should uninstall prelink as they cannot use it if the majority should uninstall something because they can't use it it must not be active in a default setup If the majority of Fedora users then yes. IMO Fedora is more used on servers but I really do not know the user base. BTW all my servers run 24x7 and *there* you do *not* need prelink because the theoretical gain in faster startup does *not matter* It does matter, I run even large builds there, regression testing, besides that even web server CGIs benefit from it. server is a wide term. so, and now come on and list some *real* benefits, Improved performance. take the wasted time into account If you find software development of performance features a wasted time then I see it really does not make sense to discuss it more. stop to defeat prelink in the default install and agree that it has to be removed from it I do not say anything about prelink in the default install as I have no idea what is the default Fedora user. But it is a useful feature at least on servers and 24x7 running workstations. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 20:25:10 +0200, Paul Wouters wrote: - complexity - complicated prelink blacklists - complicated cron job exclusion with sysconfig You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. - FIPS foot-bullets I really do not care and do not run FIPS. Disable/uninstall prelink for FIPS. - reduced alsr I do not know the details but the network facing daemons are already PIE while most of the binaries - those not facing untrusted data - have no use for PIE. Other people added: - battery drain (i dont care if its cron or not, without prelink no drain) - sluggish systems when prelink is updating This is a bug in cron and/or people not running 24x7 machine should not use prelink at all. - additional ram use when logged in for a long time Answered in: https://lists.fedoraproject.org/pipermail/devel/2013-October/190237.html So far you seem to say those are not prelink bugs. True. Just the FIPS issue for me That's for you but majority of Fedora users do not run in FIPS mode. Furthermore, in the past I've indicated that we should have support for systems booted in FIPS mode with fips=1, where though libraries and programs that could not be prelinked should be unprelinked, as the sysadmin specifically told us (via fips=1) that they value security over speed gains) OK, great, so unprelink the programs. prelink has served us in the past. It's time to let it go. People continually give up on software performance with better hardware. 64-bit systems nowadays run commonly slower than did the 8-bits in 1980s. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. Since you keep repeating this one: -O2 vs. -O0 has a significant performance gain. The message that started this thread indicates that prelink may not have a significant gain anymore. If that's the case, than _any_ effort is not worth the added complexity, overhead, etc. of prelink. -- Chris Adams li...@cmadams.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 20:54:06 +0200, Chris Adams wrote: Since you keep repeating this one: -O2 vs. -O0 has a significant performance gain. The message that started this thread indicates that prelink may not have a significant gain anymore. If that's the case, than _any_ effort is not worth the added complexity, overhead, etc. of prelink. It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done The complexity may affect software development but it should not affect end users. End users then can benefit from the increased performance. Here both developers want to avoid any increased software complexity and also in some cases users suffer due to some lasting bugs (such as the cron issue). Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. The biggest (real world) gain for prelink was always in the large applications that link many libraries, such as LibreOffice. The start of this thread said that tests showed no significant gain anymore, at least in the poster's setup. If that is indeed the case, then there's no reason to keep prelink. -- Chris Adams li...@cmadams.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 20:40, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 20:24:06 +0200, Reindl Harald wrote: Am 15.10.2013 20:07, schrieb Jan Kratochvil: This is a bug of update system it does not know if an updated service needs restarting or not. you can always point with your finger somewhere else the better way is solve the root cause The root cause is really the update system. Why should I look for what daemon to restart by hand? It should happen automatically, it is even a security hole if nightly yum updates a daemon but its old version remains running. IIRC it is some systemd feature in development but I cannot find it now. *you need to restart no-PIE binaries after prelink because otherwise no ASLR* You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. no This is a known prelink Bug: -y has false mismatches https://bugzilla.redhat.com/show_bug.cgi?id=666143 Just the evil Fedora Bugs autoclose has already closed it but it is still valid. I have provided even a single line fix there. you can always point with your finger somewhere else the better way is solve the root cause You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. *stop trolling* there are people wich shut down or suspend machines when they are not in use how do you imagine the cronjob run while they are not in use for this *majority* of users? These users really should uninstall prelink as they cannot use it if the majority should uninstall something because they can't use it it must not be active in a default setup If the majority of Fedora users then yes. IMO Fedora is more used on servers but I really do not know the user base. *lol* Fedora != RHEL/CentOS and that said from one running Fedora in production *but* i am aware that i am *not* the majority in that case BTW all my servers run 24x7 and *there* you do *not* need prelink because the theoretical gain in faster startup does *not matter* It does matter, I run even large builds there, regression testing, besides that even web server CGIs benefit from it. server is a wide term. come on where the numbers? so, and now come on and list some *real* benefits, Improved performance. come on where the numbers? take the wasted time into account If you find software development of performance features a wasted time then I see it really does not make sense to discuss it more. come on where the numbers of the better performance? stop to defeat prelink in the default install and agree that it has to be removed from it I do not say anything about prelink in the default install as I have no idea what is the default Fedora user. *this topic is about have prelink by default installed* But it is a useful feature at least on servers and 24x7 running workstations yeah, where you update, reboot, login and start your Firefox with no ASLR which runs after that 2,3,4,5 days - oh my god what a nonsense to think the 200 ms at startup of the browser is worth the time to defeat it *you have no ASLR at all* in such situations because the on-time-randomize happens after you started your browser which keeps running don't get me wrong but honestly i doubt you are the right person to discuss such technical stuff signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 20:52, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 20:25:10 +0200, Paul Wouters wrote: - complexity - complicated prelink blacklists - complicated cron job exclusion with sysconfig You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. this is just trolling - FIPS foot-bullets I really do not care and do not run FIPS. Disable/uninstall prelink for FIPS. i do not care about prelink enable/install prelink - reduced alsr I do not know the details but the network facing daemons are already PIE while most of the binaries - those not facing untrusted data - have no use for PIE. ASLR is about *untsrusted input data* you *really* think your browser, office, pdf-reader does not act with untrusted input or if that is the case this is representative for the userbase? without ASLR and prelink which is the reason not build PIE and start Firefox directly after updates you have *no randomization at all* if you think that does not matter why do you think ASLR exists So far you seem to say those are not prelink bugs. True. *where are the numbers* proving the benfits? it's faster and it's for performance are no numbers if you defeat something *prove* why or be quiet anything else is *trolling* Just the FIPS issue for me That's for you but majority of Fedora users do not run in FIPS mode. the majority of Fedora users does not care about prelink as well becaus ethey have it only installe dbecause it's default and the performance improvement is *not* that large these days *until you prove* it isa Furthermore, in the past I've indicated that we should have support for systems booted in FIPS mode with fips=1, where though libraries and programs that could not be prelinked should be unprelinked, as the sysadmin specifically told us (via fips=1) that they value security over speed gains) OK, great, so unprelink the programs. OK, great, don't prelink them without a user decided to do so prelink has served us in the past. It's time to let it go. People continually give up on software performance with better hardware. 64-bit systems nowadays run commonly slower than did the 8-bits in 1980s *prove the performance benfit with numbers* only repeat the same again and agin does not make it the truth BTW: 64Bit systems these days are in most cases *faster* because software can use CPU capabilities which did not exist not long ago and some of them are only available in 64bit mode signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 20:54, schrieb Chris Adams: Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: You can always make your software development life more simple by giving up on some useful feature. That -O2 vs. -O0 build is a good comparison. Since you keep repeating this one: -O2 vs. -O0 has a significant performance gain. The message that started this thread indicates that prelink may not have a significant gain anymore. If that's the case, than _any_ effort is not worth the added complexity, overhead, etc. of prelink forget it - he is just trolling and not interested ina technical discussion or numbers proving the gain of prelink otherwise he would prove it or if he can't do so stop defeat prelink because in case of no significant gain it's pretty clear that prelink should no longer be in the default install besides negative impact because: *any* running code with out a real beenfit is *bad* for maintainence and security reasons period signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 21:05, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 20:54:06 +0200, Chris Adams wrote: Since you keep repeating this one: -O2 vs. -O0 has a significant performance gain. The message that started this thread indicates that prelink may not have a significant gain anymore. If that's the case, than _any_ effort is not worth the added complexity, overhead, etc. of prelink. It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done where are the numbers? prove it! and after that start your brain and compare that numbers with the battery wasted by prelinking itself, consider how realistic it is that you start 1000 times a large application which benefits from prelink until it get the next update and *no* don#t come again with but CGI nobody but you is running CGI these days anybody but you is using fast-cgi which does not fire up a new process for each request and so prelink is out of the game signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 21:08:40 +0200, Reindl Harald wrote: Am 15.10.2013 21:05, schrieb Jan Kratochvil: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done where are the numbers? prove it! 33% is the number. It was 2.848s vs. 4.291s, that is 33.63%. and after that start your brain and compare that numbers with the battery wasted by prelinking itself, prelinking never runs on battery, or it is a bug if it does. At least I run notebook 24x7 on AC and only occasionally I take it out with battery, then prelink works perfectly for me as I have described it. nobody but you is running CGI these days anybody but you is using fast-cgi OT: I use mod_perl for majority of my web code. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Once upon a time, Jóhann B. Guðmundsson johan...@gmail.com said: I say we should remove it from the standard comps group thus making it optional to install for people that see some benefit in still using it. I'd actually suggest that prelink be an all-or-nothing. If it isn't in the default install, support for it in the other things that interact with it will bit-rot (and maintainers will have little incentive to fix problems). If it isn't in the default install, it should probably just be removed from the distribution. If it improves performance noticably, then keep it; if it doesn't, why should it be kept at all? It has non-zero support cost outside of the package itself. -- Chris Adams li...@cmadams.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 21:13, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 21:08:40 +0200, Reindl Harald wrote: Am 15.10.2013 21:05, schrieb Jan Kratochvil: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done where are the numbers? prove it! 33% is the number. It was 2.848s vs. 4.291s, that is 33.63%. *lol* 1.44 seconds for *thousand startups* now it's clear why you came up only with the % before asking for numbers and you think after that somebody takes any contribution in this thread coming from you (consider your other trolling too) serious? you really try to tell me that if my machine get slow due prelink is running does not do more harm than 1.44 seconds spread over how many days you need to start a big application 1000 times? and after that start your brain and compare that numbers with the battery wasted by prelinking itself, prelinking never runs on battery, or it is a bug if it does. At least I run notebook 24x7 on AC and only occasionally I take it out with battery, then prelink works perfectly for me as I have described it. so what - and the notebook on AC which get's slow and hot due prelink does not matter for *what* benfit - ah 1.4 seconds - wait 1.4 seconds ina total of 1000 startups i am unsure if i now should laugh or whine nobody but you is running CGI these days anybody but you is using fast-cgi OT: I use mod_perl for majority of my web code and why do you than defeat prelink that much? are you the developer of it or married the developer? signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 05:17:28PM +0200, Jan Kratochvil wrote: On Tue, 15 Oct 2013 17:04:05 +0200, Matthew Miller wrote: But, since prelink presents other problems on its own, Prelinked system is a good test for tools like GDB, elfutils and others they can properly handle the displacements of sections/segments. This is something that ELF does not forbid so tools no longer compatible with non-zero file address base may have problems in future if someone wants to use them that way for example on some embedded system. But another opinion can be s/he should fix them her/himself if s/he needs such a feature. There were similar arguments about ppc support (testing to reveal endian-related bugs) but that didn't stop us from dropping ppc as a primary architecture. I don't think this is an argument that has much precedent. -Toshio pgpz4B_ZySewX.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Hello, On Tue, Oct 15, 2013 at 2:19 PM, Dhiru Kholia dhiru.kho...@gmail.com wrote: - Here are some measurements (for LibreOffice [2] loading time in seconds) done using the unSPEC benchmarking suite. These numbers are repeatable and you are encouraged to try unSPEC to do independent validation of these numbers. - hkario (modern SSD based system, cache flushed): (1.816, 1.811, 1.797, 1.827 with prelink), (2.034, 2.042, 2.027, 2.016 without prelink) (mean 1.813, stdev 0.01245) vs. (mean 2.03, stdev 0.01103) with 4 samples is not statistically significant[1] even though the difference leaps to the eye: So more samples would be required to draw a conclusion. (Verification would be welcome: It's been years since my statistics course.) - halfie (T430s): (10.725, 10.095, 10.378, 10.568 with prelink), (8.901, 8.993, 9.075, 9.448, 9.489 without prelink) This doesn't make sense with what I know about prelink. Yet, similar data have been measured earlier, and IIRC there wasn't any alternative explanation for the result. (One plausible explanation is that what I know about prelink is wrong, of course.) Mirek [1] http://www.quantitativeskills.com/sisa/statistics/t-test.php?mean1=1.813mean2=2.03N1=4N2=4SD1=0.01245SD2=0.01103CI=95CIplus=trueSubmit1=Calculate -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/2013 06:40 PM, Jan Kratochvil wrote: Improved performance. I'm not sure where this is coming from but it looks like people are confusing improved performance with improved startup of applications. And afaik it can trigger false positive with security related applications as well various applications are not compatible with prelink ( is everything that can be and we ship by default compatible? ) . I say we should remove it from the standard comps group thus making it optional to install for people that see some benefit in still using it. JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/2013 07:20 PM, Chris Adams wrote: Once upon a time, Jóhann B. Guðmundssonjohan...@gmail.com said: I say we should remove it from the standard comps group thus making it optional to install for people that see some benefit in still using it. I'd actually suggest that prelink be an all-or-nothing. If it isn't in the default install, support for it in the other things that interact with it will bit-rot (and maintainers will have little incentive to fix problems). Well for the first I'm not so sure all application that can and might benefit from are using it and quite frankly I seriously doubt that based on the many half implementation of things I've come across in the distribution. If it isn't in the default install, it should probably just be removed from the distribution. If it improves performance noticably, then keep it; if it doesn't, why should it be kept at all? It has non-zero support cost outside of the package itself. And from my point as long as people are willing to maintain it ( which should indicate they are actually using it since it would be beneficial to at least them ) we should ship it but we should just make it option to install. Again it does not improve performance it improves startup of application as in the application will not work any faster with it turned on... JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/15/2013 03:20 PM, Reindl Harald wrote: Am 15.10.2013 21:13, schrieb Jan Kratochvil: OT: I use mod_perl for majority of my web code and why do you than defeat prelink that much? are you the developer of it or married the developer? Please keep conversations civil. Resorting to personal attacks is not being awesome to each other. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlJdnbMACgkQeiVVYja6o6OW+gCdGCZ6AEbjh6b8Et8kVWr+ySbM m+8An0zLAbOKM1sjnhd0VPv22RkAXraz =j0o+ -END PGP SIGNATURE- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013, Jan Kratochvil wrote: - FIPS foot-bullets I really do not care and do not run FIPS. Your personal views are irrelevant. You are a package maintainer. When other people care about FIPS, you as a package maintainer should care about playing nicely with FIPS. Disable/uninstall prelink for FIPS. I tried that. I submitted a patch for prelink to un-prelink on de-install in %preun. It has been ignored by you for over a year and I seriously had to restrain myself from not using provenpacker to fix this. https://bugzilla.redhat.com/show_bug.cgi?id=841434 https://bugzilla.redhat.com/show_bug.cgi?id=1019225 Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 21:10:34 +0200, Chris Adams wrote: Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. The goal of this example was to show that in a special case of shell script prelink does bring very significant performance improvements. Real world cases would have expectedly lower but still valid improvement. I have tested a possibly real world script: rm *;sync;(time for i in `seq 0 89`;do mplayer /dev/null -nosound -vo png -ss $i -endpos 0 video.mp4;mv 0001.png $i.png;done) 21|grep ^real|tee -a /tmp/times without prelink: 6.220s 6.212s 6.218s 7.463s 6.277s 6.216s 6.203s 6.255s 6.209s 6.205s with prelink: 6.560s 6.496s 6.501s 6.530s 6.583s 6.616s 6.511s 6.498s 6.489s 6.915s I do not think these numbers need too deep statistical analysis. -=without prelink +=with prelink runtime linker statistics: - total startup time in dynamic loader: 28010379 clock cycles + total startup time in dynamic loader: 34141440 clock cycles - time needed for relocation: 19069899 clock cycles (68.0%) + time needed for relocation: 25601766 clock cycles (74.9%) -number of relative relocations: 73592 +number of relative relocations: 0 - time needed to load objects: 8138088 clock cycles (29.0%) + time needed to load objects: 7724814 clock cycles (22.6%) From some looking at it I do not understand now what happens there. It should be debugged and fixed. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On 10/15/2013 09:10 PM, Chris Adams wrote: Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. This isn't totally invalid. I assume that some shell scripts with tight loops are the only thing that actually benefits from prelinking today. People write those, unfortunately. I'm attaching a deliberately badly written script which should be fairly representative, alas. I can' benchmark it right now because the system isn't idle, but if someone else wants to have a go at it, be my guest. -- Florian Weimer / Red Hat Product Security Team palindromes.sh Description: application/shellscript -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 21:59:13 +0200, Paul Wouters wrote: On Tue, 15 Oct 2013, Jan Kratochvil wrote: Disable/uninstall prelink for FIPS. I tried that. I submitted a patch for prelink to un-prelink on de-install in %preun. It has been ignored by you for over a year and I seriously had to restrain myself from not using provenpacker to fix this. https://bugzilla.redhat.com/show_bug.cgi?id=841434 https://bugzilla.redhat.com/show_bug.cgi?id=1019225 It is sad it happened but I am not prelink maintainer and I have never seen these Bugs. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 09:05:03PM +0200, Jan Kratochvil wrote: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done I hope we can all agree that this is not useful example. The complexity may affect software development but it should not affect end users. End users then can benefit from the increased performance. The complexity affects end uesrs. For example, a few years ago, a bad update to prelink broke everyone's systems. It also affects systems like tripwire, and even rpm -V is more fragile. Most of the concerns raised are end-user (or sysadmin) concerns, in fact. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, 15 Oct 2013 22:12:00 +0200, Matthew Miller wrote: On Tue, Oct 15, 2013 at 09:05:03PM +0200, Jan Kratochvil wrote: i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done I hope we can all agree that this is not useful example. Explained in: https://lists.fedoraproject.org/pipermail/devel/2013-October/190274.html The complexity affects end uesrs. For example, a few years ago, a bad update to prelink broke everyone's systems. We are back at the issue that every feature means more code/complexity. It also affects systems like tripwire, I do not know tripwire but I guess it does not use / it should use 'prelink -y'. and even rpm -V is more fragile. Most of the concerns raised are end-user (or sysadmin) concerns, in fact. rpm -V is still the prelink Bug with know fix/workaround there but never accepted/fixed in the package: -y has false mismatches https://bugzilla.redhat.com/show_bug.cgi?id=666143 Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 10:14:13AM -0400, Paul Wouters wrote: On Tue, 15 Oct 2013, Dhiru Kholia wrote: In short, we could not distinguish the performance gains of prelink over the background noise in many (or even most) cases. So, I was wondering if you are aware of any use-cases where prelink provides measurable benefits. In would be awesome if you could run unSPEC on your systems and report back the numbers. Please take prelink behind the barn and shoot it. Thanks. Every year people point out how obsoleted it is, yet we can't get seem to rid of it. The amount of time I've wasted on prelink in combination with FIPS is something another decade of prelink speed gains isn't going to compensate me for. I think /etc/prelink.conf.d along with the cron job and its sysconfig override is an overengineered solution (and ugly because any blacklist package has to own the directory of prelink too) Yup. Even if prelink made things faster (which it seems it doesn't) it wouldn't compensate for all my time it has wasted. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 02:25:10PM -0400, Paul Wouters wrote: On Tue, 15 Oct 2013, Jan Kratochvil wrote: I just do not understand why to give up on that negligible optimization when it brings no disadvantages. Because you did not my previous email? - complexity - complicated prelink blacklists - complicated cron job exclusion with sysconfig - FIPS foot-bullets - reduced alsr Other people added: - battery drain (i dont care if its cron or not, without prelink no drain) - sluggish systems when prelink is updating - additional ram use when logged in for a long time - randomly breaks OCaml programs - prevents libguestfs AIDE checks from working Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 22:01, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 21:10:34 +0200, Chris Adams wrote: Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. The goal of this example was to show that in a special case of shell script prelink does bring very significant performance improvements. Real world cases would have expectedly lower but still valid improvement this is *not* a example for the real world a typical shell script does *not* link a lot of dynmaic libraries [harry@srv-rhsoft:~]$ ldd /usr/bin/bash linux-vdso.so.1 = (0x7fffc9764000) libtinfo.so.5 = /lib64/libtinfo.so.5 (0x7f99b21aa000) libdl.so.2 = /lib64/libdl.so.2 (0x7f99b1fa6000) libc.so.6 = /lib64/libc.so.6 (0x7f99b1be4000) /lib64/ld-linux-x86-64.so.2 (0x7f99b23ee000) prelink has only a benfit in *large* applications loading *a lot* of libraries and those large applications like FF/TB/LibreOffice and the Desktop are *not* opened and closed hundret times each day so you should first understand first things that you defeat and not come up with proveable wrong argumentations signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 21:55, schrieb Stephen Gallagher: On 10/15/2013 03:20 PM, Reindl Harald wrote: Am 15.10.2013 21:13, schrieb Jan Kratochvil: OT: I use mod_perl for majority of my web code and why do you than defeat prelink that much? are you the developer of it or married the developer? Please keep conversations civil. Resorting to personal attacks is not being awesome to each other. sorry - i tried to find a valid explaination why someone defeats something that hard without any valid argumentation up to start trolling multiple times signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 22:17, schrieb Jan Kratochvil: On Tue, 15 Oct 2013 22:12:00 +0200, Matthew Miller wrote: On Tue, Oct 15, 2013 at 09:05:03PM +0200, Jan Kratochvil wrote: i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done I hope we can all agree that this is not useful example. Explained in: https://lists.fedoraproject.org/pipermail/devel/2013-October/190274.html The complexity affects end uesrs. For example, a few years ago, a bad update to prelink broke everyone's systems. We are back at the issue that every feature means more code/complexity *we* are the whole thread about the question if prelinks preformance gains in thereality are really worth the additional complexity *you* are *now back* to the issue signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Am 15.10.2013 22:04, schrieb Florian Weimer: On 10/15/2013 09:10 PM, Chris Adams wrote: Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. This isn't totally invalid. I assume that some shell scripts with tight loops are the only thing that actually benefits from prelinking today. People write those, unfortunately. it is - they are *not* loading a lot of dynmaic linked libraries [harry@srv-rhsoft:~]$ ldd /usr/bin/bash linux-vdso.so.1 = (0x7fffc9764000) libtinfo.so.5 = /lib64/libtinfo.so.5 (0x7f99b21aa000) libdl.so.2 = /lib64/libdl.so.2 (0x7f99b1fa6000) libc.so.6 = /lib64/libc.so.6 (0x7f99b1be4000) /lib64/ld-linux-x86-64.so.2 (0x7f99b23ee000) I'm attaching a deliberately badly written script which should be fairly representative, alas. I can' benchmark it right now because the system isn't idle, but if someone else wants to have a go at it, be my guest. if you *only* can measure it if your system is *idle* than you have what we called maske dby noise already in this thread and that is *not* significant signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
On Tue, Oct 15, 2013 at 10:27 PM, Reindl Harald h.rei...@thelounge.net wrote: Am 15.10.2013 22:04, schrieb Florian Weimer: On 10/15/2013 09:10 PM, Chris Adams wrote: Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. This isn't totally invalid. I assume that some shell scripts with tight loops are the only thing that actually benefits from prelinking today. People write those, unfortunately. it is - they are *not* loading a lot of dynmaic linked libraries [harry@srv-rhsoft:~]$ ldd /usr/bin/bash linux-vdso.so.1 = (0x7fffc9764000) libtinfo.so.5 = /lib64/libtinfo.so.5 (0x7f99b21aa000) libdl.so.2 = /lib64/libdl.so.2 (0x7f99b1fa6000) libc.so.6 = /lib64/libc.so.6 (0x7f99b1be4000) /lib64/ld-linux-x86-64.so.2 (0x7f99b23ee000) Yes because shell is a real programming language that does not have to start tons of other binaries to do useful stuff ... oh wait. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: prelink performance gains
Hi, This is one of these passionate threads I enjoy reading because I usually learn something interesting, and I also enjoy people being passionate about stuff. But this time I'm a bit disappointed about the defaults for Fedora. I'm a developer, and I work with production-like systems (and in some cases production systems) on my day job, so integrity of the system is something very important to me. I was startled when I first read that prelink touches binaries. For I'm too lazy to mount /usr as read-only (actually too lazy to mount it read/write for each yum upgrade), I naively expected binaries would be safe with default settings (assuming no attack). I've run the following commands: $ rpm -V varnish S.5T. c /etc/varnish/varnish.params $ rpm -V firefox $ rpm -V libreoffice-core prelink: /tmp/#prelink#.TZlaPL: Recorded 92 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/gengal.bin prelink: /tmp/#prelink#.3AZudQ: Recorded 87 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libavmedialo.so prelink: /tmp/#prelink#.9xDUuT: Recorded 16 dependencies, now seeing -1 S.?../usr/lib64/libreoffice/program/libbasegfxlo.so [...] Obviously, I'm ok with varnish being touched, I've changed something in the configuration. I'm also relieved that firefox's clean, because I use it heavily on a day-to-day basis. But this is rather disturbing to see prelink on rpm's output. Does it mean that rpm *itself* has been touched by prelink ? This sounds critical to me, how can I know that my rpmdb hasn't been corrupted ? Of course an attacker that would gain root access to the system could probably alter the rpmdb to hide what changed on the filesystem, but that's not my point. I've removed the prelink package: $ rpm -V libreoffice-core S.5../usr/lib64/libreoffice/program/gengal.bin S.5../usr/lib64/libreoffice/program/gnome-open-url.bin S.5../usr/lib64/libreoffice/program/libavmedialo.so S.5../usr/lib64/libreoffice/program/libbasegfxlo.so S.5../usr/lib64/libreoffice/program/libcanvastoolslo.so [...] Now libreoffice still appears to be (differently) tainted, but rpm doesn't output prelink stuff anymore (which isn't less scary). Don't get me wrong, I really enjoy Fedora on my laptops (and before on VMs) but I have a serious trust issue now: - this is part of the distribution *by default* - it is present and already acts at the very first boot AFAIU - removing it doesn't restore the binaries (I didn't expect it would) - apparently it prevents hardened builds in some cases After three reboots, I can't tell the difference between now and before, but to be fair I haven't really paid attention to the start time of the system and applications such as the ones in libreoffice. In my opinion, if there is no perceived latency, it is irrelevant. It all started as a fun thread, with interesting opinions and arguments, but now I have one question: Are there other packages installed by default that would alter my system ? Best Regards, Dridi PS. I'm a total security noob, I'm just aware of basic stuff On Tue, Oct 15, 2013 at 10:35 PM, drago01 drag...@gmail.com wrote: On Tue, Oct 15, 2013 at 10:27 PM, Reindl Harald h.rei...@thelounge.net wrote: Am 15.10.2013 22:04, schrieb Florian Weimer: On 10/15/2013 09:10 PM, Chris Adams wrote: Once upon a time, Jan Kratochvil jan.kratoch...@redhat.com said: It depends, for example in this case prelink saves 33% of time (and battery): i=0;time while [ $i -lt 1000 ];do /usr/bin/gnome-open --help /dev/null;i=$[$i+1];done Do you really run gnome-open --help 1000 times per reasonable unit of time (or ever)? Please stop using bogus comparisons and highly contrived tests. They do nothing to help your argument. This isn't totally invalid. I assume that some shell scripts with tight loops are the only thing that actually benefits from prelinking today. People write those, unfortunately. it is - they are *not* loading a lot of dynmaic linked libraries [harry@srv-rhsoft:~]$ ldd /usr/bin/bash linux-vdso.so.1 = (0x7fffc9764000) libtinfo.so.5 = /lib64/libtinfo.so.5 (0x7f99b21aa000) libdl.so.2 = /lib64/libdl.so.2 (0x7f99b1fa6000) libc.so.6 = /lib64/libc.so.6 (0x7f99b1be4000) /lib64/ld-linux-x86-64.so.2 (0x7f99b23ee000) Yes because shell is a real programming language that does not have to start tons of other binaries to do useful stuff ... oh wait. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
