Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
* Luca Barbato lu_z...@gentoo.org schrieb: I wonder if that case shouldn't be handled better with an huge ewarn so people concerned would really run it in a benchmark environment, alone. ewarns should be circumvented (make it work w/o them), IMHO. I can imagine I'm not the only person who doenst want to keep an close eye on each single build/merge. In this concrete example, the package's build system is misdesigned. They shouldn't rely on runtime benchmarks for build time decisions in the first place - this is unpredictable. But this is not an issues distros should have to cope with. cu -- - Enrico Weigelt== metux IT service - http://www.metux.de/ - Please visit the OpenSource QM Taskforce: http://wiki.metux.de/public/OpenSource_QM_Taskforce Patches / Fixes for a lot dozens of packages in dozens of versions: http://patches.metux.de/ -
Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
On 04/14/2010 08:46 AM, Justin wrote: There is at least one other example which benefits from singlular build, atlas libs. They run a benchmark suite to create platform specific headers, which is heavily influenced by the system load. So having RESTRICT=parallel would make the emerge more reliable. I wonder if that case shouldn't be handled better with an huge ewarn so people concerned would really run it in a benchmark environment, alone. lu -- Luca Barbato Gentoo/linux http://dev.gentoo.org/~lu_zero
Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
On 04/14/2010 04:12 AM, Zac Medico wrote: Hi everyone, Should we add a RESTRICT=parallel value for ebuilds that can't be built at the same time as other ebuilds? Brian says we need it for things like xorg-server which calls eselect opengl. Maybe we could live w/out calling eselect opengl that way first. The discussion about how to handle two parallel merge in a safer way seems interesting though. lu -- Luca Barbato Gentoo/linux http://dev.gentoo.org/~lu_zero
Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
On Wed, Apr 14, 2010 at 07:45:20AM +0200, Michaaa GGGrny wrote: On Tue, 13 Apr 2010 19:12:08 -0700 Zac Medico zmed...@gentoo.org wrote: Should we add a RESTRICT=parallel value for ebuilds that can't be built at the same time as other ebuilds? Brian says we need it for things like xorg-server which calls eselect opengl. I don't think that's the right solution. In most cases, xorg-server can be built in parallel with stuff which doesn't require/set specific opengl subsystem set. RESTRICT=parallel is basically a big lock that forces building to go down to one specific build/merge job- it's not at all fine grained. That said, I'm not convinced it's worth actually *trying* to be fine grained. Stuff that needs this 'lock', if you look at it from the purely academic angle is broken. The pkgs in question should be buildable without modifying the livefs. From the pragmatic angle, fixing some of those packages is a pretty huge endeavour hence this lock existing. I see no reason to encourage the usage of this lock by making it more fine grained, either. Well, in fact is there _really_ any package which won't work with switched opengl? I guess it's more of a runtime problem that buildtime, and I don't think we really should print out loudly 'libGL has been switched, please do not start OpenGL apps right now'. Runtime and buildtime actually- consider a pkg that is mesa sensitive having those headers/libs switched out mid build. That build likely is going to boom in a rather interesting way. Runtime itself, swapping out the gl resource that is used (going from ati to x11 for building xorg-server) isn't going to make new apps that start happy either. In fact, the best solution in this particular case would be to patch the buildsystem to use Gentoo location for particular OpenGL headers. Academically, you're right, it's the proper solution. That takes a fair amount of time however. More importantly, this issue *will* pop up again elsewhere, meaning we'll have to delay the pkg in question from being in the tree till it meets this higher QA standard. Or we add this functionality, level the restrict, then go and fix the ebuild- pragmatic solution. Disabling parallel emerge would be more of a workaround for the issue, and will influence much more packages than it needs to. And it won't help if user is running multiple emerge calls at the same time. Running multiple emerges in parallel is already a bad idea. The solution for that case is for the new/second emerge to feed the request into the original emerge (or a daemon). Keep in mind if support for multiple emerge invocations was implemented it would still need some RESTRICT=parallel functionality for screwed up pkgs like xorg-server. Fixing multiple emerge invocations still requires fixing RESTRICT=parallel. Another possible workaround is to enable some kind of 'eselect opengl' locking so that another package requiring access to it will wait until our build finishes. But this, of course, would require a quite good solution for maintaining the lock and dropping it whenever build process is aborted/killed. The other thing to recall is that by the time eselect is called, the ebuilds environment may already be localized to settings that eselect controls (LDPATH, that 'currently selected implementation' GL var). My first thought is any locking scheme of that sort is going to be a bigger can of worms. ~brian pgpObsKog41jr.pgp Description: PGP signature
Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
On 14/04/10 04:12, Zac Medico wrote: Hi everyone, Should we add a RESTRICT=parallel value for ebuilds that can't be built at the same time as other ebuilds? Brian says we need it for things like xorg-server which calls eselect opengl. There is at least one other example which benefits from singlular build, atlas libs. They run a benchmark suite to create platform specific headers, which is heavily influenced by the system load. So having RESTRICT=parallel would make the emerge more reliable. justin signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
On Wed, 14 Apr 2010 08:46:15 +0200 Justin j...@gentoo.org wrote: There is at least one other example which benefits from singlular build, atlas libs. They run a benchmark suite to create platform specific headers, which is heavily influenced by the system load. So having RESTRICT=parallel would make the emerge more reliable. And that's another example of working around broken ideas. There is an uncountable number of possibilities of many different factors affecting the system load, and thus the results of the benchmark. Removing one of them would indeed help in many cases but in many other it wouldn't make any difference, additionally slowing down the whole system update process. Please notice that in order to implement that correctly, emerge would have to wait until all previously running emerges are done, and avoid running further ones while the build process is running. While in the case of xorg-server, the 'lock' is needed throughout the whole build process, in your case it is needed only for a short amount of time when the benchmark is being performed --- and the 'real' part of building would unnecessarily block emerge. If this is ever to be implemented, it totally needs to be user-overridable. Else, we'll end up someday like Windows, forcing user to reboot the system and perform the merge on a dedicated runlevel. -- Best regards, Michał Górny http://mgorny.alt.pl xmpp:mgo...@jabber.ru signature.asc Description: PGP signature
[gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
Hi everyone, Should we add a RESTRICT=parallel value for ebuilds that can't be built at the same time as other ebuilds? Brian says we need it for things like xorg-server which calls eselect opengl. If we truly need this, is RESTRICT=parallel a good name? We could make it a PROPERTIES value instead, but then we'd probably end up with something like PROPERTIES=no-parallel, so maybe RESTRICT is better. -- Thanks, Zac
Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel
On Tue, 13 Apr 2010 19:12:08 -0700 Zac Medico zmed...@gentoo.org wrote: Should we add a RESTRICT=parallel value for ebuilds that can't be built at the same time as other ebuilds? Brian says we need it for things like xorg-server which calls eselect opengl. I don't think that's the right solution. In most cases, xorg-server can be built in parallel with stuff which doesn't require/set specific opengl subsystem set. Well, in fact is there _really_ any package which won't work with switched opengl? I guess it's more of a runtime problem that buildtime, and I don't think we really should print out loudly 'libGL has been switched, please do not start OpenGL apps right now'. In fact, the best solution in this particular case would be to patch the buildsystem to use Gentoo location for particular OpenGL headers. Disabling parallel emerge would be more of a workaround for the issue, and will influence much more packages than it needs to. And it won't help if user is running multiple emerge calls at the same time. Another possible workaround is to enable some kind of 'eselect opengl' locking so that another package requiring access to it will wait until our build finishes. But this, of course, would require a quite good solution for maintaining the lock and dropping it whenever build process is aborted/killed. -- Best regards, Michał Górny http://mgorny.alt.pl xmpp:mgo...@jabber.ru
