[gentoo-dev] Re: Revisions for USE flag changes
Duncan posted on Sun, 13 Aug 2017 02:52:58 + as excerpted: > Michael Orlitzky posted on Sat, 12 Aug 2017 05:58:41 -0400 as excerpted: > >> On 08/12/2017 04:39 AM, Paweł Hajdan, Jr. wrote: >> >>> There are use-cases for --changed-use / --newuse other than changed >>> IUSE. >>> >>> I find it useful to easily rebuild affected packages when changing USE >>> flags in make.conf. If the flags were removed, would we have a good >>> alternative? >>> >> I simply overlooked the global USE change in make.conf because IMO it's >> a nonsense operation > > ?? > > How so? Are you arguing that deciding to system-wide switch to/from > pulseaudio, systemd, or gstreamer is nonsense? > > If so, I suspect many gentooers including myself strongly disagree. If > not, I'd be interested in what you propose as an alternative to changing > the appropriate USE flag systemwide, for what is after all a systemwide > change. After thinking about it for a few days, I see some logic to the point... in specific use-cases at least. Not setting global USE flags works reasonably well, provided (overlapping): * You have exactly one profile that makes sense for you, or you effectively create your own. By definition, this means you either agree with or don't care about other defaults, likely including openrc instead of systemd (because otherwise you won't be able to choose any other profile instead), and either use a minor arch (including x86), or use 16-bit only apps, or simply don't care about the additional work and build-time that multilib brings. Without addins, any time you want elements of multiple profiles, say plasma, no-multilib, systemd, etc (as here), you need to start setting many global flags for the ones you can't choose, either by setting them in make.conf, or by creating your own profile to set them. * You're just fine with the global defaults for anything not in your profile, either because you simply don't care, or because you want them the default off. * Any non-profile/non-IUSE-default USE flags you /do/ care about, you care about specific packages only. In the above scenario it does make some sense not to have any USE flags set in make.conf. Of course that's rather the opposite of my policy, which needs multiple profiles so must set the non-profile flags in make.conf, which considers an unset flag as much a chosen global default as a set flag, and which doesn't like profile or IUSE-defaults changing out from under it, so uses -* as a USE= prefix in make.conf. But my case isn't every case, and there's certainly a use-case where it does make sense, now that I've thought about it. Thanks for the prod. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Revisions for USE flag changes
On Tue, Aug 15, 2017 at 11:22:54PM -0400, Michael Orlitzky wrote: > On 08/14/2017 08:01 AM, Jason Zaman wrote: > > > > I'll give an example where revbumps are significantly inferior to > > --changed-use. > > > > ... With --changed-use, only the people who need it (ie selinux > > users) will rebuild and everyone is happy (selinux users because the > > program now works and non-selinux users because they did not rebuild > > for no reason). > > But this benefit exists only for Portage users, and can only be obtained > by throwing the others under the bus. > > (If you change RDEPEND, you need to create a new revision anyway: > https://projects.gentoo.org/council/meeting-logs/20151011-summary.txt) SELinux policy packages are not strictly RDEPENDs, portage will label packages as they are installed properly. if the policy package wasnt installed by the time the package is installed, you can manually label a package with rlpkg . but obviously having things jsut work is better. and they arnt DEPEND because you dont need them to build the package. Any i know of no selinux users using other package managers. There are no policies for them so they wouldnt work anyway. so no really throwing them under the bus. I dont think the other package managers label packages properly during install anyway even if there was a policy written. So there still isnt a reason to revbump a package when 99% of the world will not want it. -- Jason
Re: [gentoo-dev] Re: Revisions for USE flag changes
On 08/13/2017 03:11 AM, Michael Orlitzky wrote: > On 08/12/2017 10:52 PM, Duncan wrote: >> >> How so? Are you arguing that deciding to system-wide switch to/from >> pulseaudio, systemd, or gstreamer is nonsense? >> > > The meaning of any one USE flag varies widely across packages. I could > never say "I want to enable USE=gstreamer" for every package in the > tree, because I have no idea what it does for most of them. Setting > USE=whatever globally essentially means "make random changes to my > system" -- hence my wording. > > The meaning of a USE flag is per-package, so per-package is the only > meaningful way to set them. > There are USE flag situations that are relevant at the global level. systemd, pulseaudio, alsa, gstreamer, openssl/libressl, libav/ffmpeg, vim-syntax, and so on. Then there's USE_EXPAND variables, which might mean different things in different packages and yet I see nothing in your argument covering them. These flags make perfect sense at the global level, because users generally want support for the choices they make, and they make choices on that *general* level first, before diving into package-specific USE flags. It's a monumental waste of developer and user time to manually set major USE flags in every relevant package. Some people are picky and will still do that, but global USE ensures that certain assumptions are made about your system. If you don't want assumptions, don't use global USE. There's no reason to deprive others of functionality you don't personally agree with or use. Granted, some flags don't belong in make.conf. But part of Gentoo's beauty is that we *do* let users proverbially saw their leg off, if that's what they really want. There are lots of use cases that would be made ridiculous in scope if we got rid of global USE. Is your only answer a megabyte-long p.use file? That said, I like your idea of clearing up revbump decisions and the angle of reducing development burden. This particular idea comes at too high a cost for my taste, as we stand to lose functionality rather than improve or gain it. ~zlg -- Daniel Campbell - Gentoo Developer OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net fpr: AE03 9064 AE00 053C 270C 1DE4 6F7A 9091 1EA0 55D6 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Revisions for USE flag changes
On Wed, Aug 16, 2017 at 11:56 AM, Duncan <1i5t5.dun...@cox.net> wrote: > > But even if that's the case (I wouldn't know), it's the case due to a > deliberate decision of those going "under the bus", because portage is > the default, and by choosing to use some other PM, they've deliberately > chosen its (non-PMS) features over those of portage. > None of this really has anything to do with PMS as far as I can tell. There is no PMS requirement for --changed-use and such. There certainly might be a usability requirement for it, which is why it usually exists in one way or another. -- Rich
[gentoo-dev] Re: Revisions for USE flag changes
Michael Orlitzky posted on Tue, 15 Aug 2017 23:22:54 -0400 as excerpted: > On 08/14/2017 08:01 AM, Jason Zaman wrote: >> >> I'll give an example where revbumps are significantly inferior to >> --changed-use. >> >> ... With --changed-use, only the people who need it (ie selinux users) >> will rebuild and everyone is happy (selinux users because the program >> now works and non-selinux users because they did not rebuild for no >> reason). > > But this benefit exists only for Portage users, and can only be obtained > by throwing the others under the bus. But even if that's the case (I wouldn't know), it's the case due to a deliberate decision of those going "under the bus", because portage is the default, and by choosing to use some other PM, they've deliberately chosen its (non-PMS) features over those of portage. Just as I, by choosing --newuse instead, have chosen to do rebuilds in such cases, even with portage. (Tho TBH I've never noticed that particular case, probably because it's lost in the noise compared to --changed-deps (enabled when static-deps were newer and I wanted to be sure, likely unneeded these days) and smart- live-rebuild of my (live) kde packages.) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Revisions for USE flag changes
On 08/14/2017 08:01 AM, Jason Zaman wrote: > > I'll give an example where revbumps are significantly inferior to > --changed-use. > > ... With --changed-use, only the people who need it (ie selinux > users) will rebuild and everyone is happy (selinux users because the > program now works and non-selinux users because they did not rebuild > for no reason). But this benefit exists only for Portage users, and can only be obtained by throwing the others under the bus. (If you change RDEPEND, you need to create a new revision anyway: https://projects.gentoo.org/council/meeting-logs/20151011-summary.txt)
Re: [gentoo-dev] Re: Revisions for USE flag changes
On Sat, Aug 12, 2017 at 09:05:58PM +1000, Michael Palimaka wrote: > On 08/12/2017 08:29 PM, Rich Freeman wrote: > > On Sat, Aug 12, 2017 at 5:57 AM, Michael Orlitzkywrote: > >> On 08/12/2017 03:03 AM, Michał Górny wrote: > >>> > >>> Please provide some examples of recent in-place USE changes that benefit > >>> from revbumps. > >>> > >> > >> There is no single example. Things only get simpler if *all* USE changes > >> come with a new revision. > >> > > This policy change would make my life easier, because for big packages > > it would encourage maintainers to not make IUSE changes until they do > > revbumps, which would save me a build. I'm running on relatively old > > hardware at this point so these rebuilds actually do cost me quite a > > bit of time. I'm not sure that not using --changed-use is a great > > option though as it will make it that much harder to keep things > > consistent when I do modify my package.use/make.conf. > > > > At least now you have the option to run without --changed-use if you > want. If inline IUSE changes are completely banned, you will definitely > see more pointless rebuilds on your old hardware. In my experience most > developers make a change when there's a change to be made, and don't > "save up" changes until some arbitrary delta is reached. We've already > an increase in revbumps like this in other areas where inline changes > are being discouraged. I'll give an example where revbumps are significantly inferior to --changed-use. The selinux useflag is hardmasked in all profiles except the selinux ones and 99% of users do not run selinux. I regularly add "selinux ? ( sec-policy/selinux-foo )" to RDEPEND of packages. With --changed-use, only the people who need it (ie selinux users) will rebuild and everyone is happy (selinux users because the program now works and non-selinux users because they did not rebuild for no reason). If i were to randomly revbump packages whenever i needed to add selinux policy deps to packages then i would make 99% of users upset for like no reason. -- Jason
Re: [gentoo-dev] Re: Revisions for USE flag changes
On 13/08/17 11:11, Michael Orlitzky wrote: > On 08/12/2017 10:52 PM, Duncan wrote: >> How so? Are you arguing that deciding to system-wide switch to/from >> pulseaudio, systemd, or gstreamer is nonsense? >> > The meaning of any one USE flag varies widely across packages. I could > never say "I want to enable USE=gstreamer" for every package in the > tree, because I have no idea what it does for most of them. Setting > USE=whatever globally essentially means "make random changes to my > system" -- hence my wording. > > The meaning of a USE flag is per-package, so per-package is the only > meaningful way to set them. > Which is why we have GLOBAL use flags and LOCAL use flags, right?! I'm not sure I'm actually reading this discussion right now?! and I'm *not* a dev ... signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Revisions for USE flag changes
On 08/12/2017 10:52 PM, Duncan wrote: > > How so? Are you arguing that deciding to system-wide switch to/from > pulseaudio, systemd, or gstreamer is nonsense? > The meaning of any one USE flag varies widely across packages. I could never say "I want to enable USE=gstreamer" for every package in the tree, because I have no idea what it does for most of them. Setting USE=whatever globally essentially means "make random changes to my system" -- hence my wording. The meaning of a USE flag is per-package, so per-package is the only meaningful way to set them.
Re: [gentoo-dev] Re: Revisions for USE flag changes
On 08/12/2017 10:32 PM, Duncan wrote: >> >> What if you fix a runtime issue by dropping a flag? It's more confusing >> than it has to be: the USE flag exception interacts weirdly with all the >> other rules. > > Bad example as it's a security vuln, which requires masking/removing > vulnerable versions Ok, change "security vulnerability" to "erases the root partition."
[gentoo-dev] Re: Revisions for USE flag changes
Michael Orlitzky posted on Sat, 12 Aug 2017 05:58:41 -0400 as excerpted: > On 08/12/2017 04:39 AM, Paweł Hajdan, Jr. wrote: > >> There are use-cases for --changed-use / --newuse other than changed >> IUSE. >> >> I find it useful to easily rebuild affected packages when changing USE >> flags in make.conf. If the flags were removed, would we have a good >> alternative? >> > I simply overlooked the global USE change in make.conf because IMO it's > a nonsense operation ?? How so? Are you arguing that deciding to system-wide switch to/from pulseaudio, systemd, or gstreamer is nonsense? If so, I suspect many gentooers including myself strongly disagree. If not, I'd be interested in what you propose as an alternative to changing the appropriate USE flag systemwide, for what is after all a systemwide change. Just seems to me an extremely surprising and unexpected argument, so I'd like to understand more of the reasoning behind it. I'll very likely learn something, as invariably the answer to questions I find myself compelled to ask due to what appears to me a transparently obvious different answer, reveal an angle I hadn't previously considered, sometimes changing my mind entirely. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
[gentoo-dev] Re: Revisions for USE flag changes
Michael Orlitzky posted on Sat, 12 Aug 2017 10:14:18 -0400 as excerpted: > On 08/12/2017 06:29 AM, Rich Freeman wrote: >> >> My gut feeling is that the change you want is probably a good thing, >> but it will never happen if you can't provide a single example of >> something bad happening due to the lack of a revbump. > > There's an unfixed security vulnerability with USE=foo, so we drop the > flag temporarily. Users who had USE=foo enabled will keep the vulnerable > code installed until they update with --changed-use or --newuse. > > Even with the devmanual improvements, the advice we give is conflicting: > > * If you fix an important runtime issue, do a revbump. > > * If you drop a USE flag, don't do a revbump. > > What if you fix a runtime issue by dropping a flag? It's more confusing > than it has to be: the USE flag exception interacts weirdly with all the > other rules. Bad example as it's a security vuln, which requires masking/removing vulnerable versions, which will require a version bump in ordered to prevent downgrades if it was the latest visible for a (stable or ~arch) keyword. So the version bump is effectively mandatory due to security overrides in any case, and that it was fixed by a temporary USE flag drop doesn't change things at all. If that security-override isn't explicit in current documentation, that'd be the bug, not the fact that use-flag drops don't on their own require a version-bump. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Revisions for USE flag changes
On Sat, Aug 12, 2017 at 7:05 AM, Michael Palimakawrote: > On 08/12/2017 08:29 PM, Rich Freeman wrote: >> On Sat, Aug 12, 2017 at 5:57 AM, Michael Orlitzky wrote: >>> On 08/12/2017 03:03 AM, Michał Górny wrote: Please provide some examples of recent in-place USE changes that benefit from revbumps. >>> >>> There is no single example. Things only get simpler if *all* USE changes >>> come with a new revision. >>> >> This policy change would make my life easier, because for big packages >> it would encourage maintainers to not make IUSE changes until they do >> revbumps, which would save me a build. I'm running on relatively old >> hardware at this point so these rebuilds actually do cost me quite a >> bit of time. I'm not sure that not using --changed-use is a great >> option though as it will make it that much harder to keep things >> consistent when I do modify my package.use/make.conf. >> > > At least now you have the option to run without --changed-use if you > want. If inline IUSE changes are completely banned, you will definitely > see more pointless rebuilds on your old hardware. True, since we now have --changed-use (I think this is a relatively recent portage feature - before there was only --newuse). Obviously if I stopped using --changed-use then my installed configuration would drift out of sync with the settings in /etc/portage. I'm not sure that this causes any other issues in this case - there certainly have been issues historically in these situations but I think most of them have been eliminated. Changed dependencies can definitely cause problems, but I'm less certain that changed IUSE does. > In my experience most > developers make a change when there's a change to be made, and don't > "save up" changes until some arbitrary delta is reached. We've already > an increase in revbumps like this in other areas where inline changes > are being discouraged. > I imagine that such practices vary. I know I personally tend to save up minor changes for major revisions to reduce the need for testing. Ultimately though I think the real question is whether not revbumping has the potential to break things. I does for dependency changes which is why that policy change was made (and I still run with --changed-deps anyway because I don't trust devs to not mess this up). I think we do need to have more clear evidence that IUSE changes break things before we should consider requiring revbumps for this. It would be nice if big packages waited for revbumps to make IUSE changes, but honestly the occassional chromium rebuild doesn't bother me that much. Most of it happens with cron anyway. -- Rich
[gentoo-dev] Re: Revisions for USE flag changes
On 08/12/2017 08:29 PM, Rich Freeman wrote: > On Sat, Aug 12, 2017 at 5:57 AM, Michael Orlitzkywrote: >> On 08/12/2017 03:03 AM, Michał Górny wrote: >>> >>> Please provide some examples of recent in-place USE changes that benefit >>> from revbumps. >>> >> >> There is no single example. Things only get simpler if *all* USE changes >> come with a new revision. >> > This policy change would make my life easier, because for big packages > it would encourage maintainers to not make IUSE changes until they do > revbumps, which would save me a build. I'm running on relatively old > hardware at this point so these rebuilds actually do cost me quite a > bit of time. I'm not sure that not using --changed-use is a great > option though as it will make it that much harder to keep things > consistent when I do modify my package.use/make.conf. > At least now you have the option to run without --changed-use if you want. If inline IUSE changes are completely banned, you will definitely see more pointless rebuilds on your old hardware. In my experience most developers make a change when there's a change to be made, and don't "save up" changes until some arbitrary delta is reached. We've already an increase in revbumps like this in other areas where inline changes are being discouraged.
[gentoo-dev] Re: Revisions for USE flag changes
On 08/12/2017 08:16 PM, Michael Orlitzky wrote: > On 08/12/2017 12:22 AM, Michael Palimaka wrote: >> >>> Q. But what if I maintain firefox, and I need to change IUSE? >>> >>> If the IUSE change isn't important, just make the new revision in a >>> branch and wait to commit it later when there are more changes >>> piled up. If it is important (like if its default value changes >>> RDEPEND), then it would have required a revision anyway. >> >> Please stop trying to force workflows on people. Using that same logic, >> I can make the IUSE change in-place and it would be propagated in the >> next version bump. >> > > I'm not trying to force anything on anyone, I'm just asking for > feedback. If it turns out to be a stupid idea, then so be it. > > If it's understood that you can make IUSE changes but that they'll only > be propagated on the next version bump, then there would be no problem. > But we're about to document a policy that says it's OK to do things that > wouldn't normally be OK, because --changed-use is there to save us: > > The examples of changes that can be done without a revision bump are: > > ... > > * adding a new USE flag or removing an existing one (since change > in USE flags is going to trigger --changed-use rebuild), > > If developers operate under that assumption and if you don't use > --changed-use, you're going to run into problems eventually. --changed-use is an optional flag and portage works just as well without it. Please provide examples of such problems. > > >>> * emerge runs a bit faster. >> >> Why will it run faster? > > The developer now indicates that IUSE has changed, so portage doesn't > have to figure it out on its own. Please provide some numbers to back up this claim. Even if we assume portage will run faster because we can remove --changed-use (which we can't, because as pointed out in other posts we still need this flag), surely any time savings gained there will be lost by pointless rebuilds?
Re: [gentoo-dev] Re: Revisions for USE flag changes
On Sat, Aug 12, 2017 at 12:22 AM, Michael Palimakawrote: > On 08/12/2017 09:50 AM, Michael Orlitzky wrote: >> Q. But what about the rebuilds? >> >> For most packages, the rebuilds simply don't matter. Unless you're >> the maintainer of libreoffice, firefox, chromium, etc. -- just do the >> revision and forget about the (quick) rebuilds. > > I really wish people would stop trotting out this false argument. Not > everyone has the latest and greatest hardware. Rebuilds have a real cost > to end users and as such we should use them wisely. > Agree, but maintainers would have the option to just not change IUSE at all until the next time they would revbump anyway. That saves rebuilds for those using --changed-use today. I'm not convinced that it is actually that easy for people to avoid --changed-use, and if they are using it then they're going to potentially get exposed to these rebuilds when IUSE changes. -- Rich
Re: [gentoo-dev] Re: Revisions for USE flag changes
On 08/12/2017 12:22 AM, Michael Palimaka wrote: > >> Q. But what if I maintain firefox, and I need to change IUSE? >> >> If the IUSE change isn't important, just make the new revision in a >> branch and wait to commit it later when there are more changes >> piled up. If it is important (like if its default value changes >> RDEPEND), then it would have required a revision anyway. > > Please stop trying to force workflows on people. Using that same logic, > I can make the IUSE change in-place and it would be propagated in the > next version bump. > I'm not trying to force anything on anyone, I'm just asking for feedback. If it turns out to be a stupid idea, then so be it. If it's understood that you can make IUSE changes but that they'll only be propagated on the next version bump, then there would be no problem. But we're about to document a policy that says it's OK to do things that wouldn't normally be OK, because --changed-use is there to save us: The examples of changes that can be done without a revision bump are: ... * adding a new USE flag or removing an existing one (since change in USE flags is going to trigger --changed-use rebuild), If developers operate under that assumption and if you don't use --changed-use, you're going to run into problems eventually. >> * emerge runs a bit faster. > > Why will it run faster? The developer now indicates that IUSE has changed, so portage doesn't have to figure it out on its own.
[gentoo-dev] Re: Revisions for USE flag changes
On 08/12/2017 09:50 AM, Michael Orlitzky wrote: > Q. But what about the rebuilds? > > For most packages, the rebuilds simply don't matter. Unless you're > the maintainer of libreoffice, firefox, chromium, etc. -- just do the > revision and forget about the (quick) rebuilds. I really wish people would stop trotting out this false argument. Not everyone has the latest and greatest hardware. Rebuilds have a real cost to end users and as such we should use them wisely. > We tell everyone to use --changed-use and --newuse if they want > things to work, so they were probably going to rebuild anyway. Who tells everyone to use these flags and where? I never use these flags day-to-day, only if I need that specific functionality for that reason > Q. But what if I maintain firefox, and I need to change IUSE? > > If the IUSE change isn't important, just make the new revision in a > branch and wait to commit it later when there are more changes > piled up. If it is important (like if its default value changes > RDEPEND), then it would have required a revision anyway. Please stop trying to force workflows on people. Using that same logic, I can make the IUSE change in-place and it would be propagated in the next version bump. > Q. But I work on a team, and we can't all work in different branches. > > If you work on a massive package and if you're collaborating with > others regularly, you can commit the new ebuild masked. This is > annoying, but is an extremely rare combination of circumstances. Again, let's not try and tell teams which workflow works best for them. > == tl;dr == > > We would be better off with respect to IUSE changes and revisions if we > deleted the --changed-use and --newuse flags right now, and just > required developers to revbump when changing IUSE. > > Package managers get simpler, documentation gets simpler, the user > interface gets simpler, and behavior becomes more uniform and predictable. > > Please let me know what you think. I disagree with this change because your proposed benefits don't hold up: > * We can delete all of the PM code for --changed-use and --newuse and > friends. As pointed out by Brian, we still need at least --changed-use even if IUSE changes in-place are banned. > * The documentation becomes much simpler: revbump if IUSE changes. We should base our policies around the cost / benefit of said policy, not how many or few words we need to write in the devmanual about it. > * Users can omit --newuse and --changed-use from their lives. They already can. > * All package managers now handle IUSE changes properly. If you want to see consistent behaviour in how package manages handle IUSE, I suggest sending patches for PMS. I don't see any problem in portage/paludis/pkgcore handling things differently. That is the point of having different package managers, after all. > * emerge runs a bit faster. Why will it run faster?