Re: Semi-serious proposal: drop all optional entries from comps
On Thu, Sep 27, 2018 at 11:09 AM Stephen Gallagher wrote: > > On Wed, Sep 26, 2018 at 9:52 AM Matthew Miller > wrote: > > > > On Tue, Sep 25, 2018 at 08:17:18AM +0200, Vít Ondruch wrote: > > > rubygem-rails (which already exists and has its purpose no matter if > > > there are comps or not) via Suggests for example. The only issue AFAIK > > > is there is no real support for Suggests in DNF :/ > > > > What would that look like? An interactive mode where you Y/N every suggested > > package? "Install everything suggested" seems too coarse. > > > > Debian/Ubuntu have apt-get just print a statement before the > confirmation in interactive mode saying "Hey, you might also want > these things" and you can cancel the current transaction and add the > additional ones you want at the command line. There's already a pull request by Igor Gnatenko to add that[1]. The newest libdnf releases have the necessary APIs to report suggested packages, which can be leveraged by dnf and dnfdaemon (and thus dnfdragora). [1]: https://github.com/rpm-software-management/dnf/pull/1125 -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Tue, Sep 25, 2018 at 10:11 PM Kevin Kofler wrote: > > Stephen Gallagher wrote: > > That used to be true, but with Recommends these days, it's considerably > > less so. > > Not really, because of: > https://github.com/openSUSE/libsolv/issues/168 > (which libsolv upstream sadly does not consider a bug). > This would be behavior that would likely be implemented at the libdnf layer anyway, since that's where SWDB is, and that's where we would know that information. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Dne 26.9.2018 v 18:35 Adam Williamson napsal(a): > On Tue, 2018-09-25 at 08:17 +0200, Vít Ondruch wrote: >> Dne 24.9.2018 v 19:32 Adam Williamson napsal(a): >>> On Mon, 2018-09-24 at 12:21 +0200, Vít Ondruch wrote: Just FTR, some while ago, I proposed to drop comps entirely: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ISCIB67JKW7WBC74KA4DSCAP6AZOUY5G/ >>> That...doesn't seem like a very serious proposal at all, given that you >>> didn't suggest *in any way* how we would replace the critical functions >>> it's currently performing. Are you suggesting metapackages? >> Well I proposed "drop comps entirely (or at least trim them down >> significantly)". At least the second part is in line with your request I >> believe. >> >> And I am not speaking necessarily about "metapackages". But we should >> really start using Recommends more and especially Suggest is unused at >> all. E.g. in comps, there is "Ruby on Rails" group installing bunch of >> packages. There is no reason to not have these dependencies specified in >> rubygem-rails (which already exists and has its purpose no matter if >> there are comps or not) via Suggests for example. The only issue AFAIK >> is there is no real support for Suggests in DNF :/ > How would this work in the identified use case of 'package managers > displaying groups of packages for browsing'? Would the package managers > have to somehow generate these views on the fly from Suggests? Frankly, package manager have not adjusted yet. They don't do anything about Recommends which are commonly used nowadays. Anyway, g-s can show plugins (you can check gVim for example), so this is very similar scenario IMO. Vít ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Mon, 2018-09-24 at 11:16 -0400, Stephen Gallagher wrote: > On Mon, Sep 24, 2018 at 11:15 AM Kevin Fenzi wrote: > > Metapackages have other issues too. They make it difficult to consume > > part of a group. You either have to have everything or nothing. With > > comps groups you can install a group, remove some packages you don't > > want and still get updates to the group, with metapackages it's all or > > nothing. > > > > That used to be true, but with Recommends these days, it's considerably less > so. True, but you wind up sort of conflating groups with soft dependencies used for other purposes, in that case. What if you want the soft dependencies of *packages* when you do 'dnf update', but not the optional packages in *groups* you have installed? How do you express that? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Wed, Sep 26, 2018 at 9:52 AM Matthew Miller wrote: > > On Tue, Sep 25, 2018 at 08:17:18AM +0200, Vít Ondruch wrote: > > rubygem-rails (which already exists and has its purpose no matter if > > there are comps or not) via Suggests for example. The only issue AFAIK > > is there is no real support for Suggests in DNF :/ > > What would that look like? An interactive mode where you Y/N every suggested > package? "Install everything suggested" seems too coarse. > Debian/Ubuntu have apt-get just print a statement before the confirmation in interactive mode saying "Hey, you might also want these things" and you can cancel the current transaction and add the additional ones you want at the command line. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Mon, 2018-09-24 at 16:28 -0400, Matthew Miller wrote: > On Sun, Sep 23, 2018 at 06:05:16AM +0100, Peter Robinson wrote: > > Some of the group stuff is also used during the compose and if things > > aren't in groups specified but needed by say a kickstart the packages > > won't be in certain places and will break things. I did this by > > accident when adding zram in F-29 and not adding it to comps. > > How much if this is an artifact of the compose being designed around trying > to pack subsets of installable media onto a DVD? Do we currently do that for > anything but Server, and is it really important that we continue? We also use the same group definitions for specifying what sets of packages go onto the live media, disk images, and into the variant trees. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Wed, Sep 26, 2018 at 12:36 PM Adam Williamson wrote: > > On Tue, 2018-09-25 at 08:17 +0200, Vít Ondruch wrote: > > > > Dne 24.9.2018 v 19:32 Adam Williamson napsal(a): > > > On Mon, 2018-09-24 at 12:21 +0200, Vít Ondruch wrote: > > > > Just FTR, some while ago, I proposed to drop comps entirely: > > > > > > > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ISCIB67JKW7WBC74KA4DSCAP6AZOUY5G/ > > > That...doesn't seem like a very serious proposal at all, given that you > > > didn't suggest *in any way* how we would replace the critical functions > > > it's currently performing. Are you suggesting metapackages? > > > > Well I proposed "drop comps entirely (or at least trim them down > > significantly)". At least the second part is in line with your request I > > believe. > > > > And I am not speaking necessarily about "metapackages". But we should > > really start using Recommends more and especially Suggest is unused at > > all. E.g. in comps, there is "Ruby on Rails" group installing bunch of > > packages. There is no reason to not have these dependencies specified in > > rubygem-rails (which already exists and has its purpose no matter if > > there are comps or not) via Suggests for example. The only issue AFAIK > > is there is no real support for Suggests in DNF :/ > > How would this work in the identified use case of 'package managers > displaying groups of packages for browsing'? Would the package managers > have to somehow generate these views on the fly from Suggests? libsolv can generate special solvables when packages are tagged correctly, which could be exposed through the DNF API such that dnfdaemon can identify it as a "group" type and represent them specially. This is how patterns work in (open)SUSE with Zypper. But actually, I'd rather not go to that, because the composition groups based categorization is too weak and incomplete when it comes to sorting packages for people to browse (which is what dnfdragora does). This is why I'd advocate for the return of the RPM Group tag, because it's a useful way to categorize the entire collection of packages. But if we insist on comps-style grouping *only*, then we could do that with metapackages. We can also do it to some extent with AppStream data, but again, that's incomplete from the perspective of the distribution as a whole. It's not useful enough for a package manager view, though it's certainly good for an app-centric view. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Tue, 2018-09-25 at 08:17 +0200, Vít Ondruch wrote: > > Dne 24.9.2018 v 19:32 Adam Williamson napsal(a): > > On Mon, 2018-09-24 at 12:21 +0200, Vít Ondruch wrote: > > > Just FTR, some while ago, I proposed to drop comps entirely: > > > > > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ISCIB67JKW7WBC74KA4DSCAP6AZOUY5G/ > > That...doesn't seem like a very serious proposal at all, given that you > > didn't suggest *in any way* how we would replace the critical functions > > it's currently performing. Are you suggesting metapackages? > > Well I proposed "drop comps entirely (or at least trim them down > significantly)". At least the second part is in line with your request I > believe. > > And I am not speaking necessarily about "metapackages". But we should > really start using Recommends more and especially Suggest is unused at > all. E.g. in comps, there is "Ruby on Rails" group installing bunch of > packages. There is no reason to not have these dependencies specified in > rubygem-rails (which already exists and has its purpose no matter if > there are comps or not) via Suggests for example. The only issue AFAIK > is there is no real support for Suggests in DNF :/ How would this work in the identified use case of 'package managers displaying groups of packages for browsing'? Would the package managers have to somehow generate these views on the fly from Suggests? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Dne 26.9.2018 v 15:51 Matthew Miller napsal(a): > On Tue, Sep 25, 2018 at 08:17:18AM +0200, Vít Ondruch wrote: >> rubygem-rails (which already exists and has its purpose no matter if >> there are comps or not) via Suggests for example. The only issue AFAIK >> is there is no real support for Suggests in DNF :/ > What would that look like? An interactive mode where you Y/N every suggested > package? "Install everything suggested" seems too coarse. > You would have to do something like "dnf install rubygem-rails --including-suggested" (whatever the option would be named, since on some places they are referred as "very weak" while guidelines refer to them as a "hints"). That would be similar to what groupinstall does. And -x to exclude particular packages should still be available. Vít ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Tue, Sep 25, 2018 at 08:17:18AM +0200, Vít Ondruch wrote: > rubygem-rails (which already exists and has its purpose no matter if > there are comps or not) via Suggests for example. The only issue AFAIK > is there is no real support for Suggests in DNF :/ What would that look like? An interactive mode where you Y/N every suggested package? "Install everything suggested" seems too coarse. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Stephen Gallagher wrote: > That used to be true, but with Recommends these days, it's considerably > less so. Not really, because of: https://github.com/openSUSE/libsolv/issues/168 (which libsolv upstream sadly does not consider a bug). Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Dne 24.9.2018 v 19:32 Adam Williamson napsal(a): > On Mon, 2018-09-24 at 12:21 +0200, Vít Ondruch wrote: >> Just FTR, some while ago, I proposed to drop comps entirely: >> >> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ISCIB67JKW7WBC74KA4DSCAP6AZOUY5G/ > That...doesn't seem like a very serious proposal at all, given that you > didn't suggest *in any way* how we would replace the critical functions > it's currently performing. Are you suggesting metapackages? Well I proposed "drop comps entirely (or at least trim them down significantly)". At least the second part is in line with your request I believe. And I am not speaking necessarily about "metapackages". But we should really start using Recommends more and especially Suggest is unused at all. E.g. in comps, there is "Ruby on Rails" group installing bunch of packages. There is no reason to not have these dependencies specified in rubygem-rails (which already exists and has its purpose no matter if there are comps or not) via Suggests for example. The only issue AFAIK is there is no real support for Suggests in DNF :/ Or conditional dependencies, e.g. fedora-release could have something like "Recommends: gnome-shell" and "Recommends: (gnome-* if gnome-shell)". Again, we have already fedora-release package (is it metapackage or not?), there is nothing new here. Frankly, I would expect such dependencies to be there already, but there was no push to do that, "because we have comps which do the job just fine". Or we could use Enhances for specifying that some application should be part of default Gnome experience. But there could be also metapackages such as "default-gnome-desktop" which would suggest all the components for default Gnome experience. There is plenty of options we could use instead of comps. Vít ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Matthew Miller (mat...@fedoraproject.org) said: > On Sun, Sep 23, 2018 at 06:05:16AM +0100, Peter Robinson wrote: > > Some of the group stuff is also used during the compose and if things > > aren't in groups specified but needed by say a kickstart the packages > > won't be in certain places and will break things. I did this by > > accident when adding zram in F-29 and not adding it to comps. > > How much if this is an artifact of the compose being designed around trying > to pack subsets of installable media onto a DVD? Do we currently do that for > anything but Server, and is it really important that we continue? Given the lack of individual package selection in Anaconda, this would mean that optional package inclusions there are solely for: - kickstarting off the DVD - using the DVD as a post-install locally mounted package source There may be people doing the first. I'd have to think there are better ways to handle the second. Bill ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sun, Sep 23, 2018 at 06:05:16AM +0100, Peter Robinson wrote: > Some of the group stuff is also used during the compose and if things > aren't in groups specified but needed by say a kickstart the packages > won't be in certain places and will break things. I did this by > accident when adding zram in F-29 and not adding it to comps. How much if this is an artifact of the compose being designed around trying to pack subsets of installable media onto a DVD? Do we currently do that for anything but Server, and is it really important that we continue? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Mon, 2018-09-24 at 08:47 -0700, Adam Williamson wrote: > If we want to fix that we'd need to somehow run the comps script on all > changes in dist-git and block those changes on introducing comps > problems, which seems like a rather more difficult problem. Well...thinking about it a bit more, it's just a *different* problem, I guess. We'd need a somewhat different script: it would keep the bit for reading the packagereq entries from comps, but instead of poking dnf repos for current packages, it would have to be able to understand what binary packages the source package in question would produce *before* the change, and what binary packages it would produce *after* the change. That seems easy, but isn't entirely, because you can have stuff like a %ifarch that produces a given subpackage on some arches but others. It's another Will Woods Problem, of rpm specs being able to do an awful lot of stuff that isn't terribly easy to interpret without actually going through the process of building the package. I suppose if we wind up having CI which actually runs a package build for every prospective change to a package, this wouldn't be too difficult, as we could just read out the subpackages...but yeah, I'd say it's kind of a *different* thing, anyway. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Mon, 2018-09-24 at 12:21 +0200, Vít Ondruch wrote: > Just FTR, some while ago, I proposed to drop comps entirely: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ISCIB67JKW7WBC74KA4DSCAP6AZOUY5G/ That...doesn't seem like a very serious proposal at all, given that you didn't suggest *in any way* how we would replace the critical functions it's currently performing. Are you suggesting metapackages? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Mon, 2018-09-24 at 08:26 +, Zbigniew Jędrzejewski-Szmek wrote: > On Sun, Sep 23, 2018 at 10:41:31AM +0200, Nicolas Mailhot wrote: > > Le samedi 22 septembre 2018 à 11:38 -0700, Adam Williamson a écrit : > > > But at the same time I think Matt's right that comps -at least as it's > > > currently set up - is kind of a really *bad* way of doing this, and > > > that seems fairly well demonstrated by the problem I'm trying to solve > > > - that comps has tons of broken entries in it. Just from looking > > > through the file as I do this, I really suspect that no-one is > > > maintaining quite a lot of the groups and the packages in them may > > > well just not make much sense any more. > > > > That's a QA problem > > > > But really, the problem is not the comps format, it's that any kind of > > classification activity is both tedious and optional (if you know how to > > classify a package you do not need the classification in the first > > place), rots fast in presence of high package churn, and can only work > > long term with lots of janitorial involvement (both human and > > automated). > > That was my first thought too. > > On Fri, Sep 21, 2018 at 06:14:51PM -0700, Adam Williamson wrote: > > I am currently working on finding and removing all comps entries that > > point to packages which don't exist any more. > > > > While doing this extremely tedious task [...] > > I would love to see some more info before any decisions are made. > I *thought* I more-or-less understand the comps process, I remember > doing some minor changes in the past, but things seem to have changed. > > How are you doing those checks? Is it > https://pagure.io/fedora-comps/pull-request/328? Yes. > It should be possible to turn those checks into CI hooks. Eh...it'd need a less janky script. That one is pretty dumb. Besides, there's an obvious problem with that idea: missing packages don't usually show up in the list via comps PRs. It's not that people make changes to comps that list packages which don't exist *right then* - not usually, anyway. What happens is packages get retired or renamed, and no-one remembers to update comps. If we want to fix that we'd need to somehow run the comps script on all changes in dist-git and block those changes on introducing comps problems, which seems like a rather more difficult problem. > There was some jenkins jobs being run on PRs in > pagure.io/fedora-comps, but it seems to have atrophied and > doesn't appear on new PRs. Do we have any automated checks for comps now? I don't think so. > Instead of designing a whole new system that needs to be > a) designed, b) agreed, c) documented, d) implemented, e) integrated > with all the other tools, and f) referenced in all the docs where > the old system is now referenced, > why not first fix the CI to do the checks? Pruning nonexistent packages > really sounds like something that can automatized. Well, it's not quite as simple as it sounds to do it correctly, in fact - it took me about five hours to handle the first 40 or so bad entries, because I didn't just unthinkingly chop out all the entries. I actually looked at each package's dist-git history and sometimes upstream history to figure out why it had been retired, whether it had been directly replaced by something else, or if there was in some other way a clear replacement for it. Besides, the reason we're now kicking around the possibility of replacing/improving the system itself is that this isn't the *only* problem with it. > Another question: people keep mentioning unexpected breakage from > changes by random contributors. But https://pagure.io/fedora-comps/ > seems *very* tightly controlled with write access by only @releng and > adamw. So is this still actually a problem? Not much (though it can sometimes be hard to completely predict the impact of all proposals). But it *used* to be, which is why we have the tight access control. Adding the tight access control, though - as KK correctly pointed out - has the negative effect of making it perhaps too *hard* to make changes that will always be trivial in impact, like adding/removing 'optional' entries to/from a group mainly intended for display in package managers. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Mon, Sep 24, 2018 at 11:15 AM Kevin Fenzi wrote: > Metapackages have other issues too. They make it difficult to consume > part of a group. You either have to have everything or nothing. With > comps groups you can install a group, remove some packages you don't > want and still get updates to the group, with metapackages it's all or > nothing. > That used to be true, but with Recommends these days, it's considerably less so. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On 09/23/2018 10:14 PM, Nicolas Mailhot wrote: > Le lundi 24 septembre 2018 à 00:57 +0200, Kevin Kofler a écrit : >> >> Another issue if you want to use metapackages for categorization is >> that UIs >> such as Dnfdragora, or whatever tool converts the metapackages to a >> representation those UIs will consume, would have to be told what >> packages >> are such categorization metapackages. > > That's trivially solvable by requiring they are named > metapackage-something > and contain a metapackage(something) provides > > That would not change the fact such categorization helps tend to rot > over time if not cared about. They would regularly miss composes due to > broken deps. Metapackages have other issues too. They make it difficult to consume part of a group. You either have to have everything or nothing. With comps groups you can install a group, remove some packages you don't want and still get updates to the group, with metapackages it's all or nothing. kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Just FTR, some while ago, I proposed to drop comps entirely: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ISCIB67JKW7WBC74KA4DSCAP6AZOUY5G/ Vít Dne 22.9.2018 v 03:14 Adam Williamson napsal(a): > Hi folks! > > I am currently working on finding and removing all comps entries that > point to packages which don't exist any more. > > There are quite a lot of them. > > A lot of them are 'optional' entries. > > While doing this extremely tedious task, it occurred to me to think: > what the hell is the *point* of these 'optional' entries any more, > anyway? > > In Ye Olde Days, there was an obvious point to them: they showed up in > the installer. Prior to Fedora 18, with the old anaconda UI, the > 'package selection' screen gave you per-package selection within the > comps groups. You could get it to show you a list of all packages in > the group. Ones which were 'mandatory' would be locked in the GUI; you > couldn't install the group and *not* install that package. Ones which > were 'default' would be selected by default when you selected the > group, but you could unselect them if you wanted. And ones which were > 'optional' would be unselected by default, but they were *displayed*, > and you could select them if you wanted. > > It looked like this: > > https://www.tecmint.com/wp-content/uploads/2012/07/rhel-15.png > > (you got to see the 'optional' packages if you clicked > on...well...'Optional packages'). > > The old gnome-packagekit, IIRC, also parsed groups and showed you all > this stuff. > > But...we don't do that any more. anaconda does not expose 'optional' > packages in any way any more (you can only pick environment groups and > their supplementary package groups in anaconda, now). GNOME Software > doesn't either. > > So do we really need these acres of 'optional' packages in comps? I > mean, there are 2519 of them in comps-f30.xml.in. That's a lot. I > suspect no-one's looked whether most of them make any sense for years. > There are entire groups that are *nothing but optional packages*, > making them almost entirely useless. > > So, OK, I think there are probably apps out there that still expose > this info. I suspect dnfdragora does, for instance. But is the minor > benefit of having these lists of 'hey maybe you'd like this thing' in > minor package managers really worth the way they turn comps into even > more of a gigantic crufty ball-of-wax than it would otherwise be? > > Is anyone really super-attached to this kinda stuff? > > 4ti2 > alt-ergo > alt-ergo-gui > atlas > automaton > automaton-javadoc > azove > blas > bliss > bowtie > brial > bwa > cantor > cddlib > chemtool > coq > coq-coqide > coq-doc > coq-emacs > cryptominisat > csdp > csdp-tools > cudd > cudd-devel > cvc4 > cvc4-devel > cvc4-doc > dx > E > eclib > EMBOSS > fastx_toolkit > fflas-ffpack-devel > fityk > flint > flocq > frama-c > freefem++ > gabedit > galculator > gap > gappa > gappalib-coq > gdl > genius > geomview > gfan > ginac > glimmer > GMT > gshhg-gmt-nc4-full > gshhg-gmt-nc4-high > GMT-doc > gnome-chemistry-utils > gpredict > grace > grads > gromacs > gromacs-openmpi > gts > hdf > hdf5 > hmmer > jmol > kpolynome > kst > lagan > lapack > latte-integrale > libmatheval > libtcd > linbox > ltl2ba > Macaulay2 > malaga > maxima-gui > meataxe > minisat2 > molsketch > mona > mona-devel > mona-emacs > mona-examples > mona-xemacs > mpfi > ncl > nco > ncview > netcdf > normaliz > openbabel > opencv > paraview > picosat > picosat-devel > plotutils > brial > polymake > pvs-sbcl > pypop > python3-biopython > python3-cvxopt > python3-networkx > python3-theano > qalculate-gtk > qalculate-kde > qepcad-B > qtoctave > root > routino > rrdtool > scidavis > seaview > sextractor > SIBsim4 > stix-math-fonts > stp > symmetrica > sympy > tcd-utils > TeXmacs > tgif > tideEditor > TOPCOM > vaspview > veusz > vinci > wgrib > wgrib2 > why > why3 > wise2 > wvs-data > xdrawchem > xgap > xtide > zenon > > can anyone even guess what package group that is? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to
Re: Semi-serious proposal: drop all optional entries from comps
On Sun, Sep 23, 2018 at 10:41:31AM +0200, Nicolas Mailhot wrote: > Le samedi 22 septembre 2018 à 11:38 -0700, Adam Williamson a écrit : > > > > But at the same time I think Matt's right that comps -at least as it's > > currently set up - is kind of a really *bad* way of doing this, and > > that seems fairly well demonstrated by the problem I'm trying to solve > > - that comps has tons of broken entries in it. Just from looking > > through the file as I do this, I really suspect that no-one is > > maintaining quite a lot of the groups and the packages in them may > > well just not make much sense any more. > > That's a QA problem > > But really, the problem is not the comps format, it's that any kind of > classification activity is both tedious and optional (if you know how to > classify a package you do not need the classification in the first > place), rots fast in presence of high package churn, and can only work > long term with lots of janitorial involvement (both human and > automated). That was my first thought too. On Fri, Sep 21, 2018 at 06:14:51PM -0700, Adam Williamson wrote: > I am currently working on finding and removing all comps entries that > point to packages which don't exist any more. > > While doing this extremely tedious task [...] I would love to see some more info before any decisions are made. I *thought* I more-or-less understand the comps process, I remember doing some minor changes in the past, but things seem to have changed. How are you doing those checks? Is it https://pagure.io/fedora-comps/pull-request/328? It should be possible to turn those checks into CI hooks. There was some jenkins jobs being run on PRs in pagure.io/fedora-comps, but it seems to have atrophied and doesn't appear on new PRs. Do we have any automated checks for comps now? Instead of designing a whole new system that needs to be a) designed, b) agreed, c) documented, d) implemented, e) integrated with all the other tools, and f) referenced in all the docs where the old system is now referenced, why not first fix the CI to do the checks? Pruning nonexistent packages really sounds like something that can automatized. Another question: people keep mentioning unexpected breakage from changes by random contributors. But https://pagure.io/fedora-comps/ seems *very* tightly controlled with write access by only @releng and adamw. So is this still actually a problem? Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Le lundi 24 septembre 2018 à 00:57 +0200, Kevin Kofler a écrit : > > Another issue if you want to use metapackages for categorization is > that UIs > such as Dnfdragora, or whatever tool converts the metapackages to a > representation those UIs will consume, would have to be told what > packages > are such categorization metapackages. That's trivially solvable by requiring they are named metapackage-something and contain a metapackage(something) provides That would not change the fact such categorization helps tend to rot over time if not cared about. They would regularly miss composes due to broken deps. -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Mon, 2018-09-24 at 00:57 +0200, Kevin Kofler wrote: > Neal Gompa wrote: > > This is where I think that transforming comps into metapackages would > > probably solve the remaining issues we have with the current workflow. > > I think metapackages could work for the distro composes, where ACL > enforcement is wanted (so we would remove @group usage from kickstarts > entirely and use the metapackages instead), but not for self-serve > categorization of the average niche leaf package. The metapackage would not > be open to all packagers (at most to provenpackagers), so it would just mean > you would have to send a pull request to the metapackage rather than to > fedora-comps. That would not really make things easier. You could use > Supplements (reverse weak dependencies) to self-serve-add your package to > the metapackage, but I am not sure that we would want such widespread use of > Supplements. Yeah, I think now we're extending the discussion a bit, any 'solution' really needs to address this use case problem. We're doing multiple really quite different things with package groups, and a significant part of the current awkwardness is that they're not really separable as they should be. I think some kind of easy, self-service, fairly sloppily'-controlled' system is fine for the "let's let package managers give some sort of categorized browsing catalog" use case. It's certainly not fine for the "let's build the distribution" use case. It would be kinda nuts for the distro to be built out of categories that any packager can tag any package into at any time, for instance. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Pierre-Yves Chibon wrote: > Cons: > - it's a free-form input field, so packager could add anything and > everything including things we do not care about. > -> So maybe we need a list of categories we care about somewhere and we > only integrate packages having one or more of these categories and > ignore all the other ones - not part of the specfile (but additional metadata), so: i) also amounts to an additional thing to fill out (with the potential to get forgotten), ii) cannot be verified during review (for new packages), iii) has to be filled in after the review is completed (which increases the potential that it gets forgotten for new packages). This does not mean that it could not possibly work, but this drawback has to be kept in mind. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Neal Gompa wrote: > This is where I think that transforming comps into metapackages would > probably solve the remaining issues we have with the current workflow. I think metapackages could work for the distro composes, where ACL enforcement is wanted (so we would remove @group usage from kickstarts entirely and use the metapackages instead), but not for self-serve categorization of the average niche leaf package. The metapackage would not be open to all packagers (at most to provenpackagers), so it would just mean you would have to send a pull request to the metapackage rather than to fedora-comps. That would not really make things easier. You could use Supplements (reverse weak dependencies) to self-serve-add your package to the metapackage, but I am not sure that we would want such widespread use of Supplements. Another issue if you want to use metapackages for categorization is that UIs such as Dnfdragora, or whatever tool converts the metapackages to a representation those UIs will consume, would have to be told what packages are such categorization metapackages. Out of the box, they would just be packages like any other, not browsable in UIs. The switch from @group to metapackages for kickstarts would also make things worse in some ways though. E.g., it is currently possible to pick a group minus one or two packages: @group -groupmember1 -groupmember2 If you use Requires in the metapackage, the kickstart has no way to do that at all. If you use Recommends in the metapackage (or Supplements in the individual package), the kickstart can do the above, but any update the user does will drag the excluded group members back in (because of https://github.com/openSUSE/libsolv/issues/168 , which libsolv upstream refuses to fix), which is also clearly not what we want. So the only workaround would be to ignore the metapackage and list every wanted package explicitly, which makes me wonder why we would carry the metapackage at all. So to keep this working with metapackages, somebody would have to address libsolv issue 168. I think that, if spin composes are the only things comps would still be used for and if tools like Dnfdragora were moved to something different (such as resurrected RPM Group tags), then we should actually consider inlining the comps groups into the spin kickstarts (for common parts across several spins, a kickstart include can be used, this is already done in several places of fedora-kickstarts). I think having the kickstart contents configured in a single place (fedora-kickstarts) rather than two (fedora-kickstarts and fedora-comps) would be an improvement. If no user of comps were left, we could then drop comps entirely, but do not forget that it is currently used by Dnfdragora and also by the traditional (non-live) installer modes of Anaconda. As for metapackages, I think they cause more problems than they solve, at least in the current state of our tooling, as detailed above. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Small correction: I wrote: > part of the review process, and fedora-review can automatically print an > error for missing Group tags (just as there is currently one if Group is > missing). I meant "just as there is currently one if Group is present". Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, Sep 22, 2018 at 12:26:31PM -0400, Matthew Miller wrote: > On Sat, Sep 22, 2018 at 01:07:27PM +0200, Kevin Kofler wrote: > > So I see only 2 alternatives: > > a) keep comps as it is now, including optional packages, OR > > b) undeprecate the RPM Group tag, readd it to all Fedora packages, and > >switch back to it. > > Any other plan will completely break Dnfdragora. > > Well, or "find plan c and work to make sure it's integrated in future versions > of dnfdragora". The RPM Group tag is very inflexible — even beyond the > "dewey decimal system" problem where the categories we had weren't very > forward-thinking, they don't allow packages to be part of more than one > group, and depend on the package maintainer rather than on group curation. Stupid idea, pagure offers the possibility to "tag" projects. Could this be useful? We could then export this list of tags via a cron job and reintegrate this data where we then want/need it. Pros: - support multiple tags (comma delimited) - in the hands of the packager (cf Kevin Kofler's email) - not in comps Cons: - it's a free-form input field, so packager could add anything and everything including things we do not care about. -> So maybe we need a list of categories we care about somewhere and we only integrate packages having one or more of these categories and ignore all the other ones Pierre ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sun, Sep 23, 2018 at 2:13 PM Kevin Kofler wrote: > > Matthew Miller wrote: > > Well, or "find plan c and work to make sure it's integrated in future > > versions of dnfdragora". > > That would have to be done BEFORE we drop the comps entries though. > > > The RPM Group tag is very inflexible — even beyond the "dewey decimal > > system" problem where the categories we had weren't very forward-thinking, > > We don't have to use the historical Red Hat categories. We already mass- > removed the old Groups from specfiles, so we can add new ones now. > > We could use any of: > * https://en.opensuse.org/openSUSE:Package_group_guidelines > * https://wiki.mageia.org/en/RPM_groups_policy > * any other existing category list from another distro, > * a new list defined specifically for Fedora. > > This is purely a policy issue. Software consuming the groups will not care > about what the list of groups is. It just needs to make sense to the users. > I'd be happy to help design a new list for Fedora, if we decide to go down this road. > > they don't allow packages to be part of more than one group, > > That can be seen as a drawback, but also as a feature. We frown upon > .desktop files showing up in more than one menu category. The same arguments > can be used here. Of course, it can make sense to have an application under > two categories (e.g., Amarok both under KDE Applications and under Sound), > but there are also arguments for picking one. > There was a proposal in RPM upstream to add a concept of "tags"/"meta tags", which could be a set of properties that an RPM could be categorized under. However, the idea was not fleshed out enough for us to seriously consider it as something to support in RPM. However, I'm not sure multi-categorization makes sense beyond setting up installation groups. > Overlap can also often be avoided by judicious definition of groups. (E.g., > we probably would not have a "KDE Applications" group, the "KDE" or "Plasma" > group would only contain the KDE Plasma Desktop Workspace itself.) The > current @kde-apps comps group is mainly needed to compose images, but we > would not use the RPM Group tags for that anyway. > > > and depend on the package maintainer rather than on group curation. > > This one, I would definitely consider a feature! > > Even now with comps, the policy says that the package maintainer is > responsible for adding the package to the comps group. The main reason it is > often forgotten is that this is yet another place to touch, outside of the > specfile. If we require the tag in the specfile instead, it will be part of > the review process, and fedora-review can automatically print an error for > missing Group tags (just as there is currently one if Group is missing). For > existing packages, we need a mass change just like "Remove Group" was. And > the really nice thing is: removed packages can never clutter a group listing > because they will just be gone, and their Group tag will be gone along with > them. So the current problem that we have obsolete packages in the comps > groups would go away completely! > This is where I think that transforming comps into metapackages would probably solve the remaining issues we have with the current workflow. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Matthew Miller wrote: > Well, or "find plan c and work to make sure it's integrated in future > versions of dnfdragora". That would have to be done BEFORE we drop the comps entries though. > The RPM Group tag is very inflexible — even beyond the "dewey decimal > system" problem where the categories we had weren't very forward-thinking, We don't have to use the historical Red Hat categories. We already mass- removed the old Groups from specfiles, so we can add new ones now. We could use any of: * https://en.opensuse.org/openSUSE:Package_group_guidelines * https://wiki.mageia.org/en/RPM_groups_policy * any other existing category list from another distro, * a new list defined specifically for Fedora. This is purely a policy issue. Software consuming the groups will not care about what the list of groups is. It just needs to make sense to the users. > they don't allow packages to be part of more than one group, That can be seen as a drawback, but also as a feature. We frown upon .desktop files showing up in more than one menu category. The same arguments can be used here. Of course, it can make sense to have an application under two categories (e.g., Amarok both under KDE Applications and under Sound), but there are also arguments for picking one. Overlap can also often be avoided by judicious definition of groups. (E.g., we probably would not have a "KDE Applications" group, the "KDE" or "Plasma" group would only contain the KDE Plasma Desktop Workspace itself.) The current @kde-apps comps group is mainly needed to compose images, but we would not use the RPM Group tags for that anyway. > and depend on the package maintainer rather than on group curation. This one, I would definitely consider a feature! Even now with comps, the policy says that the package maintainer is responsible for adding the package to the comps group. The main reason it is often forgotten is that this is yet another place to touch, outside of the specfile. If we require the tag in the specfile instead, it will be part of the review process, and fedora-review can automatically print an error for missing Group tags (just as there is currently one if Group is missing). For existing packages, we need a mass change just like "Remove Group" was. And the really nice thing is: removed packages can never clutter a group listing because they will just be gone, and their Group tag will be gone along with them. So the current problem that we have obsolete packages in the comps groups would go away completely! Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Le samedi 22 septembre 2018 à 11:38 -0700, Adam Williamson a écrit : > > But at the same time I think Matt's right that comps -at least as it's > currently set up - is kind of a really *bad* way of doing this, and > that seems fairly well demonstrated by the problem I'm trying to solve > - that comps has tons of broken entries in it. Just from looking > through the file as I do this, I really suspect that no-one is > maintaining quite a lot of the groups and the packages in them may > well just not make much sense any more. That's a QA problem But really, the problem is not the comps format, it's that any kind of classification activity is both tedious and optional (if you know how to classify a package you do not need the classification in the first place), rots fast in presence of high package churn, and can only work long term with lots of janitorial involvement (both human and automated). Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Le samedi 22 septembre 2018 à 11:29 -0700, Adam Williamson a écrit : > On Sat, 2018-09-22 at 10:26 -0700, Kevin Fenzi wrote: > > On 09/21/2018 06:14 PM, Adam Williamson wrote: > > ...snip... > > > The old gnome-packagekit, IIRC, also parsed groups and showed you > > > all > > > this stuff. > > > > > > But...we don't do that any more. anaconda does not expose > > > 'optional' > > > packages in any way any more (you can only pick environment groups > > > and > > > their supplementary package groups in anaconda, now). GNOME > > > Software > > > doesn't either. > > > > There's one other place they are used (although perhaps not much): > > dnf group install --with-optional groupname > > will install the group and all the optional stuff. > > > > I have no idea if many people use that... > > I am betting it's just about zero, because it seems really unlikely > that anyone wants *all* of the optional packages in a group. Why not? Some of the groups are pretty focused. You install defaults for groups you need, but do not use heavily, and the full optional package for things you are really interested in. Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
> On Sat, Sep 22, 2018 at 11:39:51AM -0700, Adam Williamson wrote: > > > Somewhere halfway between those, in that I've talked with the GNOME > > > Software > > > dev about it in the past. Software presents "Categories" and "Featured > > > Applications" -- the idea would be to extend that into also including > > > curated bundles. One click to select everything recommended as "Fedora > > > Design Suite". > > That would be great *if* we could use the same mechanism for actually > > doing the composes. If not it'd be pointless duplication of work and > > just introduce more errors. > > Yeah, we probably want the same mechanism for composing release-blocking > artifacts. > > Well, actually: anyone should be able define such a group, and anyone > produce images (live media, vm/cloud images, container images) from those > groups (possibly in Copr). Then we can promote certain sets of these as > official, and others can be like Coprs are now. Some of the group stuff is also used during the compose and if things aren't in groups specified but needed by say a kickstart the packages won't be in certain places and will break things. I did this by accident when adding zram in F-29 and not adding it to comps. Peter ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, Sep 22, 2018 at 11:39:51AM -0700, Adam Williamson wrote: > > Somewhere halfway between those, in that I've talked with the GNOME Software > > dev about it in the past. Software presents "Categories" and "Featured > > Applications" -- the idea would be to extend that into also including > > curated bundles. One click to select everything recommended as "Fedora > > Design Suite". > That would be great *if* we could use the same mechanism for actually > doing the composes. If not it'd be pointless duplication of work and > just introduce more errors. Yeah, we probably want the same mechanism for composing release-blocking artifacts. Well, actually: anyone should be able define such a group, and anyone produce images (live media, vm/cloud images, container images) from those groups (possibly in Copr). Then we can promote certain sets of these as official, and others can be like Coprs are now. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, Sep 22, 2018 at 2:39 PM Adam Williamson wrote: > > On Sat, 2018-09-22 at 13:07 +0200, Kevin Kofler wrote: > > Adam Williamson wrote: > > > While doing this extremely tedious task, it occurred to me to think: > > > what the hell is the *point* of these 'optional' entries any more, > > > anyway? > > > > They are required for Dnfdragora to list the available packages in a > > categorized manner. Dropping them without replacement is clearly the wrong > > approach, it will badly break Dnfdragora. (Experience with Apper has shown > > that users see browsing groups as an absolutely required feature of a > > package manager. The uncategorized "all packages" list or searches by name > > or description are not suitable replacements for most users.) > > > > If we see maintaining comps as too much of a burden (which is somewhat true, > > because it requires touching files outside of the packages, which has become > > even more tedious when the move to Pagure removed write permissions to the > > comps repository from almost all packagers, forcing us to go through pull > > requests), > > This is, however, a major win in another regard: we can stop people > breaking comps during freezes. Which *did* happen. Quite often. We've > more than once had a candidate compose fail or have a release-blocking > bug because someone tried to change kickstarts or comps during the > freeze, and got it wrong. > > That kinda points up a problem in that we have a single thing - comps - > which serves multiple roles with kinda different requirements. It's not > *really* a big problem to anyone if some group which is not included on > any media, and just exists to be shown in package managers like > dnfdragora, has some bad entries in it. It's absolutely a problem if > someone decides to change @core or @anaconda-tools or @workstation- > product and gets it wrong. It's kinda bad that these things have to be > in the same project, in the same *file*, and thus subject to the same > policies for modification, with the result that there's always going to > be some suboptimal result (either making it too easy to change the > really important stuff, or too hard to change the unimportant stuff). > > > then we should just undeprecate the RPM Group tag and move back > > to that. Dnfdragora already supports Group tags out of the box. (In fact, > > moving to them would allow Dnfdragora upstream to remove the special-case > > code for Fedora.) > > > > The rationale for deprecating RPM Group tags was that comps should be used > > instead. But if we want to get rid of that use of comps (since a comps > > without optional packages is no longer a suitable replacement for Group > > enumeration! A lot of packages that users will want to install will not be > > listed there anymore), what speaks against using Group again? > > > > So I see only 2 alternatives: > > a) keep comps as it is now, including optional packages, OR > > b) undeprecate the RPM Group tag, readd it to all Fedora packages, and > >switch back to it. > > Any other plan will completely break Dnfdragora. > > That's a fair point. I didn't really follow the 'no group tags' > proposal closely and hadn't noticed it suggested comps as an > alternative. > Personally, as dnfdragora upstream, I'd definitely prefer if we killed special cases like this. > But at the same time I think Matt's right that comps -at least as it's > currently set up - is kind of a really *bad* way of doing this, and > that seems fairly well demonstrated by the problem I'm trying to solve > - that comps has tons of broken entries in it. Just from looking > through the file as I do this, I really suspect that no-one is > maintaining quite a lot of the groups and the packages in them may well > just not make much sense any more. > Actually, we might want to consider adopting a variant of SUSE's approach. Their "patterns" used to work the same way our comps do now. However, they made the transition from externally developed and separately injected metadata to being generated from specially set up metapackages. Those metapackages could be controlled by the same kind of ACLs we have now for regular packages, and they could be used as the input to automatically generate comps.xml data, as SUSE does for generating pattern info (for versions of SUSE Linux that don't automatically map pattern requests to metapackage install requests). This would have the added advantage of simplifying our repodata creation process and allow people the flexibility to define their own in a way that the package manager can easily pick up. The only downside to us dropping comps.xml metadata is the loss of translations, unless we support the rpmmd extension SUSE developed for translating package data. I suppose we'd also lose the "inheritance merge" model of comps across repos, but I'm not sure that was a good idea to begin with... (Which of course means we really have to actually standardize the extension so that all the tools can
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, 2018-09-22 at 12:30 -0400, Matthew Miller wrote: > On Fri, Sep 21, 2018 at 07:14:37PM -0700, Adam Williamson wrote: > > > Long (or, in my dreams, medium) term, I want to see the things that are > > > now > > > "Labs" be instead software bundles that can be installed via GNOME > > > Software > > > and other tools. > > When you say 'software bundle' are you thinking of some specific > > mechanism here, or just the vague concept? > > Somewhere halfway between those, in that I've talked with the GNOME Software > dev about it in the past. Software presents "Categories" and "Featured > Applications" -- the idea would be to extend that into also including > curated bundles. One click to select everything recommended as "Fedora > Design Suite". That would be great *if* we could use the same mechanism for actually doing the composes. If not it'd be pointless duplication of work and just introduce more errors. > As I mentioned on IRC, I was really hoping that Modularity would provide us > with a good way to define those bundles, but at this point I'm far from set > on that as the approach. Hope springs eternal! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, 2018-09-22 at 13:07 +0200, Kevin Kofler wrote: > Adam Williamson wrote: > > While doing this extremely tedious task, it occurred to me to think: > > what the hell is the *point* of these 'optional' entries any more, > > anyway? > > They are required for Dnfdragora to list the available packages in a > categorized manner. Dropping them without replacement is clearly the wrong > approach, it will badly break Dnfdragora. (Experience with Apper has shown > that users see browsing groups as an absolutely required feature of a > package manager. The uncategorized "all packages" list or searches by name > or description are not suitable replacements for most users.) > > If we see maintaining comps as too much of a burden (which is somewhat true, > because it requires touching files outside of the packages, which has become > even more tedious when the move to Pagure removed write permissions to the > comps repository from almost all packagers, forcing us to go through pull > requests), This is, however, a major win in another regard: we can stop people breaking comps during freezes. Which *did* happen. Quite often. We've more than once had a candidate compose fail or have a release-blocking bug because someone tried to change kickstarts or comps during the freeze, and got it wrong. That kinda points up a problem in that we have a single thing - comps - which serves multiple roles with kinda different requirements. It's not *really* a big problem to anyone if some group which is not included on any media, and just exists to be shown in package managers like dnfdragora, has some bad entries in it. It's absolutely a problem if someone decides to change @core or @anaconda-tools or @workstation- product and gets it wrong. It's kinda bad that these things have to be in the same project, in the same *file*, and thus subject to the same policies for modification, with the result that there's always going to be some suboptimal result (either making it too easy to change the really important stuff, or too hard to change the unimportant stuff). > then we should just undeprecate the RPM Group tag and move back > to that. Dnfdragora already supports Group tags out of the box. (In fact, > moving to them would allow Dnfdragora upstream to remove the special-case > code for Fedora.) > > The rationale for deprecating RPM Group tags was that comps should be used > instead. But if we want to get rid of that use of comps (since a comps > without optional packages is no longer a suitable replacement for Group > enumeration! A lot of packages that users will want to install will not be > listed there anymore), what speaks against using Group again? > > So I see only 2 alternatives: > a) keep comps as it is now, including optional packages, OR > b) undeprecate the RPM Group tag, readd it to all Fedora packages, and >switch back to it. > Any other plan will completely break Dnfdragora. That's a fair point. I didn't really follow the 'no group tags' proposal closely and hadn't noticed it suggested comps as an alternative. But at the same time I think Matt's right that comps -at least as it's currently set up - is kind of a really *bad* way of doing this, and that seems fairly well demonstrated by the problem I'm trying to solve - that comps has tons of broken entries in it. Just from looking through the file as I do this, I really suspect that no-one is maintaining quite a lot of the groups and the packages in them may well just not make much sense any more. Of course, inventing some kind of Third Way requires someone to commit the time to doing that, and I'm not sure I'm gonna have that right now. So perhaps we can do something somewhat less drastic, like trimming some of the groups and throwing away ones that are clearly not maintained any more, like the scientific lab one. Another idea that occurred to me is that we could perhaps sneak in a sort of 'rings-lite' concept by splitting up comps. We could split up the bits into maybe three chunks: 1. bits that are basically there as compose glue - groups that aren't user-visible but *are* pulled into compose deliverables 2. critical user-visible groups that are often also part of the compose, like the Edition groups and environments, the kde group, the xfce group 3. Less important user-visible groups that just aren't that significant and don't constitute part of the compose, or at least not the release- blocking bits Something like that. Then the bits could have different access policies. All this would really require would be some kinda little script to merge the necessary bits when running composes, I guess... -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, 2018-09-22 at 10:26 -0700, Kevin Fenzi wrote: > On 09/21/2018 06:14 PM, Adam Williamson wrote: > ...snip... > > The old gnome-packagekit, IIRC, also parsed groups and showed you all > > this stuff. > > > > But...we don't do that any more. anaconda does not expose 'optional' > > packages in any way any more (you can only pick environment groups and > > their supplementary package groups in anaconda, now). GNOME Software > > doesn't either. > > There's one other place they are used (although perhaps not much): > dnf group install --with-optional groupname > will install the group and all the optional stuff. > > I have no idea if many people use that... I am betting it's just about zero, because it seems really unlikely that anyone wants *all* of the optional packages in a group. That's kinda why they were made optional in the first place, presumably. So I didn't mention this, though you're right that it technically exists. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On 09/21/2018 06:14 PM, Adam Williamson wrote: ...snip... > The old gnome-packagekit, IIRC, also parsed groups and showed you all > this stuff. > > But...we don't do that any more. anaconda does not expose 'optional' > packages in any way any more (you can only pick environment groups and > their supplementary package groups in anaconda, now). GNOME Software > doesn't either. There's one other place they are used (although perhaps not much): dnf group install --with-optional groupname will install the group and all the optional stuff. I have no idea if many people use that... > So do we really need these acres of 'optional' packages in comps? I > mean, there are 2519 of them in comps-f30.xml.in. That's a lot. I > suspect no-one's looked whether most of them make any sense for years. > There are entire groups that are *nothing but optional packages*, > making them almost entirely useless. > > So, OK, I think there are probably apps out there that still expose > this info. I suspect dnfdragora does, for instance. But is the minor > benefit of having these lists of 'hey maybe you'd like this thing' in > minor package managers really worth the way they turn comps into even > more of a gigantic crufty ball-of-wax than it would otherwise be? > > Is anyone really super-attached to this kinda stuff? I'm not. ;) I think we have moved on to where people look for an install apps or tools for their specific need, rather than installing everything in that area. kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, Sep 22, 2018 at 01:14:47PM +0200, Kevin Kofler wrote: > > +1, drop 'em. > Could you please consider the impact on the software we ship to our users, > and thus on our users themselves, before rushing to vote, without giving any > kind of rationale, for dropping essential functionality without a > replacement? The problem is that the current categorization is broken and badly maintained. It's not *really* providing the functionality it is intended to. Given that, I don't think leaving it is better than dropping it. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Fri, Sep 21, 2018 at 07:14:37PM -0700, Adam Williamson wrote: > > Long (or, in my dreams, medium) term, I want to see the things that are now > > "Labs" be instead software bundles that can be installed via GNOME Software > > and other tools. > When you say 'software bundle' are you thinking of some specific > mechanism here, or just the vague concept? Somewhere halfway between those, in that I've talked with the GNOME Software dev about it in the past. Software presents "Categories" and "Featured Applications" -- the idea would be to extend that into also including curated bundles. One click to select everything recommended as "Fedora Design Suite". As I mentioned on IRC, I was really hoping that Modularity would provide us with a good way to define those bundles, but at this point I'm far from set on that as the approach. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Sat, Sep 22, 2018 at 01:07:27PM +0200, Kevin Kofler wrote: > So I see only 2 alternatives: > a) keep comps as it is now, including optional packages, OR > b) undeprecate the RPM Group tag, readd it to all Fedora packages, and >switch back to it. > Any other plan will completely break Dnfdragora. Well, or "find plan c and work to make sure it's integrated in future versions of dnfdragora". The RPM Group tag is very inflexible — even beyond the "dewey decimal system" problem where the categories we had weren't very forward-thinking, they don't allow packages to be part of more than one group, and depend on the package maintainer rather than on group curation. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Matthew Miller wrote: > +1, drop 'em. Could you please consider the impact on the software we ship to our users, and thus on our users themselves, before rushing to vote, without giving any kind of rationale, for dropping essential functionality without a replacement? Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
Adam Williamson wrote: > While doing this extremely tedious task, it occurred to me to think: > what the hell is the *point* of these 'optional' entries any more, > anyway? They are required for Dnfdragora to list the available packages in a categorized manner. Dropping them without replacement is clearly the wrong approach, it will badly break Dnfdragora. (Experience with Apper has shown that users see browsing groups as an absolutely required feature of a package manager. The uncategorized "all packages" list or searches by name or description are not suitable replacements for most users.) If we see maintaining comps as too much of a burden (which is somewhat true, because it requires touching files outside of the packages, which has become even more tedious when the move to Pagure removed write permissions to the comps repository from almost all packagers, forcing us to go through pull requests), then we should just undeprecate the RPM Group tag and move back to that. Dnfdragora already supports Group tags out of the box. (In fact, moving to them would allow Dnfdragora upstream to remove the special-case code for Fedora.) The rationale for deprecating RPM Group tags was that comps should be used instead. But if we want to get rid of that use of comps (since a comps without optional packages is no longer a suitable replacement for Group enumeration! A lot of packages that users will want to install will not be listed there anymore), what speaks against using Group again? So I see only 2 alternatives: a) keep comps as it is now, including optional packages, OR b) undeprecate the RPM Group tag, readd it to all Fedora packages, and switch back to it. Any other plan will completely break Dnfdragora. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Fri, 2018-09-21 at 21:55 -0400, Matthew Miller wrote: > On Fri, Sep 21, 2018 at 06:14:51PM -0700, Adam Williamson wrote: > > So do we really need these acres of 'optional' packages in comps? I > > mean, there are 2519 of them in comps-f30.xml.in. That's a lot. I > > suspect no-one's looked whether most of them make any sense for years. > > There are entire groups that are *nothing but optional packages*, > > making them almost entirely useless. > > +1, drop 'em. > > Long (or, in my dreams, medium) term, I want to see the things that are now > "Labs" be instead software bundles that can be installed via GNOME Software > and other tools. When you say 'software bundle' are you thinking of some specific mechanism here, or just the vague concept? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Semi-serious proposal: drop all optional entries from comps
On Fri, Sep 21, 2018 at 06:14:51PM -0700, Adam Williamson wrote: > So do we really need these acres of 'optional' packages in comps? I > mean, there are 2519 of them in comps-f30.xml.in. That's a lot. I > suspect no-one's looked whether most of them make any sense for years. > There are entire groups that are *nothing but optional packages*, > making them almost entirely useless. +1, drop 'em. Long (or, in my dreams, medium) term, I want to see the things that are now "Labs" be instead software bundles that can be installed via GNOME Software and other tools. I think these kind of functional bundles are a better replacement for the old idea of comps (and before that, rpm groups!), and maybe we could use some, like, more modern and more distributed system for assembling those. > why > why3 > wise2 > wvs-data > xdrawchem > xgap > xtide > zenon > > can anyone even guess what package group that is? From the packages, I guess something sciency? (Cheats...) Yes, Engineering and Scientific. So, exactly something that seems suited to a functional bundle. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
