Re: Package categorization and distribution construction
On Wed, Jan 18, 2012 at 8:04 AM, Bill Nottingham nott...@redhat.com wrote: Tom Callaway (tcall...@redhat.com) said: On 01/18/2012 09:30 AM, Bill Nottingham wrote: Later it was brought up that it may just be simpler to create a second file of metadata, similar to the comps file, that just contains lists of categorized packages. (i.e., #3 above.) Opinions? One of the ideas that we've been seriously considering implementing is a Fedora specific extension to DOAP, where the DOAP files would be checked into git. It should be straightforward to add a Fedora extension field for tags, allowing the maintainer to add tags, along with other useful metadata about the package that we don't know from other sources. Then, tagger/packages can simply read in the DOAP files and use those tags in addition to the user-generated ones. The concern that was raised from infrastructure is that scouring package git for DOAP (or any other metadata files) might be prohibitive in cost to do regularly. PackageDB would be simpler, of course (maybe have git hooks that populate PackageDB on commit?) We discussed pkgdb but one of the concerns was that this is information about binary packages and the part of pkgdb that we were thinking of keeping (as opposed to the portion that we want to hand off to tagger/packages) deals with the srpm/git repo level view. For similar reasons, the git repository isn't a natural fit for the information. It could be the interface to adding the information but if we wanted to do any querying of the information it would make more sense to have it available in something that deals with the built rpm level rather than source rpm/git repo. -Toshio -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On Thu, 2012-01-19 at 08:54 -0500, Bill Nottingham wrote: Peter Robinson (pbrobin...@gmail.com) said: Great idea, I would also love to see a clear out of the packages that aren't core/part of particular categories. MTAs in minimal would be one that comes to mind but there's lots of other examples. Yeah, I'd like to clean this up. The sad thing is that even if I define the minimal group as: kernel dracut util-linux systemd systemd-units initscripts yum selinux-policy-targeted policycoreutils it merely drops it from 524MB/186 packages to 503MB/152 packages. I could make it smaller by dropping yum/rpm and dependencies, but I can't in good conscience ship an *installation target* minimal install that doesn't allow you to get updates or add-in packages. A few interesting things: . The above is installed size, download size is only 101MB¹. . There are a lot of small deps. . The only two big packages that I see which _might_ be able to be trimmed (split into smaller ones, maybe) are python-libs and cracklib-dict. . glibc-common contains locale-archive which is 100MB installed (IIRC they mmap it, so code would need to be changed ... but a simple gzip takes that to 23MB which is like ~17% savings). . For all the gory details (on download size), upstream repoquery now support --qf on the tree output. Sample version (with above packages) at: http://james.fedorapeople.org/repoquery-deptree-size.txt . Doing install @core is actually smaller, and less packages than the above² 8. Which makes me assume something is missing from @core. ¹ rawhide: install above 8 packages Install 8 Packages (+134 Dependent packages) Total download size: 101 M Installed size: 445 M ² rawhide: install @core Install 41 Packages (+129 Dependent packages) Total download size: 72 M Installed size: 323 M Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
James Antill (ja...@fedoraproject.org) said: . Doing install @core is actually smaller, and less packages than the above² 8. Which makes me assume something is missing from @core. The kernel; it's brought in by anaconda for a minimal *install*, but not explicitly mentioned because it's not technically required in a chroot environment. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On Thu, Jan 19, 2012 at 11:48:38AM -0500, Przemek Klosowski wrote: On 01/19/2012 10:43 AM, Richard W.M. Jones wrote: I wrote a little graphical tool called rpmdepsize (it's in Fedora) which may be useful. Unfortunately it only works with a single package, eg: rpmdepsize kernel Interesting--but I tried it on my F15 box and it froze. I tried 'rpdepsize kernel' and on a very simple (no deps) package 'setup': it prints an error: (rpmdepsize:18625): LablGTK-CRITICAL **: GSourceFunc: callback raised an exception and just sits there with gray screen and message 'Loading kernel ...' It runs 'yum' at this point, so it's likely that yum is just being slow. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
Bill Nottingham wrote: These could be separate groups, (i.e., XFCE's 'Office Suite' group may not have LibreOffice). So there would be the ability to customize that. Yes, that makes sense. Right now, if you enable Sound and Video, you get Totem forced in (and with it, plenty of GNOME dependencies such as Tracker) even if you chose only the KDE Plasma Workspaces instead of GNOME. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On Wed, Jan 18, 2012 at 2:30 PM, Bill Nottingham nott...@redhat.com wrote: Following up with notes from FUDCon. Bill Nottingham (nott...@redhat.com) said: == Distribution construction == For this, we will continue to use groups in comps. PRO: - Don't have to change any distribution tools - Don't have to change kickstarts CON that can be fixed: - Doesn't allow for tracking what groups a user has installed - Doesn't allow for adjusting installations to new groups - The 'group removal' operation does not behave in a way users expect By using something like what's suggested in: http://lists.baseurl.org/pipermail/yum-devel/2010-December/007740.html we can make groups persistent objects in yum, such that it's stored what groups the user has installed, what packages are in those groups, and so on. This was discussed at FUDCon, and there was general agreement that this is the way to go. However, there are a couple of implementation notes here: 1) If we are both reorganizing groups (splitting, renaming, etc.) and making them first-class objects in yum, we need to do both *at the same time*. 2) The anaconda UI change is not landing in F-17. If what we're doing affects anaconda package selection deeply, we need to wait until F-18. I'm going to investigate how much of this can be done without significant anaconda changes; if it can't be done well, then we'll wait until F-18. == Package categorization for browsing == In PackageKit parlance, these are 'collections'. Here, I have a few proposals. 1) Continue to use groups in comps PRO: - Don't have to change any package tools CON: - Requires editing to list a package - Need to separate them from groups used for distribution construction 2) Give packages in tags. Sort packages for choosing by them Options would be: - as a header in the RPM package, extracted into metadata - as a entry in the package database, extracted into some useable form - as part of the pkgtags yum metadata PRO: - Decentralized. Allows packages to organize themselves CON: - Requires touching every package (if it's set in the package) - Requires RPM changes (if it's set in the package) - Requires a mechanism to generate metadata - Requires modifying packaging tools (mostly PackageKit) to support this metadata - Decentralized. Any package can mess up the display and organization by using a garbage tag. 3) Invent some similar curated listing, similar to groups in comps, but not in comps CON: - ... why bother, other than to just move the data? During the talk at FUDCon there was a lot of discussion about how to do tags for packages that would be set by the maintainer, that could be used in addition to the tags in the database set by the Fedora Tagger. One option was tags in a file in the package git repo, that could then be pulled into the metadata; the pros and cons of this were discussed. Later it was brought up that it may just be simpler to create a second file of metadata, similar to the comps file, that just contains lists of categorized packages. (i.e., #3 above.) Opinions? Great idea, I would also love to see a clear out of the packages that aren't core/part of particular categories. MTAs in minimal would be one that comes to mind but there's lots of other examples. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
Bill Nottingham wrote: Opinions? What I would like to see is to have desktop application (but not desktop environment) groups like Sound and Video include different default packages depending on the chosen desktop environment: GNOME users probably prefer GTK+/GNOME packages, but KDE Plasma Desktop users probably prefer KDE Platform or Qt packages, the defaults should account for that. The way I'd see that working in Anaconda would be to have a desktop selection before the package selection, so that the choice of default packages could then be keyed on the selected desktop environments. The current situation where you have to deselect everything other than KDE to get KDE Platform applications, and where the KDE group includes plenty of applications beyond the KDE Plasma Workspaces (multimedia applications etc.) exactly because of this is quite broken. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
Kevin Kofler (kevin.kof...@chello.at) said: Bill Nottingham wrote: Opinions? What I would like to see is to have desktop application (but not desktop environment) groups like Sound and Video include different default packages depending on the chosen desktop environment: GNOME users probably prefer GTK+/GNOME packages, but KDE Plasma Desktop users probably prefer KDE Platform or Qt packages, the defaults should account for that. The way I'd see that working in Anaconda would be to have a desktop selection before the package selection, so that the choice of default packages could then be keyed on the selected desktop environments. What we have handwavingly designed is that, in anaconda, you get a variety of install options: GNOME Desktop KDE Desktop XFCE Desktop Minimal Server etc., which would all be shorthands for a lists of group(s). Once you select one of these, you would get options for that install, so for GNOME/KDE/XFCE/etc you could have: Office Suite Development Environment while for the Minimal Server, the optional add-ons would be something like: Web Server DNS Server CIFS Server etc. These could be separate groups, (i.e., XFCE's 'Office Suite' group may not have LibreOffice). So there would be the ability to customize that. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
Peter Robinson (pbrobin...@gmail.com) said: Great idea, I would also love to see a clear out of the packages that aren't core/part of particular categories. MTAs in minimal would be one that comes to mind but there's lots of other examples. Yeah, I'd like to clean this up. The sad thing is that even if I define the minimal group as: kernel dracut util-linux systemd systemd-units initscripts yum selinux-policy-targeted policycoreutils it merely drops it from 524MB/186 packages to 503MB/152 packages. I could make it smaller by dropping yum/rpm and dependencies, but I can't in good conscience ship an *installation target* minimal install that doesn't allow you to get updates or add-in packages. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/19/2012 08:54 AM, Bill Nottingham wrote: Peter Robinson (pbrobin...@gmail.com) said: Great idea, I would also love to see a clear out of the packages that aren't core/part of particular categories. MTAs in minimal would be one that comes to mind but there's lots of other examples. Yeah, I'd like to clean this up. The sad thing is that even if I define the minimal group as: kernel dracut util-linux systemd systemd-units initscripts yum selinux-policy-targeted policycoreutils it merely drops it from 524MB/186 packages to 503MB/152 packages. I could make it smaller by dropping yum/rpm and dependencies, but I can't in good conscience ship an *installation target* minimal install that doesn't allow you to get updates or add-in packages. Bill Any headscratchers in the list or rpms? I just saw that policycoreutils had a Require for bunzip2, which it does not need. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8YK3EACgkQrlYvE4MpobOZKgCgk2ANmxNUQabb6H5FfgLh///t fXsAoM1qtNT5EYY4BpX1yGYYDWZX5cdt =lHF1 -END PGP SIGNATURE- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On Thu, Jan 19, 2012 at 1:54 PM, Bill Nottingham nott...@redhat.com wrote: Peter Robinson (pbrobin...@gmail.com) said: Great idea, I would also love to see a clear out of the packages that aren't core/part of particular categories. MTAs in minimal would be one that comes to mind but there's lots of other examples. Yeah, I'd like to clean this up. The sad thing is that even if I define the minimal group as: kernel dracut util-linux systemd systemd-units initscripts yum selinux-policy-targeted policycoreutils it merely drops it from 524MB/186 packages to 503MB/152 packages. I could make it smaller by dropping yum/rpm and dependencies, but I can't in good conscience ship an *installation target* minimal install that doesn't allow you to get updates or add-in packages. Yes, but its a good start and people can then review the dependencies on those to see what is relevant or otherwise. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On Thu, 19 Jan 2012 14:45:34 + Peter Robinson pbrobin...@gmail.com wrote: kernel dracut util-linux systemd systemd-units initscripts yum selinux-policy-targeted policycoreutils If anyone wants to see the full tree for any of these: repoquery --requires --recursive --output=ascii-tree pkgname -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On Thu, Jan 19, 2012 at 09:40:56AM -0500, Daniel J Walsh wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/19/2012 08:54 AM, Bill Nottingham wrote: Peter Robinson (pbrobin...@gmail.com) said: Great idea, I would also love to see a clear out of the packages that aren't core/part of particular categories. MTAs in minimal would be one that comes to mind but there's lots of other examples. Yeah, I'd like to clean this up. The sad thing is that even if I define the minimal group as: kernel dracut util-linux systemd systemd-units initscripts yum selinux-policy-targeted policycoreutils it merely drops it from 524MB/186 packages to 503MB/152 packages. I could make it smaller by dropping yum/rpm and dependencies, but I can't in good conscience ship an *installation target* minimal install that doesn't allow you to get updates or add-in packages. Bill Any headscratchers in the list or rpms? I just saw that policycoreutils had a Require for bunzip2, which it does not need. I wrote a little graphical tool called rpmdepsize (it's in Fedora) which may be useful. Unfortunately it only works with a single package, eg: rpmdepsize kernel Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On 01/19/2012 10:43 AM, Richard W.M. Jones wrote:
I wrote a little graphical tool called rpmdepsize (it's in Fedora)
which may be useful. Unfortunately it only works with a single
package, eg:
rpmdepsize kernel
Interesting--but I tried it on my F15 box and it froze. I tried
'rpdepsize kernel' and on a very simple (no deps) package 'setup':
it prints an error:
(rpmdepsize:18625): LablGTK-CRITICAL **: GSourceFunc: callback raised an
exception
and just sits there with gray screen and message 'Loading kernel ...'
The strace output just around the freeze:
[pid 18691] rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
[pid 18691] rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 18691] exit_group(0) = ?
Process 18691 detached
... read resumed , 4096)= 0
write(2, \n(rpmdepsize:18690): LablGTK-CRI..., 84
(rpmdepsize:18690): LablGTK-CRITICAL **: GSourceFunc: callback raised an
exception
) = 84
read(3, \34\0\v\1\3\0@\3\177\1\0\0\377\227\373\262\0, 4096) = 32
read(3, 0x8dd73a8, 4096) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, 0) = 0 (Timeout)
read(3, 0x8dd73a8, 4096) = -1 EAGAIN (Resource temporarily unavailable)
read(3, 0x8dd73a8, 4096) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, -1) = ?
ERESTART_RESTARTBLOCK (To be restarted)
--- {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=18691, si_status=0,
si_utime=140, si_stime=70} (Child exited) ---
restart_syscall(... resuming interrupted call ...) = 1
read(3, \n\3\v\1\4\0@\3\0\0.0\0\0\0\0\0\0\0\0\0\0..., 4096) = 64
read(3, 0x8dd73a8, 4096) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, 0) = 0 (Timeout)
read(3, 0x8dd73a8, 4096) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, -1) = 1 ([{fd=3,
revents=POLLIN}])
read(3, \17\0\v\1\3\0@\3\1\0\0\0\0.\0\0\0\0\0, 4096) = 32
read(3, 0x8dd73a8, 4096)= -1 EAGAIN (Resource
temporarily unavailable)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, 0) = 0 (Timeout)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, 0) = 0 (Timeout)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, 0) = 0 (Timeout)
read(3, 0x8dd73a8, 4096) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 2, -1
fd=4 is anon_inode:[eventfd]
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On Thu, Jan 19, 2012 at 03:43:43PM +, Richard W.M. Jones wrote: On Thu, Jan 19, 2012 at 09:40:56AM -0500, Daniel J Walsh wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/19/2012 08:54 AM, Bill Nottingham wrote: Peter Robinson (pbrobin...@gmail.com) said: Great idea, I would also love to see a clear out of the packages that aren't core/part of particular categories. MTAs in minimal would be one that comes to mind but there's lots of other examples. Yeah, I'd like to clean this up. The sad thing is that even if I define the minimal group as: kernel dracut util-linux systemd systemd-units initscripts yum selinux-policy-targeted policycoreutils it merely drops it from 524MB/186 packages to 503MB/152 packages. I could make it smaller by dropping yum/rpm and dependencies, but I can't in good conscience ship an *installation target* minimal install that doesn't allow you to get updates or add-in packages. Bill Any headscratchers in the list or rpms? I just saw that policycoreutils had a Require for bunzip2, which it does not need. I wrote a little graphical tool called rpmdepsize (it's in Fedora) which may be useful. Unfortunately it only works with a single package, eg: rpmdepsize kernel There is also rpmreaper. It does not allow to show the size of a package including all its dependencies (it would be a nice feature :-), but it is really good for examining dependencies, reverse dependencies and dependency cycles of installed packages. D. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
Following up with notes from FUDCon. Bill Nottingham (nott...@redhat.com) said: == Distribution construction == For this, we will continue to use groups in comps. PRO: - Don't have to change any distribution tools - Don't have to change kickstarts CON that can be fixed: - Doesn't allow for tracking what groups a user has installed - Doesn't allow for adjusting installations to new groups - The 'group removal' operation does not behave in a way users expect By using something like what's suggested in: http://lists.baseurl.org/pipermail/yum-devel/2010-December/007740.html we can make groups persistent objects in yum, such that it's stored what groups the user has installed, what packages are in those groups, and so on. This was discussed at FUDCon, and there was general agreement that this is the way to go. However, there are a couple of implementation notes here: 1) If we are both reorganizing groups (splitting, renaming, etc.) and making them first-class objects in yum, we need to do both *at the same time*. 2) The anaconda UI change is not landing in F-17. If what we're doing affects anaconda package selection deeply, we need to wait until F-18. I'm going to investigate how much of this can be done without significant anaconda changes; if it can't be done well, then we'll wait until F-18. == Package categorization for browsing == In PackageKit parlance, these are 'collections'. Here, I have a few proposals. 1) Continue to use groups in comps PRO: - Don't have to change any package tools CON: - Requires editing to list a package - Need to separate them from groups used for distribution construction 2) Give packages in tags. Sort packages for choosing by them Options would be: - as a header in the RPM package, extracted into metadata - as a entry in the package database, extracted into some useable form - as part of the pkgtags yum metadata PRO: - Decentralized. Allows packages to organize themselves CON: - Requires touching every package (if it's set in the package) - Requires RPM changes (if it's set in the package) - Requires a mechanism to generate metadata - Requires modifying packaging tools (mostly PackageKit) to support this metadata - Decentralized. Any package can mess up the display and organization by using a garbage tag. 3) Invent some similar curated listing, similar to groups in comps, but not in comps CON: - ... why bother, other than to just move the data? During the talk at FUDCon there was a lot of discussion about how to do tags for packages that would be set by the maintainer, that could be used in addition to the tags in the database set by the Fedora Tagger. One option was tags in a file in the package git repo, that could then be pulled into the metadata; the pros and cons of this were discussed. Later it was brought up that it may just be simpler to create a second file of metadata, similar to the comps file, that just contains lists of categorized packages. (i.e., #3 above.) Opinions? Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On 01/18/2012 09:30 AM, Bill Nottingham wrote: Later it was brought up that it may just be simpler to create a second file of metadata, similar to the comps file, that just contains lists of categorized packages. (i.e., #3 above.) Opinions? One of the ideas that we've been seriously considering implementing is a Fedora specific extension to DOAP, where the DOAP files would be checked into git. It should be straightforward to add a Fedora extension field for tags, allowing the maintainer to add tags, along with other useful metadata about the package that we don't know from other sources. Then, tagger/packages can simply read in the DOAP files and use those tags in addition to the user-generated ones. ~tom == Fedora Project -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
Tom Callaway (tcall...@redhat.com) said: On 01/18/2012 09:30 AM, Bill Nottingham wrote: Later it was brought up that it may just be simpler to create a second file of metadata, similar to the comps file, that just contains lists of categorized packages. (i.e., #3 above.) Opinions? One of the ideas that we've been seriously considering implementing is a Fedora specific extension to DOAP, where the DOAP files would be checked into git. It should be straightforward to add a Fedora extension field for tags, allowing the maintainer to add tags, along with other useful metadata about the package that we don't know from other sources. Then, tagger/packages can simply read in the DOAP files and use those tags in addition to the user-generated ones. The concern that was raised from infrastructure is that scouring package git for DOAP (or any other metadata files) might be prohibitive in cost to do regularly. PackageDB would be simpler, of course (maybe have git hooks that populate PackageDB on commit?) Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Package categorization and distribution construction
On 01/18/2012 11:04 AM, Bill Nottingham wrote: The concern that was raised from infrastructure is that scouring package git for DOAP (or any other metadata files) might be prohibitive in cost to do regularly. PackageDB would be simpler, of course (maybe have git hooks that populate PackageDB on commit?) Yeah, I could see that, although, I'm not convinced it's true, or that we'd need to scour it regularly. ~tom == Fedora Project -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Package categorization and distribution construction
AKA, taking a blowtorch to the comps file. TL;DR version - come to the talk at FUDCon! I'm here to propose a reworking of how we handle the data in the comps file. (If you don't know what that is, it likely doesn't concern you.) Currently, we have two main use cases for the package groups and listings in the comps file: 1) Distribution construction We define groups of packages, that are then organized into: - Live spins - Install media - Installation package sets (via package selection in anaconda, yum, kickstart, etc.) Examples would be 'core', 'base', 'gnome-desktop', 'electronics-lab'. 2) Package categorization for browsing We define lists of packages that can be individually selected from, in: - anaconda - assorted yum/PK frontends (gnome-packagekit, apper, yumex, etc.) Examples would be 'games'. In the context of the anaconda redesign, it occured to me that these two use cases really aren't aligned that much, and likely shouldn't be using the same data store. So, I propose that for Fedora 17, we split these use cases into two separate logical data stores (that could still live in the same file), and adjust our processes and technologies accordingly. Given that assumption, we need to simply define how we want to tackle the two cases. Here's what I proposed: == Distribution construction == For this, we will continue to use groups in comps. PRO: - Don't have to change any distribution tools - Don't have to change kickstarts CON that can be fixed: - Doesn't allow for tracking what groups a user has installed - Doesn't allow for adjusting installations to new groups - The 'group removal' operation does not behave in a way users expect By using something like what's suggested in: http://lists.baseurl.org/pipermail/yum-devel/2010-December/007740.html we can make groups persistent objects in yum, such that it's stored what groups the user has installed, what packages are in those groups, and so on. CON that still needs solved: - Doesn't allow for defining higher level organization of groups (other than categories, which suck from a spin and presentation perspective) An example: Someone wants to create a desktop product. They'll define the product as: MyDesktop -- @gnome-desktop |-- @fonts |-- @input-methods |-- @x11 |-- @core |-- @base They also want to define some set of addons that could be optionally presented in the installer, such as: MyDekstopAddons - @office-suite - @eclipse Right now, we merely have the 'categories' heirarchy in comps, which is pushed to the anaconda UI. Anything that is conceptually an addon (such as office-suite, or eclipse, is a common group that applies to any product that might use the groups. What would be good to have is something where each product that is defined can define its own addons. In any case, I intend to start working on this now. What this would mean is that for any group that is essentially a 'distribution construction' group, it becomes a single entity with *NO* optional components. It is also no longer exposed for post-install package selection. As part of this, we'd create more of these sorts of groups by subdividing some of the existing groups into smaller, feature based units. == Package categorization for browsing == In PackageKit parlance, these are 'collections'. Here, I have a few proposals. 1) Continue to use groups in comps PRO: - Don't have to change any package tools CON: - Requires editing to list a package - Need to separate them from groups used for distribution construction 2) Give packages in tags. Sort packages for choosing by them Options would be: - as a header in the RPM package, extracted into metadata - as a entry in the package database, extracted into some useable form - as part of the pkgtags yum metadata PRO: - Decentralized. Allows packages to organize themselves CON: - Requires touching every package (if it's set in the package) - Requires RPM changes (if it's set in the package) - Requires a mechanism to generate metadata - Requires modifying packaging tools (mostly PackageKit) to support this metadata - Decentralized. Any package can mess up the display and organization by using a garbage tag. 3) Invent some similar curated listing, similar to groups in comps, but not in comps CON: - ... why bother, other than to just move the data? == Summary == In any case, I'm pitching a FUDCon talk about this. So, if you're going to be at FUDCon Blacksburg, stop by... -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
