Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On 07/09/2017 06:53 AM, William L. Thomson Jr. wrote: > On Sun, 9 Jul 2017 00:42:46 -0700 > Daniel Campbell wrote: > >>> - Sets used in profiles cannot have use expansion, versions or >>> anything beyond cat/pkg. >> This would break some set behavior, at least in Portage. Specifying a >> single version (or better, a slot) in a set is less work than adding >> lines to p.mask *and* the set file(s), and p.mask doesn't appear to >> support "!=cat/pkg-1.0" syntax to mimic the same functionality >> achieved by a versioned atom in a set. It also makes sense to put >> packages you want in a set instead of a mask. ">=" or "<=" may be >> adequate if you only want one slot or version installed, but the >> entire point of slots is to allow multiple versions to be installed >> simultaneously. Versioned package names in sets achieve this. > > Valid point, and along those lines to make the rules for sets in > profiles easier. > > - Sets in profiles can contain anything that is valid in a > profile/packages file, less the * symbol. > > I think that addresses both versions and slots. The rest, like use > expansion I believe is handled via package.use in profiles and not in > packages. > Yeah, that could work. As convenient as it is to mix USE flags with sets, there's a better place to put it and I'm unsure of any situation that would require more than two lines (one in the set, one in p.use) to achieve a given USE constraint. -- Daniel Campbell - Gentoo Developer OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net fpr: AE03 9064 AE00 053C 270C 1DE4 6F7A 9091 1EA0 55D6 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, 8 Jul 2017 21:30:06 -0700 Zac Medico wrote: > > Yeah, but it's not going to happen without an EAPI/PMS extension. > There needs to be a standard way for the package manager to check if > there has been an upstream change for a given package since the last > time it was rebuilt. Something to add to the next version possibly. > > Could have cron run that, and then an interval in portage is not > > necessary. > > Doing updates via cron is inappropriate for most scenarios. I was > thinking that we'd have an option to rebuild if an interval has > expired *and* there has been an upstream change. That way, you can > limit the rate of rebuilds if upstream is changing very frequently. I was thinking of cron like regular system updates. May want to rebuild it nightly, etc. Interval can be done in a variety of ways. The main issue is detecting if upstream has changed or not. > > I was more thinking some sort of hook to upstream to see if > > there have been any commits or changes. No reason to rebuild a live > > package if nothing changed :) > > Yeah, that's what I was thinking all along. That's why we need an > EAPI/PMS extension, in order to implement behavior like > smart-live-rebuild. Hopefully that will be in a future version so such can be added. Up to PMS authors. -- William L. Thomson Jr. pgpLNRirHG5wH.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sun, 9 Jul 2017 00:42:46 -0700 Daniel Campbell wrote: > > - Sets used in profiles cannot have use expansion, versions or > > anything beyond cat/pkg. > This would break some set behavior, at least in Portage. Specifying a > single version (or better, a slot) in a set is less work than adding > lines to p.mask *and* the set file(s), and p.mask doesn't appear to > support "!=cat/pkg-1.0" syntax to mimic the same functionality > achieved by a versioned atom in a set. It also makes sense to put > packages you want in a set instead of a mask. ">=" or "<=" may be > adequate if you only want one slot or version installed, but the > entire point of slots is to allow multiple versions to be installed > simultaneously. Versioned package names in sets achieve this. Valid point, and along those lines to make the rules for sets in profiles easier. - Sets in profiles can contain anything that is valid in a profile/packages file, less the * symbol. I think that addresses both versions and slots. The rest, like use expansion I believe is handled via package.use in profiles and not in packages. -- William L. Thomson Jr. pgpg4_wmxj4MG.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On 07/08/2017 06:23 PM, William L. Thomson Jr. wrote: > On Sun, 9 Jul 2017 01:10:11 +0100 > Ciaran McCreesh wrote: > >> On Sat, 8 Jul 2017 19:58:13 -0400 >> "William L. Thomson Jr." wrote: >>> On Sun, 9 Jul 2017 00:49:57 +0100 >>> Ciaran McCreesh wrote: On Sat, 8 Jul 2017 19:39:33 -0400 "William L. Thomson Jr." wrote: > The two ways are not the same, and there is a reason sets exist > in the first place. People seem to be over looking that fact. I > did not add sets. They are not new. I am simply trying to > expand their use. Sets exist because people keep saying "let's have sets!" without agreeing on what sets actually are or how they are to be used. >>> >>> Do they need to agree? Isn't Gentoo about choice? Maybe your use of >>> sets is different from mine. Is that not acceptable to have >>> choice? >> >> Well yes, they do need to agree, because otherwise when two different >> developers put sets in a profile expecting different effects, then at >> least two developers are going to end up confused, disappointed, and >> quite probably breaking user systems. > > Valid points, so basically need a set of guidelines or rules for sets > used in profiles. Which should not be that complex, as usage is minimal. > Offhand, likely could be more; > > - Sets used in profiles are "lists of packages" for users to > emerge/re-emerge, and as such should be minimal list only. Similar to > the contents of a profile/packages, less the * symbol. > > - Sets used in profiles cannot have use expansion, versions or anything >beyond cat/pkg. This would break some set behavior, at least in Portage. Specifying a single version (or better, a slot) in a set is less work than adding lines to p.mask *and* the set file(s), and p.mask doesn't appear to support "!=cat/pkg-1.0" syntax to mimic the same functionality achieved by a versioned atom in a set. It also makes sense to put packages you want in a set instead of a mask. ">=" or "<=" may be adequate if you only want one slot or version installed, but the entire point of slots is to allow multiple versions to be installed simultaneously. Versioned package names in sets achieve this. > > - Sets should not have the same file listed, in that case inherit the > other set if using overlapping packages or split into smaller > -- Daniel Campbell - Gentoo Developer OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net fpr: AE03 9064 AE00 053C 270C 1DE4 6F7A 9091 1EA0 55D6 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, Jul 8, 2017 at 6:39 PM, William L. Thomson Jr. wrote: > On Sat, 8 Jul 2017 18:30:10 -0700 > Zac Medico wrote: > >> On Sat, Jul 8, 2017 at 4:46 PM, William L. Thomson Jr. >> wrote: >> > On Sat, 8 Jul 2017 16:35:34 -0700 >> > Zac Medico wrote: >> > >> >> For live-rebuild, it would be >> >> much nicer to have a framework that automatically triggers rebuilds >> >> when upstream changes are detected, like smart-live-rebuild. >> > >> > Which would require some sort of check to upstream to detect >> > changes on some interval. >> >> What I imagine is an option like --newuse that rebuilds packages with >> upstream changes. I suppose you could also have an option to rebuild >> if some interval has expired since the last rebuild. > > That could be useful for all live packages without requiring a set. It > could also be used for packages that are part of a set. Like if you > have a set of live ebuilds, plus some others one your system > > emerge --live world > > Updates all live ebuilds, in a set or not. That would be useful. I tend > to avoid emerging live ebuilds due to having to always re-emerge them. > Sets would help there. But a emerge option would be even better!!! Yeah, but it's not going to happen without an EAPI/PMS extension. There needs to be a standard way for the package manager to check if there has been an upstream change for a given package since the last time it was rebuilt. > Could have cron run that, and then an interval in portage is not > necessary. Doing updates via cron is inappropriate for most scenarios. I was thinking that we'd have an option to rebuild if an interval has expired *and* there has been an upstream change. That way, you can limit the rate of rebuilds if upstream is changing very frequently. > I was more thinking some sort of hook to upstream to see if > there have been any commits or changes. No reason to rebuild a live > package if nothing changed :) Yeah, that's what I was thinking all along. That's why we need an EAPI/PMS extension, in order to implement behavior like smart-live-rebuild. -- Thanks, Zac
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, 8 Jul 2017 18:30:10 -0700 Zac Medico wrote: > On Sat, Jul 8, 2017 at 4:46 PM, William L. Thomson Jr. > wrote: > > On Sat, 8 Jul 2017 16:35:34 -0700 > > Zac Medico wrote: > > > >> For live-rebuild, it would be > >> much nicer to have a framework that automatically triggers rebuilds > >> when upstream changes are detected, like smart-live-rebuild. > > > > Which would require some sort of check to upstream to detect > > changes on some interval. > > What I imagine is an option like --newuse that rebuilds packages with > upstream changes. I suppose you could also have an option to rebuild > if some interval has expired since the last rebuild. That could be useful for all live packages without requiring a set. It could also be used for packages that are part of a set. Like if you have a set of live ebuilds, plus some others one your system emerge --live world Updates all live ebuilds, in a set or not. That would be useful. I tend to avoid emerging live ebuilds due to having to always re-emerge them. Sets would help there. But a emerge option would be even better!!! Could have cron run that, and then an interval in portage is not necessary. I was more thinking some sort of hook to upstream to see if there have been any commits or changes. No reason to rebuild a live package if nothing changed :) -- William L. Thomson Jr. pgpk_2ZygI7q8.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, Jul 8, 2017 at 4:46 PM, William L. Thomson Jr. wrote: > On Sat, 8 Jul 2017 16:35:34 -0700 > Zac Medico wrote: > >> For live-rebuild, it would be >> much nicer to have a framework that automatically triggers rebuilds >> when upstream changes are detected, like smart-live-rebuild. > > Which would require some sort of check to upstream to detect changes on > some interval. What I imagine is an option like --newuse that rebuilds packages with upstream changes. I suppose you could also have an option to rebuild if some interval has expired since the last rebuild. -- Thanks, Zac
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sun, 9 Jul 2017 01:10:11 +0100 Ciaran McCreesh wrote: > On Sat, 8 Jul 2017 19:58:13 -0400 > "William L. Thomson Jr." wrote: > > On Sun, 9 Jul 2017 00:49:57 +0100 > > Ciaran McCreesh wrote: > > > On Sat, 8 Jul 2017 19:39:33 -0400 > > > "William L. Thomson Jr." wrote: > > > > The two ways are not the same, and there is a reason sets exist > > > > in the first place. People seem to be over looking that fact. I > > > > did not add sets. They are not new. I am simply trying to > > > > expand their use. > > > > > > Sets exist because people keep saying "let's have sets!" without > > > agreeing on what sets actually are or how they are to be used. > > > > Do they need to agree? Isn't Gentoo about choice? Maybe your use of > > sets is different from mine. Is that not acceptable to have > > choice? > > Well yes, they do need to agree, because otherwise when two different > developers put sets in a profile expecting different effects, then at > least two developers are going to end up confused, disappointed, and > quite probably breaking user systems. Valid points, so basically need a set of guidelines or rules for sets used in profiles. Which should not be that complex, as usage is minimal. Offhand, likely could be more; - Sets used in profiles are "lists of packages" for users to emerge/re-emerge, and as such should be minimal list only. Similar to the contents of a profile/packages, less the * symbol. - Sets used in profiles cannot have use expansion, versions or anything beyond cat/pkg. - Sets should not have the same file listed, in that case inherit the other set if using overlapping packages or split into smaller -- William L. Thomson Jr. pgpPXW60ZeBxe.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, 8 Jul 2017 19:58:13 -0400 "William L. Thomson Jr." wrote: > On Sun, 9 Jul 2017 00:49:57 +0100 > Ciaran McCreesh wrote: > > On Sat, 8 Jul 2017 19:39:33 -0400 > > "William L. Thomson Jr." wrote: > > > The two ways are not the same, and there is a reason sets exist in > > > the first place. People seem to be over looking that fact. I did > > > not add sets. They are not new. I am simply trying to expand > > > their use. > > > > Sets exist because people keep saying "let's have sets!" without > > agreeing on what sets actually are or how they are to be used. > > Do they need to agree? Isn't Gentoo about choice? Maybe your use of > sets is different from mine. Is that not acceptable to have choice? Well yes, they do need to agree, because otherwise when two different developers put sets in a profile expecting different effects, then at least two developers are going to end up confused, disappointed, and quite probably breaking user systems. -- Ciaran McCreesh
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sun, 9 Jul 2017 00:49:57 +0100 Ciaran McCreesh wrote: > On Sat, 8 Jul 2017 19:39:33 -0400 > "William L. Thomson Jr." wrote: > > The two ways are not the same, and there is a reason sets exist in > > the first place. People seem to be over looking that fact. I did > > not add sets. They are not new. I am simply trying to expand their > > use. > > Sets exist because people keep saying "let's have sets!" without > agreeing on what sets actually are or how they are to be used. Do they need to agree? Isn't Gentoo about choice? Maybe your use of sets is different from mine. Is that not acceptable to have choice? > Sets remain half-baked because it turns out they don't make > consistent sense in different contexts when you give them a > non-superficial amount of thought. Much of the world is half baked, but we manage despite such. There is much to be said about over thinking over engineering something. Few if anything in life is perfect. Many of the things we depend on are far from perfect. C'est la vie. This must be a Gentoo thing. Sets have nothing to do with me. Seems they have been around since ~2009. Someone coded in support for them etc. Seems some find use for them, and others have objection. Having been around Gentoo for some time, and just now coming across sets. I find them quite useful! Just ran into one limitation of them. -- William L. Thomson Jr. pgpgA_Jg_1aNe.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, 8 Jul 2017 19:39:33 -0400 "William L. Thomson Jr." wrote: > The two ways are not the same, and there is a reason sets exist in the > first place. People seem to be over looking that fact. I did not add > sets. They are not new. I am simply trying to expand their use. Sets exist because people keep saying "let's have sets!" without agreeing on what sets actually are or how they are to be used. Sets remain half-baked because it turns out they don't make consistent sense in different contexts when you give them a non-superficial amount of thought. -- Ciaran McCreesh
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, 8 Jul 2017 16:35:34 -0700 Zac Medico wrote: > On Sat, Jul 8, 2017 at 4:09 PM, William L. Thomson Jr. > wrote: > > Sets are also used for package rebuilds, like x11-module-rebuild, > > live-rebuild, and others. > > Usually there are better ways to trigger rebuilds. Those have nothing to do with me, and I am not using sets for rebuild purposes. Sets in a profile per my interest and topic have nothing to do with rebuilding. Just pointing out others current usage of sets. > For example, slot operator dependencies for rebuilds due to subslot > changes, and --newuse for USE changes. Problem is portage does not catch all changes all the time. Many times I will update world with binaries, Just to find out if I run without binaries more packages need to be updated. However if there are no changes, and the user wants to rebuild. How do they do that? Sets make it very easy. > For live-rebuild, it would be > much nicer to have a framework that automatically triggers rebuilds > when upstream changes are detected, like smart-live-rebuild. Which would require some sort of check to upstream to detect changes on some interval. If the user wants to do that manually. How do they do that? Sets make it very easy. > We can add EAPI/PMS extensions that allow package managers to do what > smart-live-rebuild does. Will that allow a user to rebuild on demand for their own reasons with no changes to system that are "detectable"? Again rebuilding is not my interest in sets. That is others usage of such. However they seem like they could serve a purpose in such situations. Giving the user really fine grain control without having to list a set of packages or mess with making various meta ebuilds. -- William L. Thomson Jr. pgpQKu4C4tITh.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, 8 Jul 2017 19:24:46 -0400 Rich Freeman wrote: > > I don't see why a package manager couldn't offer the same > functionality for a meta package. As was pointed out the set behavior > for unmerging isn't always desirable. Your missing that sets maybe made by the user, Making a meta ebuild is a bit more complex then dropping package names into a file for a set, no digest, etc. Sets seem to serve a different purpose. With regard to unmerging not being desirable. That is if someone does something stupid like putting a system package into a set. But as I mentioned that the package is part of another set, system or world. It would be pulled back into the system. There are warnings and other stuff that take place when a critical package is removed. A user can remove those now just as they could if they added it to a set. Which really makes that argument moot. Do dumb stuff, you will get undesired results. Like removing a system package, via any means, set, directly, meta dep, etc. > > > > world and system are sets we all have. Not sure about PMS. It is > > something portage has supported for some time. You likely have many > > sets already on your system > > Certainly. You just can't depend on them and so on without having > them in PMS, because portage isn't the only package manager we > "support." Not sure about other package managers, but I would think they have similar function. If not then anything listed under emerge --list-sets, would be portage specific. That would likely break other things. > It just strikes me that we're probably better off picking one way of > doing this and putting lots of support behind it, versus having two > ways of doing this and some features work with one but not the other. > Of the two meta packages seem like they're the most generic. The two ways are not the same, and there is a reason sets exist in the first place. People seem to be over looking that fact. I did not add sets. They are not new. I am simply trying to expand their use. If someone wants to "try" out some packages a set is an ideal way of doing such. If someone wants a subset of what is in a meta ebuild. Again a set is an ideal way to do that. If for no other reason, then if the user wants to remove them. They just emerge -C @my_set. I find sets very useful! Only limitation is the profile aspect. -- William L. Thomson Jr. pgpQ7noeVFu3a.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, Jul 8, 2017 at 4:09 PM, William L. Thomson Jr. wrote: > Sets are also used for package rebuilds, like x11-module-rebuild, > live-rebuild, and others. Usually there are better ways to trigger rebuilds. For example, slot operator dependencies for rebuilds due to subslot changes, and --newuse for USE changes. For live-rebuild, it would be much nicer to have a framework that automatically triggers rebuilds when upstream changes are detected, like smart-live-rebuild. We can add EAPI/PMS extensions that allow package managers to do what smart-live-rebuild does. -- Thanks, Zac
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, Jul 8, 2017 at 7:09 PM, William L. Thomson Jr. wrote: > On Sat, 8 Jul 2017 18:34:55 -0400 > Rich Freeman wrote: >> >> What do sets get us that packages do not? Why not move the other >> direction and just have packages instead of sets? > > The blog entry I provided a link to I think made the best case example > of usage of sets and their benefits. > https://makuro.wordpress.com/2010/12/12/intro-to-portage-sets/ > > The biggest advantage is ability to re-emerge all without additional > steps or arguments. Simple emerge @my_set just like emerge world, etc. > Even more useful you can remove a set directly, without depclean. I don't see why a package manager couldn't offer the same functionality for a meta package. As was pointed out the set behavior for unmerging isn't always desirable. > > world and system are sets we all have. Not sure about PMS. It is > something portage has supported for some time. You likely have many > sets already on your system Certainly. You just can't depend on them and so on without having them in PMS, because portage isn't the only package manager we "support." It just strikes me that we're probably better off picking one way of doing this and putting lots of support behind it, versus having two ways of doing this and some features work with one but not the other. Of the two meta packages seem like they're the most generic. -- Rich
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Sat, 8 Jul 2017 18:34:55 -0400 Rich Freeman wrote: > > What do sets get us that packages do not? Why not move the other > direction and just have packages instead of sets? The blog entry I provided a link to I think made the best case example of usage of sets and their benefits. https://makuro.wordpress.com/2010/12/12/intro-to-portage-sets/ The biggest advantage is ability to re-emerge all without additional steps or arguments. Simple emerge @my_set just like emerge world, etc. Even more useful you can remove a set directly, without depclean. If you remove a meta package, you have to run depclean to remove the actual packages in the meta ebuild. Sets save you from this step. Sets are also used for package rebuilds, like x11-module-rebuild, live-rebuild, and others. > One issue I see with sets is that as far as I can tell they aren't > specified in PMS at all, so they can't go into the tree at all, and > not all package managers may support them in the same way. Certainly > this could be standardized, but I'm not sure what they actually get > us. world and system are sets we all have. Not sure about PMS. It is something portage has supported for some time. You likely have many sets already on your system emerge --list-sets https://wiki.gentoo.org/wiki/World_set_(Portage) https://wiki.gentoo.org/wiki/System_set_(Portage) https://wiki.gentoo.org/wiki//etc/portage/sets https://dev.gentoo.org/~zmedico/portage/doc/ch02s02.html -- William L. Thomson Jr. pgpwHyLbUNjFu.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Sets vs Meta ebuilds
On Fri, Jul 7, 2017 at 10:21 PM, Michael Palimaka wrote: > > Bug #272488[0] proposed a PROPERTIES="set" feature to combine the power > of sets with the flexibility of ebuilds. > > 1: https://bugs.gentoo.org/show_bug.cgi?id=272488 > What do sets get us that packages do not? Why not move the other direction and just have packages instead of sets? One issue I see with sets is that as far as I can tell they aren't specified in PMS at all, so they can't go into the tree at all, and not all package managers may support them in the same way. Certainly this could be standardized, but I'm not sure what they actually get us. -- Rich