Re: [gentoo-dev] RFC: Portage local package revisions
On Sun, Oct 22, 2006 at 09:42:44PM +0200, Marius Mauch wrote: On Sat, 21 Oct 2006 13:39:26 -0700 Brian Harring [EMAIL PROTECTED] wrote: -r* is an ebuild convention; upstream (exemption of older daft portage releases) doesn't use it, as such we define it; should define it as simple as possible without castrating it's use. So to you having to understand two slightly different comparison algorithms is simpler than one? Can't agree with that, the simplest defintion for `bar` is `see foo` if `foo` is already known. Except the majority of folk think that version comparison is strict int comparison, when it's not. And as for the final letter in versions/revisions: If upstream sometimes prefers this naming scheme, why are you so sure that other people (users) won't prefer it? Folks use [a-z] for version components... which is fine; it's seperate from revision however. Which is saner, -r1.05.2 is less then -r1.4.2 (ignoring whether using .05 in the rev is sane), or -r1.05.2 being -r1.5.2, being greater then 1.4.2 ~harring pgpkIuKk6Jm9i.pgp Description: PGP signature
Re: [gentoo-dev] RFC: Portage local package revisions
On Sun, Oct 22, 2006 at 09:42:44PM +0200, Marius Mauch wrote: On Sat, 21 Oct 2006 13:39:26 -0700 Brian Harring [EMAIL PROTECTED] wrote: -r* is an ebuild convention; upstream (exemption of older daft portage releases) doesn't use it, as such we define it; should define it as simple as possible without castrating it's use. So to you having to understand two slightly different comparison algorithms is simpler than one? Can't agree with that, the simplest defintion for `bar` is `see foo` if `foo` is already known. And as for the final letter in versions/revisions: If upstream sometimes prefers this naming scheme, why are you so sure that other people (users) won't prefer it? I don't think we need the versioning to be this complex. All we want to provide is the capability for people to have local patch levels. As far as the =apache-2.0.58-r2.1.3 issue not working in dependencies, we can just ignore everything after the major revision number in dependency calculation and only compare it against apache-2.0.58-r2. I'm going to hack up a patch with floated revision numbers, but in one sense I think my original patch is actually a little better: It will make it obvious to ebuild developers that this is a feature they shouldn't be using in gentoo-x86 tree. It would also be a little easier to drop the -l1 in apache-2.0.58-r2-l1 than it would be to remove certain parts of -r#.# from a version number. Marius -- gentoo-dev@gentoo.org mailing list -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: Portage local package revisions
On Sat, 21 Oct 2006 13:39:26 -0700 Brian Harring [EMAIL PROTECTED] wrote: -r* is an ebuild convention; upstream (exemption of older daft portage releases) doesn't use it, as such we define it; should define it as simple as possible without castrating it's use. So to you having to understand two slightly different comparison algorithms is simpler than one? Can't agree with that, the simplest defintion for `bar` is `see foo` if `foo` is already known. And as for the final letter in versions/revisions: If upstream sometimes prefers this naming scheme, why are you so sure that other people (users) won't prefer it? Marius -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: Portage local package revisions
субота, 21. жовтень 2006 01:05, [EMAIL PROTECTED] Ви написали: [...] I'm writing to ask for your opinion on a change to sys-apps/portage that would allow users to maintain local revisions of ebuilds, such as net-www/apache-2.0.58-r2-local1. [...] Um, I am not clear on what exactly your approach proposes that is not already available via just using a local portage overlay? From the description or cited cases this seems to be exactly this feature, or am I missing something? George -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: Portage local package revisions
On Fri, Oct 20, 2006 at 05:26:00PM -0700, Brian Harring wrote: On Fri, Oct 20, 2006 at 11:05:22PM +, [EMAIL PROTECTED] wrote: Hello all, In designing an enterprise infrastructure around Gentoo at my place of employment, I have discovered a feature that would improve Gentoo's usefulness greatly in this field. I'm writing to ask for your opinion on a change to sys-apps/portage that would allow users to maintain local revisions of ebuilds, such as net-www/apache-2.0.58-r2-local1. This would require a modification to the ebuild version specifications and a patch to two portage source files which you can review here: Use -rX.Y instead; existing portage will choke on it the same as it'll choke on -local, upshot of -rX.Y is that it's less chars, bit more clear in the intention of trying to sneak an additional version component betwee -rN and -rN+1 I suppose I see the benefit of this and as long as it accomplishes the same thing I'm looking for here, I would not be opposed. However, I think having a local version of apache-2.0.58 resulting in apache-2.0.58-r0.1 is kinda silly. Regardless of which method we choose, it needs to be strictly enforced that packages in the gentoo-x86 tree NOT use these revision numbers or it will ruin the whole point of them. http://dev.gentoo.org/~malverian/portage_local_version.patch (Please don't complain about the code quality, I cleaned up areas where it was desparately needed such as the string.atof() areas, but for the sake of code coherence, I tried to use the same methodology used elsewhere in the code as much as possible, such as unqualified except statements :P) Bah. If you know that catch-all except's are bad, that means you can't use that excuse for having it in your new code :P I get a little slack since the patch is simply a proof of concept ;) 2) You are using binary packages and need to simulate a version bump to force re-installation of a binary package with modified USE flags. 3) You are using binary packages and need to simulate a version bump to force re-installation of binary packages that were rebuilt during revdep-rebuild Failing here is that it's not a 'simulated' verbump, it *is* a verbump for any deps that are locked via =; =dev-util/diffball-0.7.1 will not pick up =dev-util/diffball-0.7.1-local1 since =dev-util/diffball-0.7.1 is implicitly =dev-util/diffball-0.7.1-local0 . Folks can live with that issue, but should be clear that its there (same for revbumps of course, just picking at the wording). This is a good point, but I think the types of people that would be using this feature will not have a problem making changes like this. Optionally I guess we could drop the -local# in dep calculation? It's two separate pieces of code already as you can see from the patch. In all of the above cases, one could simply bump the package up one revision by creating an ebuild in an overlay for apache-2.0.58-r3. However, using this solution will result in apache not being upgraded when apache-2.0.58-r3 is actually committed to the portage tree unless you perpetuate this bad habit ad nauseum. This particular issue (force usage of an ebuild from PORTDIR if the installed exact version is from an overlay) can be addressed without changing version rules, although admittedly it's a single revbump, no way to do multiple bumps (again, wording is all). I don't see any benefit to this method over the one I proposed. - The new (completely backward compatible) version priority order would be: apache-2.0.58 apache-2.0.58-r1 apache-2.0.58-r2 apache-2.0.58-r2-local1 apache-2.0.58-r2-local2 apache-2.0.58-r3 apache-2.0.59 portage_version.ver_regexp specifically anchors -r\d+ to the end of the string, as such this is *not* backwards compatible, 2.1 would choke if it saw these versions in the vdb or in an overlay... That's correct, the regexp in current portage does require -r# to be anchored to the end. However, since the gentoo-x86 tree will not have packages with -local# someone would have to specifically try to use this new feature in an old version of portage in order to have issues. This argument here can also be applied to the -r#.# solution you mentioned, so I think the decision between -r#.# and -local# is really just a matter of aesthetics. I'm on the fence as to which is best. Would you be in support of a change that allowed for either -r#.# or -local# local revisions? ~harring -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: Portage local package revisions
Instead of -rY-localX, I do -rX0Y the following in my local overlays. This gets the same effect and maintains both version numbers. And if you are worried about a revision number exceeding 99, then just do -rX00Y. This works without requiring code change to portage. [EMAIL PROTECTED] wrote:[Fri Oct 20 2006, 07:05:22PM EDT] Hello all, In designing an enterprise infrastructure around Gentoo at my place of employment, I have discovered a feature that would improve Gentoo's usefulness greatly in this field. I'm writing to ask for your opinion on a change to sys-apps/portage that would allow users to maintain local revisions of ebuilds, such as net-www/apache-2.0.58-r2-local1. This would require a modification to the ebuild version specifications and a patch to two portage source files which you can review here: http://dev.gentoo.org/~malverian/portage_local_version.patch (Please don't complain about the code quality, I cleaned up areas where it was desparately needed such as the string.atof() areas, but for the sake of code coherence, I tried to use the same methodology used elsewhere in the code as much as possible, such as unqualified except statements :P) There are a number of scenarios where such a feature is useful, most of which revolve around the need for a local version bump: 1) You have machines using apache-2.0.58-r2 with an unpatched security hole which you would like to immediately patch locally until a fix is committed to the portage tree. 2) You are using binary packages and need to simulate a version bump to force re-installation of a binary package with modified USE flags. 3) You are using binary packages and need to simulate a version bump to force re-installation of binary packages that were rebuilt during revdep-rebuild In all of the above cases, one could simply bump the package up one revision by creating an ebuild in an overlay for apache-2.0.58-r3. However, using this solution will result in apache not being upgraded when apache-2.0.58-r3 is actually committed to the portage tree unless you perpetuate this bad habit ad nauseum. To give a better explanation of #3, consider the following scenario: - You have 60 servers with mysql-4.0.28 and php-5.1.6-r2 - You want to upgrade to mysql-5.0.30 and continue to use php-5.1.6-r2 - You use binary packages which are built on a staging machine - Your servers know to upgrade via a pull method with the help of cfengine which tells each server what packages SHOULD be installed on it. This also makes it very easy later to build another copy of a machine in the case of hardware failure by using the same description files (cfengine config programs) To accomplish the above in an enterprise environment, you would need to perform the following steps: 1) Install mysql-5.0.30 on the staging machine and build binary packages 2) Perform a revdep-rebuild on all packages using libmysql client libraries, building new binary packages for each of them 3) Tell your other 60 servers that it is time to upgrade mysql (and in this case, reinstall php) Assuming you have a description file such as the following: __CUT__ webserver_packages = ( dev-db/mysql-4.0.28 dev-lang/php-5.1.6-r2 ) ... __CUT__ It is obvious what change must be made to install the new version of MySQL, but how do you update PHP without bumping the version of PHP? You would need some extra metadata to tell the servers if they are upgrading from mysql-4.0.28 to mysql-5.0.30 that they should reinstall PHP. This is fine for a few packages, but it can quickly become a maintanence nightmare. Having local revision numbers solves this problem very simply, and provides quite a bit of flexibility as a side-effect. NOTES: - The -local# revisions MUST NOT be used in the main portage tree. It is something that system administrators would have exclusively for their own purposes such as those described above. This SHOULD probably be enforced by repoman in addition to policy changes. - The new (completely backward compatible) version priority order would be: apache-2.0.58 apache-2.0.58-r1 apache-2.0.58-r2 apache-2.0.58-r2-local1 apache-2.0.58-r2-local2 apache-2.0.58-r3 apache-2.0.59 -- gentoo-dev@gentoo.org mailing list Joel Martin (kanaka) Open Source no BILL . no GATES Costs nothing . Open to all pgp7wmNuCLafO.pgp Description: PGP signature
Re: [gentoo-dev] RFC: Portage local package revisions
On Sat, 21 Oct 2006 12:51:19 + Philip Walls [EMAIL PROTECTED] wrote: | This argument here can also be applied to the -r#.# solution you | mentioned, so I think the decision between -r#.# and -local# is really | just a matter of aesthetics. I'm on the fence as to which is best. The -r#.#.#.# solution is cleaner IMO. With -local# you'd still need to handle -local#.# or -local#-reallylocal-# to allow users to override overlays that override the main repository. -- Ciaran McCreesh Mail: ciaranm at ciaranm.org Web : http://ciaranm.org/ as-needed is broken : http://ciaranm.org/show_post.pl?post_id=13 signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: Portage local package revisions
Simon Stelling wrote: [Sat Oct 21 2006, 09:27:24AM EDT] This only assures that your version will be preferred as long as the version number is the same, but is really not what malverian is after. true. -- Joel Martin (kanaka) Open Source no BILL . no GATES Costs nothing . Open to all pgp11kTppl20s.pgp Description: PGP signature
Re: [gentoo-dev] RFC: Portage local package revisions
On Sat, Oct 21, 2006 at 03:27:24PM +0200, Simon Stelling wrote: Joel Martin wrote: Instead of -rY-localX, I do -rX0Y the following in my local overlays. This gets the same effect and maintains both version numbers. And if you are worried about a revision number exceeding 99, then just do -rX00Y. This works without requiring code change to portage. This only assures that your version will be preferred as long as the version number is the same, but is really not what malverian is after. That being said, I think this whole debate is really up to the portage team. After all, this change won't affect the Gentoo Devs other than that they can use the feature for their networks too, if they like it. The people who have to maintain the code that implements the feature should just give an ACK/NACK and go on or leave it, IMO. Should I copy the gentoo-portage-dev ML on this? I posted to this mailing list originally per zmedico. -- Kind Regards, Simon Stelling Gentoo/AMD64 developer -- gentoo-dev@gentoo.org mailing list -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: Portage local package revisions
On Sat, Oct 21, 2006 at 02:27:19PM +, Philip Walls wrote: On Sat, Oct 21, 2006 at 02:34:08PM +0100, Ciaran McCreesh wrote: On Sat, 21 Oct 2006 12:51:19 + Philip Walls [EMAIL PROTECTED] wrote: | This argument here can also be applied to the -r#.# solution you | mentioned, so I think the decision between -r#.# and -local# is really | just a matter of aesthetics. I'm on the fence as to which is best. The -r#.#.#.# solution is cleaner IMO. With -local# you'd still need to handle -local#.# or -local#-reallylocal-# to allow users to override overlays that override the main repository. I have a feeling the above is a solution waiting for problem, but from a flexibility standpoint I actually really like it :) Adding this kind of revision number will convolute the version comparison a bit, and the dep calculation will have to be patched to allow this (currently it only allows for real floating point numbers). I think we'll want to refactor the vercmp() so that there is a separate function for comparing version numbers (eg. 1.5 vs. 1.100 vs. 1.02) which the revision comparison can use as well. Be aware that if you reuse the vercmp logic, you're getting the special case float comparison rules, meaning 1.02 is less then 1.1 in comparison... Wouldn't introduce that for rx.y personally unless you've got a good reason for it. ~harring pgp8ParXGG4T7.pgp Description: PGP signature
Re: [gentoo-dev] RFC: Portage local package revisions
On Sat, 21 Oct 2006 21:35:06 +0200 Marius Mauch [EMAIL PROTECTED] wrote: | Still have the issue with the = operator though, not sure which way | to go there: The = operator (without a *) shouldn't ever be used in ebuilds or profiles. Not using the ~ operator is asking for disaster... -- Ciaran McCreesh Mail: ciaranm at ciaranm.org Web : http://ciaranm.org/ as-needed is broken : http://ciaranm.org/show_post.pl?post_id=13 signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: Portage local package revisions
On Sat, 21 Oct 2006 20:40:45 +0100 Ciaran McCreesh [EMAIL PROTECTED] wrote: On Sat, 21 Oct 2006 21:35:06 +0200 Marius Mauch [EMAIL PROTECTED] wrote: | Still have the issue with the = operator though, not sure which way | to go there: The = operator (without a *) shouldn't ever be used in ebuilds or profiles. Not using the ~ operator is asking for disaster... Have to disagree there. While in most cases you're right, there are/will be the usual exceptions, for example if you need a subset of ~. Not that it really matters for this discussion though. Marius -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: Portage local package revisions
On Sat, Oct 21, 2006 at 09:35:06PM +0200, Marius Mauch wrote: On Sat, 21 Oct 2006 08:31:31 -0700 Brian Harring [EMAIL PROTECTED] wrote: Be aware that if you reuse the vercmp logic, you're getting the special case float comparison rules, meaning 1.02 is less then 1.1 in comparison... Wouldn't introduce that for rx.y personally unless you've got a good reason for it. Do you have a good reason to a) limit -r to X.Y instead of a full version specifier (i.e. -r1.2.3a)? While I've said -rx.y, -rx.y(.z)* would fly, avoiding a repeat of the base issue for overlays of overlays. That said, allowig [a-z] is daft imo; it's in version (non-rev) components now since it mirrors semi-common upstream practices. -r* is an ebuild convention; upstream (exemption of older daft portage releases) doesn't use it, as such we define it; should define it as simple as possible without castrating it's use. So lecture aside, [a-z] seems a bit a pointless; example above, could just do -r1.2.3.1 instead of -r1.2.3a b) use different semantics for [subversion component float comparison rules]? Better question; why spread it further? For version components (nonrev), float makes some sense to match some whacky upstreams, that said, -r* is a *ebuild* convention so their isn't any reason to continue it. Fairly sure most folk aren't aware of the float comparison rules anyways for version components ;) ~harring pgp4m9bYJk5Y4.pgp Description: PGP signature
[gentoo-dev] RFC: Portage local package revisions
Hello all, In designing an enterprise infrastructure around Gentoo at my place of employment, I have discovered a feature that would improve Gentoo's usefulness greatly in this field. I'm writing to ask for your opinion on a change to sys-apps/portage that would allow users to maintain local revisions of ebuilds, such as net-www/apache-2.0.58-r2-local1. This would require a modification to the ebuild version specifications and a patch to two portage source files which you can review here: http://dev.gentoo.org/~malverian/portage_local_version.patch (Please don't complain about the code quality, I cleaned up areas where it was desparately needed such as the string.atof() areas, but for the sake of code coherence, I tried to use the same methodology used elsewhere in the code as much as possible, such as unqualified except statements :P) There are a number of scenarios where such a feature is useful, most of which revolve around the need for a local version bump: 1) You have machines using apache-2.0.58-r2 with an unpatched security hole which you would like to immediately patch locally until a fix is committed to the portage tree. 2) You are using binary packages and need to simulate a version bump to force re-installation of a binary package with modified USE flags. 3) You are using binary packages and need to simulate a version bump to force re-installation of binary packages that were rebuilt during revdep-rebuild In all of the above cases, one could simply bump the package up one revision by creating an ebuild in an overlay for apache-2.0.58-r3. However, using this solution will result in apache not being upgraded when apache-2.0.58-r3 is actually committed to the portage tree unless you perpetuate this bad habit ad nauseum. To give a better explanation of #3, consider the following scenario: - You have 60 servers with mysql-4.0.28 and php-5.1.6-r2 - You want to upgrade to mysql-5.0.30 and continue to use php-5.1.6-r2 - You use binary packages which are built on a staging machine - Your servers know to upgrade via a pull method with the help of cfengine which tells each server what packages SHOULD be installed on it. This also makes it very easy later to build another copy of a machine in the case of hardware failure by using the same description files (cfengine config programs) To accomplish the above in an enterprise environment, you would need to perform the following steps: 1) Install mysql-5.0.30 on the staging machine and build binary packages 2) Perform a revdep-rebuild on all packages using libmysql client libraries, building new binary packages for each of them 3) Tell your other 60 servers that it is time to upgrade mysql (and in this case, reinstall php) Assuming you have a description file such as the following: __CUT__ webserver_packages = ( dev-db/mysql-4.0.28 dev-lang/php-5.1.6-r2 ) ... __CUT__ It is obvious what change must be made to install the new version of MySQL, but how do you update PHP without bumping the version of PHP? You would need some extra metadata to tell the servers if they are upgrading from mysql-4.0.28 to mysql-5.0.30 that they should reinstall PHP. This is fine for a few packages, but it can quickly become a maintanence nightmare. Having local revision numbers solves this problem very simply, and provides quite a bit of flexibility as a side-effect. NOTES: - The -local# revisions MUST NOT be used in the main portage tree. It is something that system administrators would have exclusively for their own purposes such as those described above. This SHOULD probably be enforced by repoman in addition to policy changes. - The new (completely backward compatible) version priority order would be: apache-2.0.58 apache-2.0.58-r1 apache-2.0.58-r2 apache-2.0.58-r2-local1 apache-2.0.58-r2-local2 apache-2.0.58-r3 apache-2.0.59 -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: Portage local package revisions
On Fri, Oct 20, 2006 at 11:05:22PM +, [EMAIL PROTECTED] wrote: Hello all, In designing an enterprise infrastructure around Gentoo at my place of employment, I have discovered a feature that would improve Gentoo's usefulness greatly in this field. I'm writing to ask for your opinion on a change to sys-apps/portage that would allow users to maintain local revisions of ebuilds, such as net-www/apache-2.0.58-r2-local1. This would require a modification to the ebuild version specifications and a patch to two portage source files which you can review here: Use -rX.Y instead; existing portage will choke on it the same as it'll choke on -local, upshot of -rX.Y is that it's less chars, bit more clear in the intention of trying to sneak an additional version component betwee -rN and -rN+1 http://dev.gentoo.org/~malverian/portage_local_version.patch (Please don't complain about the code quality, I cleaned up areas where it was desparately needed such as the string.atof() areas, but for the sake of code coherence, I tried to use the same methodology used elsewhere in the code as much as possible, such as unqualified except statements :P) Bah. If you know that catch-all except's are bad, that means you can't use that excuse for having it in your new code :P 2) You are using binary packages and need to simulate a version bump to force re-installation of a binary package with modified USE flags. 3) You are using binary packages and need to simulate a version bump to force re-installation of binary packages that were rebuilt during revdep-rebuild Failing here is that it's not a 'simulated' verbump, it *is* a verbump for any deps that are locked via =; =dev-util/diffball-0.7.1 will not pick up =dev-util/diffball-0.7.1-local1 since =dev-util/diffball-0.7.1 is implicitly =dev-util/diffball-0.7.1-local0 . Folks can live with that issue, but should be clear that its there (same for revbumps of course, just picking at the wording). In all of the above cases, one could simply bump the package up one revision by creating an ebuild in an overlay for apache-2.0.58-r3. However, using this solution will result in apache not being upgraded when apache-2.0.58-r3 is actually committed to the portage tree unless you perpetuate this bad habit ad nauseum. This particular issue (force usage of an ebuild from PORTDIR if the installed exact version is from an overlay) can be addressed without changing version rules, although admittedly it's a single revbump, no way to do multiple bumps (again, wording is all). - The new (completely backward compatible) version priority order would be: apache-2.0.58 apache-2.0.58-r1 apache-2.0.58-r2 apache-2.0.58-r2-local1 apache-2.0.58-r2-local2 apache-2.0.58-r3 apache-2.0.59 portage_version.ver_regexp specifically anchors -r\d+ to the end of the string, as such this is *not* backwards compatible, 2.1 would choke if it saw these versions in the vdb or in an overlay... ~harring pgpshCSYqk7sx.pgp Description: PGP signature
