Re: [gentoo-dev] Re: Versioning the tree
On 11/28/06, Andrew Gaffney [EMAIL PROTECTED] wrote: You make it sound like releng doesn't care at all about non-desktop packages. That wasn't how it was meant. Was simply meant as a statement of fact. Releng activities are currently exclusively desktop-oriented. Until that changes, releng snapshots aren't fit for the purpose of being a non-moving tree, as far as servers are concerned. The reason for the exclusivity is that the media that's typically built for release (GRP, LiveCD) is targetted for the largest audience...desktop users. If someone wants to volunteer to create a set of server-related GRP and a server LiveCD (as silly as this is for most things), they wouldn't be blocked outright. I'd like to see some figures proving that our largest audience is desktop users. I'm not prepared to take that on faith. (Alas, producing these figures is non-trivial in the extreme, if not impossible). b) Release trees have a nasty habit of picking up last minute changes (such as gcc 4.1) to suit the release, not stability. Gcc 4.1.1 wasn't a last minute change. I can't agree with you there. It doesn't matter how many months of planning and work you guys put into getting gcc-4.1 fit for stable. If you're doing it off in your own little corner of the world, and then springing it on the rest of us just days before the release happens, then to the much larger dev community, it comes as a last minute change. If you're testing the crap out of something, but only in an exclusively desktop-oriented way ... well, that can only really be partial testing, can't it? The release tree isn't really for minimal breakage. But that is what Steve (who started this thread) asked for. And what he has asked for in his previous thread too. The *real* intent (at least from my POV) is to have a non-moving target for vendors to certify their software against (wouldn't it be nice for Oracle to be actually supported on Gentoo 2007.0 or something like that?), Well, there's a dichotamy here. Sun were able to certify Gentoo against their hardware without such a tree. Has anyone approached Oracle and asked them what their actual requirements are? Do Oracle actually want to certify Oracle on Gentoo at all? I personally deplore this habit of trying to second guess what someone else wants. Assumptions are the mother of all fuckups. Let's see an email to -dev from someone at Oracle w/ their shopping list of needs, and then base the discussion around that. and so admins don't have to do the upgrade dance once a week or even every day (like I do). A slower-moving tree will substantially reduce this amount of work, but it isn't going to go away, unless your boxes are on a private network w/ no local security threats at all. There'll always be GLSA's to respond to. That's another issue that needs to be handled w/ a slow-moving tree. Are you going to restrict changes in the slow-moving tree only to changes against a GLSA? The non-stagnant nature of Gentoo isn't the only reason that people use Gentoo. People use Gentoo for the configurability and customizability. As someone who admins more than a handful of Gentoo servers, I would absolutely *love* the combo of Gentoo's flexibility and a non-moving tree to make upgrades easier to deal with. I honestly don't think you're ever going to get that out of Gentoo, because of the lack of backporting. Can you live with a slower-moving tree? Or do you personally really need a non-moving tree? If you really need a non-moving tree, I think you're better off with RHES or Ubuntu. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
Stuart Herbert wrote: b) Release trees have a nasty habit of picking up last minute changes (such as gcc 4.1) to suit the release, not stability. Gcc 4.1.1 wasn't a last minute change. I can't agree with you there. It doesn't matter how many months of planning and work you guys put into getting gcc-4.1 fit for stable. If you're doing it off in your own little corner of the world, and then springing it on the rest of us just days before the release happens, then to the much larger dev community, it comes as a last minute change. The 3-4 weeks of releng filing a ton of doesn't build with gcc-4.1.1 bugs wasn't a big enough clue? :) Also, the arch teams (or at least the arch's release coordinator...if they didn't tell the rest of their team, that's not releng's fault) that were moving to it and people in base-system working on it were in the know. The release tree isn't really for minimal breakage. But that is what Steve (who started this thread) asked for. And what he has asked for in his previous thread too. Well, wolf31o2 has been floating around this idea for quite a while, and I'm speaking from the POV of his ideas. The minimal b0rkage tree is far less likely to happen due to the extra manpower involved. The *real* intent (at least from my POV) is to have a non-moving target for vendors to certify their software against (wouldn't it be nice for Oracle to be actually supported on Gentoo 2007.0 or something like that?), Well, there's a dichotamy here. Sun were able to certify Gentoo against their hardware without such a tree. Has anyone approached Oracle and asked them what their actual requirements are? Do Oracle actually want to certify Oracle on Gentoo at all? Certifying hardware and software are 2 completely different things. Also, I'm not sure that Sun certifying their hardware even meant anything. The T2000 dev box has been seeing random lockups and other weird problems, although, that may be related to the fact that it recently lost 16GB of memory due to an error detected by the diagnostics. As for the Oracle example, it was just that...an example. and so admins don't have to do the upgrade dance once a week or even every day (like I do). A slower-moving tree will substantially reduce this amount of work, but it isn't going to go away, unless your boxes are on a private network w/ no local security threats at all. There'll always be GLSA's to respond to. That's another issue that needs to be handled w/ a slow-moving tree. Are you going to restrict changes in the slow-moving tree only to changes against a GLSA? Yes, that's part of wolf31o2's idea. The tree would be non-moving except for GLSA's and any dependencies required by the updated version of the security-affected package. The non-stagnant nature of Gentoo isn't the only reason that people use Gentoo. People use Gentoo for the configurability and customizability. As someone who admins more than a handful of Gentoo servers, I would absolutely *love* the combo of Gentoo's flexibility and a non-moving tree to make upgrades easier to deal with. I honestly don't think you're ever going to get that out of Gentoo, because of the lack of backporting. Can you live with a slower-moving tree? Or do you personally really need a non-moving tree? A slower-moving (or non-moving with security updates) tree is perfect for me, and I'm sure for many other people as well. -- Andrew Gaffneyhttp://dev.gentoo.org/~agaffney/ Gentoo Linux Developer Installer Project Today's lesson in political correctness: Go asphyxiate on a phallus -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On 11/29/06, Andrew Gaffney [EMAIL PROTECTED] wrote: The 3-4 weeks of releng filing a ton of doesn't build with gcc-4.1.1 bugs wasn't a big enough clue? :) No. We get those all the time; there's always someone trying out an unsupported release of gcc. Also, the arch teams (or at least the arch's release coordinator...if they didn't tell the rest of their team, that's not releng's fault) that were moving to it and people in base-system working on it were in the know. What about everyone else, who isn't part of an arch team? Sorry, but I just don't get how you expected us to know it was coming, when you didn't announce it, and you don't even feel that an announcement was your (releng's) responsibility (even though stabilisation of gcc-4.1 was for you). You personally went out of your way earlier this year to critisise me about bad communication, just for announcing that work had started on something in Gentoo. And yet here you are today, somehow expecting the rest of us to rely on clairvoyance to know in advance about a change that your team pushed onto the entire tree. It's a great illustration why the releng snapshots, as things stand today, aren't the right way to deliver a meaningfully stable tree to our users. It's simply difficult to trust you with the way you choose to behave today. Yes, that's part of wolf31o2's idea. The tree would be non-moving except for GLSA's and any dependencies required by the updated version of the security-affected package. How are you going to ensure that all the security fixes that never get a GLSA get into the tree? A slower-moving (or non-moving with security updates) tree is perfect for me, and I'm sure for many other people as well. Before you release these trees to users, I trust you'll be doing the responsible thing, and ensuring that upgrades from one tree to the next work? You can't take that for granted; package maintainers cannot be relied upon to test upgrades spanning that length of time. (Which is why upgrade early, upgrade often is a practical way to manage Gentoo boxes) Best regards, Stu -- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
Stuart Herbert wrote: On 11/29/06, Andrew Gaffney [EMAIL PROTECTED] wrote: The 3-4 weeks of releng filing a ton of doesn't build with gcc-4.1.1 bugs wasn't a big enough clue? :) No. We get those all the time; there's always someone trying out an unsupported release of gcc. From other developers, most of which were releng members? Also, the arch teams (or at least the arch's release coordinator...if they didn't tell the rest of their team, that's not releng's fault) that were moving to it and people in base-system working on it were in the know. What about everyone else, who isn't part of an arch team? Sorry, but I just don't get how you expected us to know it was coming, when you didn't announce it, and you don't even feel that an announcement was your (releng's) responsibility (even though stabilisation of gcc-4.1 was for you). It's stupid to blame releng for the stabilization of gcc-4.1.1. It would have been stabilized soon as part of the normal process of package maintenance whether releng wanted it or not. And really, it was up to each arch to say they wanted it for the release, not releng. We didn't force it on people. And as always, if you wanted to know what was going on as part of the release, the info was there for everyone to see in the usual places (such as the gentoo-releng mailing list or the #gentoo-releng IRC channel). This argument about people not knowing what releng is doing seems to come up after every release. It's crap. Yes, that's part of wolf31o2's idea. The tree would be non-moving except for GLSA's and any dependencies required by the updated version of the security-affected package. How are you going to ensure that all the security fixes that never get a GLSA get into the tree? If it doesn't get a GLSA, it doesn't get in the release tree. This may mean that we need to modify the GLSA process a bit so that ~arch packages found to be vulnerable get GLSAs as well. Although, release tree users won't have these ~arch versions anyway, so I don't see it being *that* big of an issue. A slower-moving (or non-moving with security updates) tree is perfect for me, and I'm sure for many other people as well. Before you release these trees to users, I trust you'll be doing the responsible thing, and ensuring that upgrades from one tree to the next work? You can't take that for granted; package maintainers cannot be relied upon to test upgrades spanning that length of time. (Which is why upgrade early, upgrade often is a practical way to manage Gentoo boxes) Absolutely, it would be stupid to release it to users without testing that the upgrade path is even feasible. However, we can't test the upgrade with *all* packages in the tree, but we can certainly do it with certain profiles (gnome desktop, kde desktop, LAMP server, etc.) to try to cover as many bases as possible. This testing can be easily scripted. -- Andrew Gaffneyhttp://dev.gentoo.org/~agaffney/ Gentoo Linux Developer Installer Project Today's lesson in political correctness: Go asphyxiate on a phallus -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On Wednesday 29 November 2006 09:37, Stuart Herbert wrote: Gcc 4.1.1 wasn't a last minute change. I can't agree with you there. It doesn't matter how many months of planning and work you guys put into getting gcc-4.1 fit for stable. If you're doing it off in your own little corner of the world, and then springing it on the rest of us just days before the release happens, then to the much larger dev community, it comes as a last minute change. It was announced two months before 2006.1 was released. http://thread.gmane.org/gmane.linux.gentoo.devel/39848 -- Bo Andresen pgp1vg97DV5SS.pgp Description: PGP signature
Re: [gentoo-dev] Re: Versioning the tree
On Wed, 2006-11-29 at 08:37 +, Stuart Herbert wrote: On 11/28/06, Andrew Gaffney [EMAIL PROTECTED] wrote: You make it sound like releng doesn't care at all about non-desktop packages. That wasn't how it was meant. Was simply meant as a statement of fact. Releng activities are currently exclusively desktop-oriented. I'm sorry, but how the hell do you know? You are not a member of Release Engineering, and have *NO CLUE* what we do over there. What we release isn't the only thing we do. Until that changes, releng snapshots aren't fit for the purpose of being a non-moving tree, as far as servers are concerned. Luckily, I'm not asking you. Instead, I'm asking interested developers to assist us in making what we plan on doing much more viable. Feel free to sit over there and naysay until you're blue in the face. We'll be over here getting something accomplished via teamwork. b) Release trees have a nasty habit of picking up last minute changes (such as gcc 4.1) to suit the release, not stability. Gcc 4.1.1 wasn't a last minute change. I can't agree with you there. It doesn't matter how many months of planning and work you guys put into getting gcc-4.1 fit for stable. If you're doing it off in your own little corner of the world, and then springing it on the rest of us just days before the release happens, then to the much larger dev community, it comes as a last minute change. Except it was announced before we even made the snapshot, and we worked with the toolchain guys to get it to happen and entirely with their blessing. Just because we didn't take the time out to stop and make sure you were personally comfortable with the change doesn't mean we didn't prepare for it and announce it. If you're testing the crap out of something, but only in an exclusively desktop-oriented way ... well, that can only really be partial testing, can't it? Again, you don't know what you're talking about, so I'd really appreciate it if you just shut the hell up until you decide to get yourself informed on the facts. There'll always be GLSA's to respond to. That's another issue that needs to be handled w/ a slow-moving tree. Are you going to restrict changes in the slow-moving tree only to changes against a GLSA? That's what we've said. I honestly don't think you're ever going to get that out of Gentoo, because of the lack of backporting. Can you live with a slower-moving tree? Or do you personally really need a non-moving tree? If you really need a non-moving tree, I think you're better off with RHES or Ubuntu. While I truly appreciate your ability to give your opinion, I don't care. As I said, I am working on this concept as an experiment. It is being done by Release Engineering. We aren't really *asking* anyone for their opinion. We're simply stating what we plan on working on and will be asking people who *want* to participate to do so. Anyone not interested in participating in this Release Engineering-driven project is welcome to completely ignore us, as we will them. -- Chris Gianelloni Release Engineering Strategic Lead Alpha/AMD64/x86 Architecture Teams Games Developer/Council Member/Foundation Trustee Gentoo Foundation signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Re: Versioning the tree
On 11/29/06, Andrew Gaffney [EMAIL PROTECTED] wrote: From other developers, most of which were releng members? I get most of mine from users, who are normally kind enough to submit the required patches at the same time. It's stupid to blame releng for the stabilization of gcc-4.1.1. We didn't force it on people. You're responsible for it being stabilised when it was. It's disappointing that you choose to disavow all responsibility for that happening. I'm not arguing that gcc-4.1 shouldn't have been stabilised at some point in time (although the performance penalty it introduces is a shame). I'm just pointing out a flaw with the idea that the releng snapshots are fit for the purpose of being a non-moving tree for our users. And as always, if you wanted to know what was going on as part of the release, the info was there for everyone to see in the usual places (such as the gentoo-releng mailing list or the #gentoo-releng IRC channel). This argument about people not knowing what releng is doing seems to come up after every release. It's crap. I don't agree with you, sorry. Releng members in the past have complained about not knowing what is going on elsewhere in Gentoo; they have _specifically_ complained because information was not _actively_ sent in their direction. But, when the point is raised in the opposite direction, your answer is it's crap. I'm left very disappointed by that. If it doesn't get a GLSA, it doesn't get in the release tree. This may mean that we need to modify the GLSA process a bit so that ~arch packages found to be vulnerable get GLSAs as well. Although, release tree users won't have these ~arch versions anyway, so I don't see it being *that* big of an issue. It would be worth checking to make sure that all security bugs for all stable packages get GLSAs first. I don't know whether they do or they don't. Absolutely, it would be stupid to release it to users without testing that the upgrade path is even feasible. However, we can't test the upgrade with *all* packages in the tree, but we can certainly do it with certain profiles (gnome desktop, kde desktop, LAMP server, etc.) to try to cover as many bases as possible. This testing can be easily scripted. Cool. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
Chris Gianelloni wrote: While I truly appreciate your ability to give your opinion, I don't care. As I said, I am working on this concept as an experiment. It is being done by Release Engineering. We aren't really *asking* anyone for their opinion. We're simply stating what we plan on working on and will be asking people who *want* to participate to do so. Anyone not interested in participating in this Release Engineering-driven project is welcome to completely ignore us, as we will them. Now that I think about it, regardless of the technical merits of moving trees etc., Chris's statement here is more or less like the one issued by Stuart in the creation of the Seeds project. @Stuart: That's something to keep in mind. It looks like for the time being this will be an in-house project by releng, so if they want to spend the manpower, resources, and time to work on it, why not let 'em? After all, such reasoning went into the Seeds project, didn't it? signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Versioning the tree
On 11/29/06, Chris Gianelloni [EMAIL PROTECTED] wrote: I'm sorry, but how the hell do you know? You are not a member of Release Engineering, and have *NO CLUE* what we do over there. What we release isn't the only thing we do. Then this is a great opportunity to set the record straight, by explaining what server-oriented work releng do with each release. Luckily, I'm not asking you. Instead, I'm asking interested developers to assist us in making what we plan on doing much more viable. Feel free to sit over there and naysay until you're blue in the face. We'll be over here getting something accomplished via teamwork. Odd; I'm trying to get involved, by providing feedback and asking questions. Except it was announced before we even made the snapshot, Sorry, I've looked, but the only announcement I found on gentoo-dev was posted two days before gcc-4.1 was stabilised [1]. I must have missed the earlier announcement? Just because we didn't take the time out to stop and make sure you were personally comfortable with the change doesn't mean we didn't prepare for it and announce it. I'm sorry that you've gone with the I always know best, you're a fucking chump so shut the fuck up type of response :( That seems to be your answer of choice to all feedback these days. I'm sorry you feel that my input isn't welcome in your world. [1] http://marc.theaimsgroup.com/?l=gentoo-devm=115672836418923w=2 Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
Stuart Herbert wrote: Sorry, I've looked, but the only announcement I found on gentoo-dev was posted two days before gcc-4.1 was stabilised [1]. I must have missed the earlier announcement? Do you just ignore the rest of the thread when responding to individual emails? About 2 hours ago, someone posted the following link in this thread in response to your earlier post about gcc-4.1.1 being a last minute change. http://thread.gmane.org/gmane.linux.gentoo.devel/39848 -- Andrew Gaffneyhttp://dev.gentoo.org/~agaffney/ Gentoo Linux Developer Installer Project Today's lesson in political correctness: Go asphyxiate on a phallus -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On Wednesday 29 November 2006 18:11, Stuart Herbert wrote: Except it was announced before we even made the snapshot, Sorry, I've looked, but the only announcement I found on gentoo-dev was posted two days before gcc-4.1 was stabilised [1]. I must have missed the earlier announcement? Maybe you should read the replies you got the first time you made this claim on this list [1]. I've stated my opinion on the matter once at the gentoo-user mailing list [2] so I'll refrain from doing so here. Also just for completeness I repeat the link to the original announce [3] two months prior to the release. [1] http://thread.gmane.org/gmane.linux.gentoo.devel/41923/focus=41944 [2] http://thread.gmane.org/gmane.linux.gentoo.user/169041/focus=169455 [3] http://thread.gmane.org/gmane.linux.gentoo.devel/39848 -- Bo Andresen pgpg6IgkTv7zc.pgp Description: PGP signature
Re: [gentoo-dev] Re: Versioning the tree
Stuart Herbert wrote: Just because we didn't take the time out to stop and make sure you were personally comfortable with the change doesn't mean we didn't prepare for it and announce it. I'm sorry that you've gone with the I always know best, you're a fucking chump so shut the fuck up type of response :( That seems to be your answer of choice to all feedback these days. Please keep the name-calling off-list. While it is nice to have input on a particular project and we should strive to work together, remember that this is a project headed up by Chris and Release Engineering. As such they have the right to ignore all other input if they so choose. If their goals don't coincide with yours then that is a shame, but don't push it too far please. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
Hi, On 11/29/06, Bo Ørsted Andresen [EMAIL PROTECTED] wrote: Maybe you should read the replies you got the first time you made this claim on this list [1]. Many thanks for these links. I didn't see your original email. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On 11/29/06, Stuart Herbert [EMAIL PROTECTED] wrote: Hi, On 11/29/06, Bo Ørsted Andresen [EMAIL PROTECTED] wrote: Maybe you should read the replies you got the first time you made this claim on this list [1]. Many thanks for these links. I didn't see your original email. Wanna add a comment here: :-) http://bugs.gentoo.org/show_bug.cgi?id=141904 -Richard -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On Wed, 29 Nov 2006 13:01:37 -0700 Richard Fish [EMAIL PROTECTED] wrote: | On 11/29/06, Stuart Herbert [EMAIL PROTECTED] wrote: | On 11/29/06, Bo Ørsted Andresen [EMAIL PROTECTED] wrote: | Maybe you should read the replies you got the first time you made | this claim on this list [1]. | | Many thanks for these links. I didn't see your original email. | | Wanna add a comment here: :-) | | http://bugs.gentoo.org/show_bug.cgi?id=141904 By didn't see, he means he was so busy participating in his favourite game of Chris bashing that he didn't get around to reading any of the relevant material first... -- Ciaran McCreesh Mail: ciaranm at ciaranm.org Web : http://ciaranm.org/ as-needed is broken : http://ciaranm.org/show_post.pl?post_id=13 signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Versioning the tree
On Wed, 2006-11-29 at 20:10 +, Ciaran McCreesh wrote: By didn't see, he means he was so busy participating in his favourite game of Chris bashing that he didn't get around to reading any of the relevant material first... Could be, or (as happened here) mails arrived at different times. I saw responses before the parent message in a few sub-threads, today. Enough with the bashing (there's an irony to your own stuart bashing above), already. -- Seemant Kulleen Developer, Gentoo Linux -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On Mon, 2006-11-27 at 13:02 +, Stuart Herbert wrote: I think the original poster hit the nail on the head. The real barrier preventing a slower-moving tree is cultural. Somewhat. As I have said, I've mentioned several times the idea of doing a release tree to go along with each release. Support on these trees would be essentially equivalent to our security support model. We would support only packages marked stable. Of course, we also understand that since we're building off upstream's work, there *will* be times when a version bump is necessary to reduce our workload. Basically, our rule is kinda like this. No version changes on any packages, except those which are necessary due to a security violation, or a vulnerable package's dependencies. Now, I don't know if it is the best approach, but it is the best one that I could come up with on my own that allows for a slow-moving tree with higher QA done on it (as the release trees are what we use for releases, they undergo an enormous amount of testing, in the default configuration, of course) and minimal changes. Something that would be useful would be for package maintainers that wish to participate to perform further testing and validation on packages in the release snapshot prior to its final freeze. This would give us even more eyes on the tree and hopefully provide a much higher quality snapshot once we're done. Since it seems that there is still interest in the idea, I'll start working on it some more. -- Chris Gianelloni Release Engineering Strategic Lead Alpha/AMD64/x86 Architecture Teams Games Developer/Council Member/Foundation Trustee Gentoo Foundation signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Re: Versioning the tree
On Mon, 2006-11-27 at 16:18 +0100, Kevin F. Quinn wrote: One method could be to snapshot all package versions at the time that Release Engineering make a release, building a package.mask file out of it masking out all packages of higher revisions (i.e. having 'CPVR' entry for every package in the tree in stable, and 'CPVR' if no versions are stable). It would be infinitely easier to create a tree just for this. Then, rather than back-port security fixes, this list would be updated with security-fixed version numbers, along with minimum required updates to dependent packages. The lists could be stored in the tree; for example as profiles/default-linux/x86/stable.mask. This is essentially the idea that my release tree uses, except the tree itself is completely stripped down to stable packages. I have a script which already does several things: #1. grabs best_visible for stable on each arch #2. repeat for each SLOT #3. purge unnecessary files from FILESDIR #4. strip to only stable profiles from profiles.desc #5. purge unnecessary USE from use.local.desc and use.desc #6. strip unused eclasses #7. regenerate Manifest (via ebuild $foo digest) I had also planned on it doing the following, but they haven't been implemented just yet: - strip all USE flags that aren't used from use.mask (per-profile) - strip all packages that aren't available from package.mask (per-profile) What this gives us is a *much* smaller tree, as it only includes stable packages/versions. Obviously maintaining that list is some work; predominantly watching the announcements from the security team and fixing up dependencies, and masking out new packages (for what that's worth). It could be regenerated on some or all releases, or perhaps on a yearly basis. It would also mean that versions listed there cannot be removed from the tree (until they're bumped in the list). Well, the simpler approach was to simply copy over the newer, secure ebuilds, and any required dependencies, into the release tree. There'd be no need to update mask files and such this way. It would also be generated with the releases, automatically. Some of that maintenance could be tool-assisted; in particular masking new packages and finding the minimum required bumps to support a package that was bumped for a security fix. People who want to use it could then just soft-link from /etc/portage/package.mask to that list. It's just a suggestion - I'm not prepared to do the work ;) However it might be a simple but effective method to help people maintain secure but relatively stable systems, without having to upgrade umpteen packages a week. Well, I am willing, and have been doing, some of the work. The one disadvantage to my design is it needs infra. It needs it's own repository and rsync. Basically, it makes the system act a bit more like some other development models. We end up with the following: rsync://rsync.gentoo.org/gentoo-portage (this is equivalent to -current) rsync://rsync.gentoo.org/2007.0-release (this is the release tree) Now, the release trees are non-moving. The 2007.0-release tree is *always* the 2007.0-release tree for as long as we decide to support it. Likely, this support period would begin as a single release and get extended as volunteers came around to support it. New releases get their own tree. I also started writing a tool to upgrade the distribution. The tool reads pre and post scripts for the upgrade, and performs the necessary steps. Basically, a user can dist-upgrade 2007.1 and the system would: - switch to the 2007.1 tree and emerge --sync - perform all pre steps from 2007.1 - update world + revdep-rebuild + whatever else is necessary - perform all post steps With this, there would be a special version called current which would put the user on the gentoo-portage tree, AKA the tree we all know and love/loathe. Release trees wouldn't have arch and ~arch as everything would be stable there. Testing would be done in the main tree. This, in essence, gives us testing, release candidate and stable environments, with the release trees being stable, and the current tree becoming test/release candidate. Anything marked stable in the current tree would be a candidate for stable in the next release tree. Users who want to use the current portage tree get what they want. Users who want a more stable tree get what they want. Basically, everyone (hopefully) is happy, or at least as happy as we can feasibly make them. -- Chris Gianelloni Release Engineering Strategic Lead Alpha/AMD64/x86 Architecture Teams Games Developer/Council Member/Foundation Trustee Gentoo Foundation signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Re: Versioning the tree
On 11/28/06, Chris Gianelloni [EMAIL PROTECTED] wrote: On Mon, 2006-11-27 at 16:18 +0100, Kevin F. Quinn wrote: One method could be to snapshot all package versions at the time that Release Engineering make a release, building a package.mask file out of it masking out all packages of higher revisions (i.e. having 'CPVR' entry for every package in the tree in stable, and 'CPVR' if no versions are stable). It would be infinitely easier to create a tree just for this. Then, rather than back-port security fixes, this list would be updated with security-fixed version numbers, along with minimum required updates to dependent packages. The lists could be stored in the tree; for example as profiles/default-linux/x86/stable.mask. This is essentially the idea that my release tree uses, except the tree itself is completely stripped down to stable packages. I have a script which already does several things: #1. grabs best_visible for stable on each arch #2. repeat for each SLOT #3. purge unnecessary files from FILESDIR #4. strip to only stable profiles from profiles.desc #5. purge unnecessary USE from use.local.desc and use.desc #6. strip unused eclasses #7. regenerate Manifest (via ebuild $foo digest) I had also planned on it doing the following, but they haven't been implemented just yet: - strip all USE flags that aren't used from use.mask (per-profile) - strip all packages that aren't available from package.mask (per-profile) What this gives us is a *much* smaller tree, as it only includes stable packages/versions. Obviously maintaining that list is some work; predominantly watching the announcements from the security team and fixing up dependencies, and masking out new packages (for what that's worth). It could be regenerated on some or all releases, or perhaps on a yearly basis. It would also mean that versions listed there cannot be removed from the tree (until they're bumped in the list). Well, the simpler approach was to simply copy over the newer, secure ebuilds, and any required dependencies, into the release tree. There'd be no need to update mask files and such this way. It would also be generated with the releases, automatically. Some of that maintenance could be tool-assisted; in particular masking new packages and finding the minimum required bumps to support a package that was bumped for a security fix. People who want to use it could then just soft-link from /etc/portage/package.mask to that list. It's just a suggestion - I'm not prepared to do the work ;) However it might be a simple but effective method to help people maintain secure but relatively stable systems, without having to upgrade umpteen packages a week. Well, I am willing, and have been doing, some of the work. The one disadvantage to my design is it needs infra. It needs it's own repository and rsync. Basically, it makes the system act a bit more like some other development models. We end up with the following: rsync://rsync.gentoo.org/gentoo-portage (this is equivalent to -current) rsync://rsync.gentoo.org/2007.0-release (this is the release tree) Now, the release trees are non-moving. The 2007.0-release tree is *always* the 2007.0-release tree for as long as we decide to support it. Likely, this support period would begin as a single release and get extended as volunteers came around to support it. New releases get their own tree. I also started writing a tool to upgrade the distribution. The tool reads pre and post scripts for the upgrade, and performs the necessary steps. Basically, a user can dist-upgrade 2007.1 and the system would: - switch to the 2007.1 tree and emerge --sync - perform all pre steps from 2007.1 - update world + revdep-rebuild + whatever else is necessary - perform all post steps With this, there would be a special version called current which would put the user on the gentoo-portage tree, AKA the tree we all know and love/loathe. Release trees wouldn't have arch and ~arch as everything would be stable there. Testing would be done in the main tree. This, in essence, gives us testing, release candidate and stable environments, with the release trees being stable, and the current tree becoming test/release candidate. Anything marked stable in the current tree would be a candidate for stable in the next release tree. Users who want to use the current portage tree get what they want. Users who want a more stable tree get what they want. Basically, everyone (hopefully) is happy, or at least as happy as we can feasibly make them. This looks good on the surface, Chris, but what happens in the case where somebody wants to use the Release tree, but also wants (or needs) one or more packages from the Live tree, and doesn't want to switch completely over to the live tree? If I understand what you want to do correctly, the Release tree would include only stable packages. Other packages wouldn't just be masked, they would be completely unavailable to
Re: [gentoo-dev] Re: Versioning the tree
they could pull the more current ebuilds and put them in an overlay. also correct me if I'm wrong isn't it possible only to sync certain parts of the tree using excludes. maybe some additional functionality saying only sync package X for updates. On 11/28/06, James Potts [EMAIL PROTECTED] wrote: On 11/28/06, Chris Gianelloni [EMAIL PROTECTED] wrote: On Mon, 2006-11-27 at 16:18 +0100, Kevin F. Quinn wrote: One method could be to snapshot all package versions at the time that Release Engineering make a release, building a package.mask file out of it masking out all packages of higher revisions (i.e. having 'CPVR' entry for every package in the tree in stable, and 'CPVR' if no versions are stable). It would be infinitely easier to create a tree just for this. Then, rather than back-port security fixes, this list would be updated with security-fixed version numbers, along with minimum required updates to dependent packages. The lists could be stored in the tree; for example as profiles/default-linux/x86/stable.mask. This is essentially the idea that my release tree uses, except the tree itself is completely stripped down to stable packages. I have a script which already does several things: #1. grabs best_visible for stable on each arch #2. repeat for each SLOT #3. purge unnecessary files from FILESDIR #4. strip to only stable profiles from profiles.desc #5. purge unnecessary USE from use.local.desc and use.desc #6. strip unused eclasses #7. regenerate Manifest (via ebuild $foo digest) I had also planned on it doing the following, but they haven't been implemented just yet: - strip all USE flags that aren't used from use.mask (per-profile) - strip all packages that aren't available from package.mask (per-profile) What this gives us is a *much* smaller tree, as it only includes stable packages/versions. Obviously maintaining that list is some work; predominantly watching the announcements from the security team and fixing up dependencies, and masking out new packages (for what that's worth). It could be regenerated on some or all releases, or perhaps on a yearly basis. It would also mean that versions listed there cannot be removed from the tree (until they're bumped in the list). Well, the simpler approach was to simply copy over the newer, secure ebuilds, and any required dependencies, into the release tree. There'd be no need to update mask files and such this way. It would also be generated with the releases, automatically. Some of that maintenance could be tool-assisted; in particular masking new packages and finding the minimum required bumps to support a package that was bumped for a security fix. People who want to use it could then just soft-link from /etc/portage/package.mask to that list. It's just a suggestion - I'm not prepared to do the work ;) However it might be a simple but effective method to help people maintain secure but relatively stable systems, without having to upgrade umpteen packages a week. Well, I am willing, and have been doing, some of the work. The one disadvantage to my design is it needs infra. It needs it's own repository and rsync. Basically, it makes the system act a bit more like some other development models. We end up with the following: rsync://rsync.gentoo.org/gentoo-portage (this is equivalent to -current) rsync://rsync.gentoo.org/2007.0-release (this is the release tree) Now, the release trees are non-moving. The 2007.0-release tree is *always* the 2007.0-release tree for as long as we decide to support it. Likely, this support period would begin as a single release and get extended as volunteers came around to support it. New releases get their own tree. I also started writing a tool to upgrade the distribution. The tool reads pre and post scripts for the upgrade, and performs the necessary steps. Basically, a user can dist-upgrade 2007.1 and the system would: - switch to the 2007.1 tree and emerge --sync - perform all pre steps from 2007.1 - update world + revdep-rebuild + whatever else is necessary - perform all post steps With this, there would be a special version called current which would put the user on the gentoo-portage tree, AKA the tree we all know and love/loathe. Release trees wouldn't have arch and ~arch as everything would be stable there. Testing would be done in the main tree. This, in essence, gives us testing, release candidate and stable environments, with the release trees being stable, and the current tree becoming test/release candidate. Anything marked stable in the current tree would be a candidate for stable in the next release tree. Users who want to use the current portage tree get what they want. Users who want a more stable tree get what they want. Basically, everyone (hopefully) is happy, or at least as happy as we can feasibly make them. This looks good on the surface, Chris, but what
Re: [gentoo-dev] Re: Versioning the tree
On Tue, 2006-11-28 at 14:56 -0600, James Potts wrote: This looks good on the surface, Chris, but what happens in the case where somebody wants to use the Release tree, but also wants (or needs) one or more packages from the Live tree, and doesn't want to switch completely over to the live tree? If I understand what you want to do correctly, the Release tree would include only stable packages. Other packages wouldn't just be masked, they would be completely unavailable to anybody using that tree. Correct. The whole concept of the release tree is it is a complete package. All of the parts are supposed to work together. There's *nothing* that is not considered stable. If you need other packages, you add them to an overlay and support them yourself. I like the idea of having a stable p.mask much better, which says profile to me. Any thoughts on a special profile just for releases? It is completely unworkable with the 11,000+ packages we currently have in the tree, whereas an automated script that parses the current tree is much simpler. Quite simply, when GLEP19 was being worked on, it became painfully obvious almost immediately that even with the 10+ people who volunteered to work on it, that it would be impossible to maintain a profile-based system. We estimated that it would take us more than twice as many developers working on *only* the stable tree than we had working on the live tree to even keep up. To put it simply, I have *zero* interest in any profile/mask-based concepts for providing stable as they don't work. All they do is create an enormous bottleneck and a massive amount of workload for every single developer. With the release trees, essentially only those interested in supporting the tree are required to work on it. The tree is created entirely by scripts and is tested *before* it's released on the public. -- Chris Gianelloni Release Engineering Strategic Lead Alpha/AMD64/x86 Architecture Teams Games Developer/Council Member/Foundation Trustee Gentoo Foundation signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Re: Versioning the tree
James Potts wrote: This looks good on the surface, Chris, but what happens in the case where somebody wants to use the Release tree, but also wants (or needs) one or more packages from the Live tree, and doesn't want to switch completely over to the live tree? If I understand what you want to do correctly, the Release tree would include only stable packages. Other packages wouldn't just be masked, they would be completely unavailable to anybody using that tree. This is what overlays are for :) -- Andrew Gaffneyhttp://dev.gentoo.org/~agaffney/ Gentoo Linux Developer Installer Project Today's lesson in political correctness: Go asphyxiate on a phallus -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On 11/28/06, Chris Gianelloni [EMAIL PROTECTED] wrote: As I have said, I've mentioned several times the idea of doing a release tree to go along with each release. The release tree is not the basis for this. a) Releases (and the releng work that goes into it) are exclusively desktop-oriented. b) Release trees have a nasty habit of picking up last minute changes (such as gcc 4.1) to suit the release, not stability. No version changes on any packages, except those which are necessary due to a security violation, or a vulnerable package's dependencies. Tying a minimal-b0rkage tree to the arbitrary schedule of our releases does not serve all of our users. We are back to the same arguments we had when I said that the Seeds project would have to have its own independent release schedules :( Thereś little merit in us creating mostly stagnant trees. Other Linux distros are already very good at doing that - far better than we will be at it - because they have advantages such as a paid workforce and more upstream developers on their books. A minimal-b0rkage tree needs to move to reflect the packages that we believe our users should be using for a stable environment. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On Tue, 28 Nov 2006 14:56:47 -0600 James Potts [EMAIL PROTECTED] wrote: This looks good on the surface, Chris, but what happens in the case where somebody wants to use the Release tree, but also wants (or needs) one or more packages from the Live tree, and doesn't want to switch completely over to the live tree? If I understand what you want to do correctly, the Release tree would include only stable packages. Other packages wouldn't just be masked, they would be completely unavailable to anybody using that tree. I like the idea of having a stable p.mask much better, which says profile to me. Any thoughts on a special profile just for releases? Sounds like a perfect situation to use a package manager with true multiple repository support. -Steve signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Versioning the tree
Stuart Herbert wrote: On 11/28/06, Chris Gianelloni [EMAIL PROTECTED] wrote: As I have said, I've mentioned several times the idea of doing a release tree to go along with each release. The release tree is not the basis for this. a) Releases (and the releng work that goes into it) are exclusively desktop-oriented. You make it sound like releng doesn't care at all about non-desktop packages. The reason for the exclusivity is that the media that's typically built for release (GRP, LiveCD) is targetted for the largest audience...desktop users. If someone wants to volunteer to create a set of server-related GRP and a server LiveCD (as silly as this is for most things), they wouldn't be blocked outright. b) Release trees have a nasty habit of picking up last minute changes (such as gcc 4.1) to suit the release, not stability. Gcc 4.1.1 wasn't a last minute change. The release snapshots are originally created a month or more before the actual release. As stuff is stabilized in the tree, it's stabilized in the release snapshot as well. During the release cycle, we planned for gcc-4.1.1 to be stable by release (at least for x86, amd64, and a few other arches). We had it stable in the release snapshot a few weeks before the tree did and were testing the crap out of it. No version changes on any packages, except those which are necessary due to a security violation, or a vulnerable package's dependencies. Tying a minimal-b0rkage tree to the arbitrary schedule of our releases does not serve all of our users. We are back to the same arguments we had when I said that the Seeds project would have to have its own independent release schedules :( The release tree isn't really for minimal breakage. The *real* intent (at least from my POV) is to have a non-moving target for vendors to certify their software against (wouldn't it be nice for Oracle to be actually supported on Gentoo 2007.0 or something like that?), and so admins don't have to do the upgrade dance once a week or even every day (like I do). Thereś little merit in us creating mostly stagnant trees. Other Linux distros are already very good at doing that - far better than we will be at it - because they have advantages such as a paid workforce and more upstream developers on their books. The non-stagnant nature of Gentoo isn't the only reason that people use Gentoo. People use Gentoo for the configurability and customizability. As someone who admins more than a handful of Gentoo servers, I would absolutely *love* the combo of Gentoo's flexibility and a non-moving tree to make upgrades easier to deal with. -- Andrew Gaffneyhttp://dev.gentoo.org/~agaffney/ Gentoo Linux Developer Installer Project Today's lesson in political correctness: Go asphyxiate on a phallus -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
Steve Long schrieb: In any event, what I'd like to raise is the issue of having a (semi-)official version of gentoo that lags behind the cutting-edge distro for stability. Is this feasible? Apologies if this is already being discussed elsewhere. I appreciate that there is GLEP 19 according to earlier discussion on this list (from 2004). The GLEP has been withdrawn by the author. I guess I'm asking whether it's a) more feasible now (I'm guessing yes) and b) whether it's something that would have any support from current devs. Not in terms of workload, but culturally. You can't take workload out of the picture since it's the main issue here. Stable tree means backport fixes and I don't see this happening as it can't be automated: -there is no agreed machine parsable format/central location for CVEs/bugs. -you can't rely on upstream issuing security patches not tied to new releases (with new bugs/features). So who is going to watch all that mailing lists, pulling code from upstream, patching, testing, etc.? Don't get me wrong. I'd really love to have a more stable base system at least for core packages and those prone to break remote reboots ;) cheers Paul -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On 11/27/06, paul [EMAIL PROTECTED] wrote: You can't take workload out of the picture since it's the main issue here. Stable tree means backport fixes and I don't see this happening as it can't be automated: Stable tree means backport fixes is an assumption, not a requirement. It's one reason why the word stable is a poor choice for any such tree, just as it is a poor choice for the current Portage tree. I think the original poster hit the nail on the head. The real barrier preventing a slower-moving tree is cultural. Best regards, Stu -- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: Versioning the tree
On Mon, 27 Nov 2006 13:02:17 + Stuart Herbert [EMAIL PROTECTED] wrote: On 11/27/06, paul [EMAIL PROTECTED] wrote: You can't take workload out of the picture since it's the main issue here. Stable tree means backport fixes and I don't see this happening as it can't be automated: Stable tree means backport fixes is an assumption, not a requirement. It's one reason why the word stable is a poor choice for any such tree, just as it is a poor choice for the current Portage tree. I think the original poster hit the nail on the head. The real barrier preventing a slower-moving tree is cultural. One method could be to snapshot all package versions at the time that Release Engineering make a release, building a package.mask file out of it masking out all packages of higher revisions (i.e. having 'CPVR' entry for every package in the tree in stable, and 'CPVR' if no versions are stable). Then, rather than back-port security fixes, this list would be updated with security-fixed version numbers, along with minimum required updates to dependent packages. The lists could be stored in the tree; for example as profiles/default-linux/x86/stable.mask. Obviously maintaining that list is some work; predominantly watching the announcements from the security team and fixing up dependencies, and masking out new packages (for what that's worth). It could be regenerated on some or all releases, or perhaps on a yearly basis. It would also mean that versions listed there cannot be removed from the tree (until they're bumped in the list). Some of that maintenance could be tool-assisted; in particular masking new packages and finding the minimum required bumps to support a package that was bumped for a security fix. People who want to use it could then just soft-link from /etc/portage/package.mask to that list. It's just a suggestion - I'm not prepared to do the work ;) However it might be a simple but effective method to help people maintain secure but relatively stable systems, without having to upgrade umpteen packages a week. -- Kevin F. Quinn signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Versioning the tree
On Mon, 27 Nov 2006 11:33:58 + Steve Long [EMAIL PROTECTED] wrote: In any event, what I'd like to raise is the issue of having a (semi-)official version of gentoo that lags behind the cutting-edge distro for stability. Is this feasible? Apologies if this is already being discussed elsewhere. I appreciate that there is GLEP 19 according to earlier discussion on this list (from 2004). I guess I'm asking whether it's a) more feasible now (I'm guessing yes) and b) whether it's something that would have any support from current devs. Not in terms of workload, but culturally. Sorry for not being clear straight off. See http://forums.gentoo.org/viewtopic-p-3730878.html#3730878 for a related discussion. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
