Re: To semi-rolling or not to semi-rolling, that is the question...
On 09/03/10 19:06, Doug Ledford wrote: On 03/09/2010 11:45 AM, Kevin Kofler wrote: Jesse Keating wrote: Slight variation on this. All builds from devel/ (or master in the new git world) would go to the koji tag dist-rawhide-candidate. Builds which are tagged with dist-rawhide-candidate trigger AutoQA tests, of the nature rawhide acceptance (this would have to get fleshed out at some point, but it'd be something that builds upon the tests we would already do post-build). Builds that pass rawhide acceptance would get tagged into dist-rawhide, would show up in the build roots, and would be part of the nightly rawhide compose. Builds that do not remain in dist-rawhide-candidate. A maintainer can review the failed cases and make a decision, override the system or do a new build. Egregious use of system overriding would trigger FESCo attention and perhaps some sort of retraining or sanctioning. The assumption is that automated QA catches all possible breakage, which is not true. In fact *no* QA will catch all the Rawhide breakage as some is caused by the mere fact of things being different, which is intentional and part of what Rawhide is about (e.g. the libata change in the kernel, the change from KDE 3 to 4 etc.). Releases are needed to handle this kind of changes in a smooth way. But automated QA will also miss many actual bugs. Things like the libata kernel change and KDE 3 to 4 migration are intentional events and all that's needed to make the transition smooth is to coordinate the release of a seamless upgrade package set. You make it sound like these things are impossible when nothing could be further from the truth. And it's expected that a responsible maintainer is aware of these sorts of intentional update events and all that needs to be done is for them to conscientiously build their packages into the rawhide-candidate dist repo and coordinate a release of a complete set of packages. Automated QA need not catch this type of event. And automated QA *will* catch many more bugs than it misses (speaking from the mouth of experience knowing all the automated QA checks my rhel packages must pass prior to each update), and there was never an assumption that it would catch *all* issues. In fact the suggestion that automated QA would need to catch *all* issues before the proposal meets your approval makes it very apparent how disingenuous your stance on this proposal is. FACT: the semi-rolling update model you so love today is not foolproof and does not keep users from experiencing periodic, occasional breakage (see KDE update thread). FACT: you are strongly in support of the current semi-rolling model that you practice today (see multiple threads). FACT: you have purported that even with things occasionally breaking as they did on the recent KDE update, that these things are impossible to prevent 100% and that the system is working as designed (see KDE thread). So, for you to say this automated QA wouldn't catch *all* issues and therefore this proposal is unworkable, and then on the other hand say that major updates in RELEASED products that cause breakage are OK and working as designed puts your hypocrisy very much in the spotlight. A consumable rawhide need only meet the level that our released products do today (a level that you personally have helped to drag down BTW) and at that point you have *NO* valid grounds to object to it. And if we made rawhide meet that level of consumability, we should at the same time be *raising* the bar for our released products. Your argument appears to be that since we can't make rawhide meet what should possibly be the raised bar of the released products then the idea won't work so lets lower the bar across the board and give up. I'm sorry Kevin, but you and I will simply have to agree to disagree. I will *not* capitulate to your stance on this issue. god you don't half talk a lot of nonsense! if you want another ubuntu go work for canonical and be done with it, fedora is great the way it is, and if you asked your users instead of assuming you know what they want you will find this out! -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On 03/09/2010 07:46 PM, Kevin Kofler wrote: Doug Ledford wrote: Things like the libata kernel change and KDE 3 to 4 migration are intentional events That's the whole problem. Under our current model, we have places and times where to perform those intentional disruptive changes, they're called releases. In a consumable Rawhide, we don't (*), and that's an unavoidable limit to its consumability. Rawhide is a poor substitute for our current release model. (*) Sure, we can add flag days as you propose, but that's too often for users (at least 6 times as often, anything less frequent would make development basically impossible) You're assuming that each flag day will in fact be one where the user has to do something. That's not necessarily true. The hda-sda switch happened, what, 2 years or more ago? Yeah, it was a big deal. We've not really had an event like that since, and don't currently have one on the horizon. So, the FUD that 1 flag day per month means we'll actually have 1 user intervention required event per month is highly unlikely. No amount of testing, coordination etc. is going to make it acceptable to e.g. dump KDE 4 on KDE 3 users overnight. You handle a flag day like a mini-release. It's coordinate, users know about it ahead of time, there is no dumping things on people overnight. Jesse is proposing to have automated QA as the ONLY filter for packages to go to a supposedly consumable repository. So I replied that this cannot work because it cannot catch all issues. At the very least, the maintainer must be able to manually flag things as not being suitable for the consumable repository just yet. Sure. And to have something consumable enough to replace (at least for a class of users) releases, something like updates- testing is needed. Sure. All you have to do is build into rawhide-unstable then tell users to yum --enablerepo=rawhide-unstable upgrade foo to get that behavior if you want. You could split things out into two repos, but I'm not sure it's entirely worth it. But in general, yes, a testing repo would be wise to have. No, my argument is that Rawhide cannot even meet what is the CURRENT bar of our released products. For example, in KDE SIG, we DO NOT push changes we know to be disruptive, e.g.: * KDE 4 as an update to a release which shipped with KDE 3, * Amarok 2 as an update to a release which shipped with Amarok 1, * KDevelop 4 as an update to a release which shipped with KDevelop 3, and similarly the kernel maintainers DID NOT enable libata in update kernels for releases which shipped without libata, even when they pushed a new kernel where upstream enabled libata by default. We also do not push feature upgrades such as KDE 4.4 without long and tedious testing (as I explained further up). The current update system is NOT the free-for-all you seem to think it is. The updates are actually very carefully weighed for risks vs. benefits. It's NOT a consumable Rawhide, and any attempt to replace them with a Rawhide made consumable is bound to fail. In your opinion. I say it *could* succeed, and could be consumable. I say you lack the desire to see how things could be instead of how things are. You say I have rose colored glasses on. We get it. I'm sorry Kevin, but you and I will simply have to agree to disagree. I will *not* capitulate to your stance on this issue. But I think you're entirely missing my point! No, I very much get your point, I just think you are wrong. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
Doug Ledford wrote: You're assuming that each flag day will in fact be one where the user has to do something. That's not necessarily true. The hda-sda switch happened, what, 2 years or more ago? Yeah, it was a big deal. We've not really had an event like that since, and don't currently have one on the horizon. So, the FUD that 1 flag day per month means we'll actually have 1 user intervention required event per month is highly unlikely. There are many more changes which require user intervention, even if it's not as obvious (as in don't intervene and your system won't boot). For example, after upgrading from F8 to F9, i.e. moving from KDE 3 to KDE 4, I had to tweak my desktop configuration in a few places. And even less blatant changes sometimes require some sort of user intervention (such as recreating some configuration because it's represented differently in the new version and upstream didn't provide a way to migrate it automatically). And required user intervention is not the only source of trouble, things like feature regressions, data (e.g. savegame) compatibility etc. are, too. You handle a flag day like a mini-release. It's coordinate, users know about it ahead of time, there is no dumping things on people overnight. Except it's not a mini-release. You can't just stay on the old branch (and still get security, bugfix and other nondisruptive updates) if you aren't ready for the change as you can with our releases. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On Tue, Mar 09, 2010 at 14:06:12 -0500, Doug Ledford dledf...@redhat.com wrote: Things like the libata kernel change and KDE 3 to 4 migration are intentional events and all that's needed to make the transition smooth is to coordinate the release of a seamless upgrade package set. You I lived through the libata change and a coordinated release of updates is not all that was needed in that case. For that change guinea pigs were needed to test specific hardware. In my case my ATA card was having problems because there were similarly identified cards that needed different drivers and it took a while before the two sets were correctly identified. For a significant part of that rawhide development the set of cards that worked switched back and forth. See https://bugzilla.redhat.com/show_bug.cgi?id=227281 for the saga. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
Doug Ledford wrote: Things like the libata kernel change and KDE 3 to 4 migration are intentional events That's the whole problem. Under our current model, we have places and times where to perform those intentional disruptive changes, they're called releases. In a consumable Rawhide, we don't (*), and that's an unavoidable limit to its consumability. Rawhide is a poor substitute for our current release model. (*) Sure, we can add flag days as you propose, but that's too often for users (at least 6 times as often, anything less frequent would make development basically impossible) and there's also no way for a user to say I'm not ready for a flag day now, I'll just skip one or move at some point between this flag day and the next one and still get updates, unlike now where they can skip a release and still get updates all along. and all that's needed to make the transition smooth is to coordinate the release of a seamless upgrade package set. No. The problem with the above changes is not the consistency of the update set, it's that they do major changes to the user's machine, such as feature regressions, new bugs, requiring manual adjustment of settings (e.g. s/hda/sda/ in some config files) etc. Let's call the old state (e.g. KDE 3) S and the new state (e.g. KDE 4) S'. The big issue here is not the consistency, quality or whatever attributes of the new state S', but the state transition from S to S'. Even if we fix all the inherent problems of S', that still doesn't make it a valid state for S to transition into. You make it sound like these things are impossible when nothing could be further from the truth. I only make those things sound impossible which actually ARE impossible. See above. No amount of testing, coordination etc. is going to make it acceptable to e.g. dump KDE 4 on KDE 3 users overnight. If my only 2 alternatives are to get that kind of updates (consumable Rawhide) or to get no new features (and quite possibly even only limited bugfixes) at all (conservative updates), there is no alternative suitable for me. Nor for the dozens of users who voted adventurous in Adam Williamson's poll. (No matter whether you consider that sample representative or not, you can't argue away the over a hundred users who voted that way in the sample itself! Claiming they were all misled by the question is also not really credible.) And automated QA *will* catch many more bugs than it misses (speaking from the mouth of experience knowing all the automated QA checks my rhel packages must pass prior to each update), and there was never an assumption that it would catch *all* issues. In fact the suggestion that automated QA would need to catch *all* issues before the proposal meets your approval makes it very apparent how disingenuous your stance on this proposal is. Jesse is proposing to have automated QA as the ONLY filter for packages to go to a supposedly consumable repository. So I replied that this cannot work because it cannot catch all issues. At the very least, the maintainer must be able to manually flag things as not being suitable for the consumable repository just yet. And to have something consumable enough to replace (at least for a class of users) releases, something like updates- testing is needed. (No, I don't think ALL updates need to go through updates-testing. There are several cases where I think pushing directly to stable is the correct solution. But that doesn't mean that updates-testing is useless nor that users who want non-conservative updates want untested updates!) FACT: the semi-rolling update model you so love today is not foolproof and does not keep users from experiencing periodic, occasional breakage (see KDE update thread). FACT: you are strongly in support of the current semi-rolling model that you practice today (see multiple threads). FACT: you have purported that even with things occasionally breaking as they did on the recent KDE update, that these things are impossible to prevent 100% and that the system is working as designed (see KDE thread). I still think we did the right thing pushing KDE 4.4.0. It fixed a lot more issues than it caused. (Upstream counts thousands of fixed bugs.) And for many people it just works. There's that slight annoyance in the form of a message box when starting up kdepim apps which can be easily worked around (either just click it away, or enable Nepomuk and have it gone for good), and which should be solved for good with the kde-settings update we're pushing (Nepomuk will be enabled by default), but other than that, I haven't seen any widespread issues. It's NOT an update like KDE 3 to 4 which would be insane to push to a stable release. I'll also point out that 4.4.0 has been through a lot of testing, we've had prereleases in Rawhide and kde-redhat unstable, then 4.4.0 itself was also tested for more than 2 weeks before we declared it stable. During all this time,
Re: To semi-rolling or not to semi-rolling, that is the question...
Jesse Keating wrote: Slight variation on this. All builds from devel/ (or master in the new git world) would go to the koji tag dist-rawhide-candidate. Builds which are tagged with dist-rawhide-candidate trigger AutoQA tests, of the nature rawhide acceptance (this would have to get fleshed out at some point, but it'd be something that builds upon the tests we would already do post-build). Builds that pass rawhide acceptance would get tagged into dist-rawhide, would show up in the build roots, and would be part of the nightly rawhide compose. Builds that do not remain in dist-rawhide-candidate. A maintainer can review the failed cases and make a decision, override the system or do a new build. Egregious use of system overriding would trigger FESCo attention and perhaps some sort of retraining or sanctioning. The assumption is that automated QA catches all possible breakage, which is not true. In fact *no* QA will catch all the Rawhide breakage as some is caused by the mere fact of things being different, which is intentional and part of what Rawhide is about (e.g. the libata change in the kernel, the change from KDE 3 to 4 etc.). Releases are needed to handle this kind of changes in a smooth way. But automated QA will also miss many actual bugs. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On Tue, 2010-03-09 at 17:45 +0100, Kevin Kofler wrote: The assumption is that automated QA catches all possible breakage, which is not true. In fact *no* QA will catch all the Rawhide breakage as some is caused by the mere fact of things being different, which is intentional and part of what Rawhide is about (e.g. the libata change in the kernel, the change from KDE 3 to 4 etc.). Releases are needed to handle this kind of changes in a smooth way. But automated QA will also miss many actual bugs. Yes, you bear some risk in using rawhide. There is no reward without risk. We can mitigate some of that risk by placing automated testing between the builds and the users. Some reduction in risk is far better than no reduction is it not? Would it not be nice to see rawhide reports without the huge list of broken deps? Would it not be nice to have a rawhide build update that doesn't segfault upon execution? These are the kinds of things that happen now, that AutoQA could prevent. That makes rawhide vastly more consumable than it currently is. Perfect is the enemy of good. -- Jesse Keating Fedora -- Freedom² is a feature! identi.ca: http://identi.ca/jkeating signature.asc Description: This is a digitally signed message part -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On 03/09/2010 11:45 AM, Kevin Kofler wrote: Jesse Keating wrote: Slight variation on this. All builds from devel/ (or master in the new git world) would go to the koji tag dist-rawhide-candidate. Builds which are tagged with dist-rawhide-candidate trigger AutoQA tests, of the nature rawhide acceptance (this would have to get fleshed out at some point, but it'd be something that builds upon the tests we would already do post-build). Builds that pass rawhide acceptance would get tagged into dist-rawhide, would show up in the build roots, and would be part of the nightly rawhide compose. Builds that do not remain in dist-rawhide-candidate. A maintainer can review the failed cases and make a decision, override the system or do a new build. Egregious use of system overriding would trigger FESCo attention and perhaps some sort of retraining or sanctioning. The assumption is that automated QA catches all possible breakage, which is not true. In fact *no* QA will catch all the Rawhide breakage as some is caused by the mere fact of things being different, which is intentional and part of what Rawhide is about (e.g. the libata change in the kernel, the change from KDE 3 to 4 etc.). Releases are needed to handle this kind of changes in a smooth way. But automated QA will also miss many actual bugs. Things like the libata kernel change and KDE 3 to 4 migration are intentional events and all that's needed to make the transition smooth is to coordinate the release of a seamless upgrade package set. You make it sound like these things are impossible when nothing could be further from the truth. And it's expected that a responsible maintainer is aware of these sorts of intentional update events and all that needs to be done is for them to conscientiously build their packages into the rawhide-candidate dist repo and coordinate a release of a complete set of packages. Automated QA need not catch this type of event. And automated QA *will* catch many more bugs than it misses (speaking from the mouth of experience knowing all the automated QA checks my rhel packages must pass prior to each update), and there was never an assumption that it would catch *all* issues. In fact the suggestion that automated QA would need to catch *all* issues before the proposal meets your approval makes it very apparent how disingenuous your stance on this proposal is. FACT: the semi-rolling update model you so love today is not foolproof and does not keep users from experiencing periodic, occasional breakage (see KDE update thread). FACT: you are strongly in support of the current semi-rolling model that you practice today (see multiple threads). FACT: you have purported that even with things occasionally breaking as they did on the recent KDE update, that these things are impossible to prevent 100% and that the system is working as designed (see KDE thread). So, for you to say this automated QA wouldn't catch *all* issues and therefore this proposal is unworkable, and then on the other hand say that major updates in RELEASED products that cause breakage are OK and working as designed puts your hypocrisy very much in the spotlight. A consumable rawhide need only meet the level that our released products do today (a level that you personally have helped to drag down BTW) and at that point you have *NO* valid grounds to object to it. And if we made rawhide meet that level of consumability, we should at the same time be *raising* the bar for our released products. Your argument appears to be that since we can't make rawhide meet what should possibly be the raised bar of the released products then the idea won't work so lets lower the bar across the board and give up. I'm sorry Kevin, but you and I will simply have to agree to disagree. I will *not* capitulate to your stance on this issue. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On Thu, 2010-03-04 at 15:59 -0500, Doug Ledford wrote: 1) Make rawhide consumable. A) Create rawhide-unstable. Any time a known disruptive change is being worked on, it should be built here by the developer. In addition, add rpmdiff checks to all builds from devel into dist-rawhide and if any of certain rpmdiff checks trip positive, move the package from rawhide to rawhide-unstable. Additional checks can be added as AutoQA gets into full swing, and so we can add more ways to catch breakage and move things to rawhide-unstable as needed. Slight variation on this. All builds from devel/ (or master in the new git world) would go to the koji tag dist-rawhide-candidate. Builds which are tagged with dist-rawhide-candidate trigger AutoQA tests, of the nature rawhide acceptance (this would have to get fleshed out at some point, but it'd be something that builds upon the tests we would already do post-build). Builds that pass rawhide acceptance would get tagged into dist-rawhide, would show up in the build roots, and would be part of the nightly rawhide compose. Builds that do not remain in dist-rawhide-candidate. A maintainer can review the failed cases and make a decision, override the system or do a new build. Egregious use of system overriding would trigger FESCo attention and perhaps some sort of retraining or sanctioning. B) Non disruptive changes go into rawhide directly, and on a regular basis we run a compose on the rawhide tree to create install disks/images for use. I suggest once a week we recreate the images. I'd leave the install images creation out of this for now, that's primarily a decision for the anaconda developers to make, as to when they wish to have mirrored install images created and used for bug reporting. C) On a regular basis, we have a flag day to move items from rawhide-unstable to rawhide. I originally said as-needed in my first proposal, but on more reflection I would like to suggest we make this a regular scheduled event on a monthly basis. In this way we have 6 regular rawhide-unstable==rawhide checkpoints between each release cycle, and we can make the last one or two prior to the next fedora release do double duty as both the normal checkpoint and the fedora alpha/beta freeze. This also has the side benefit that people working on major changes, like say anaconda install updates, have more points at which they can get their code into a consumable segment of the repo and start getting feedback. That's an interesting thought, coordinated moves from either untagged or dist-rawhide-candidate over to dist-rawhide. D) Anyone who likes that rawhide allows them to develop directly on their OS can simply enable the rawhide-unstable repo in their yum configs. Like enabling updates on a regular release, it would inherit from rawhide and also include all the distruptive changes. This makes a system with rawhide-unstable enabled identical to rawhide today. I think we could accomplish this with a simple koji static repo. It's far cheaper than trying to do a full multilib mashed repo every night of this content, and it gets updated far more frequently. E) Without rawhide-unstable, you get a semi-rolling release that allows for regular checkpoints to introduce disruptive changes, soname bumps, etc. only on a more frequent basis than the current fedora release cycle. It is hoped that by having this higher frequency, disruptive changes in the semi-rolling release that is rawhide can be handled more easily. Kind of along the lines of easier to deal with by taking more, smaller bites instead of huge, hard to swallow bites. I think there is room to do soname changes more frequently than the scheduled disruption days, provided that all in Fedora consumers of a soname are bumped at the same, or at least moved into -rawhide at the same time. This would require a higher level of coordination, and more specific koji buildroots. A tax on the system, to be sure. 2) Make Fedora releases adhere to a stable release policy. This already covered rather well in proposals put forth by other people. Mike McGrath's snapshot proposal and Matt Domsch's Slowing rate of change in older releases proposal are the two I would pair with my proposal in order to satisfy both groups. I don't see any reason to rehash them here. -- Jesse Keating Fedora -- Freedom² is a feature! identi.ca: http://identi.ca/jkeating signature.asc Description: This is a digitally signed message part -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
Till Maas wrote: F13 updates will be supported until F15 Alpha is created, so everyone has a about a three month update window to get from FN-updates to F(N+1)-updates or F(N+1)-updates-stable. FN-updates to F(N+1)-updates-stable is unlikely to work, because FN-updates will be including stuff which is only in F(N+1)-updates, not F(N+1)-updates- stable. (And FWIW, I'd call them conservative rather than stable, I don't like the implication that the other stream would be unstable, which to most users means crashy.) Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On Fri, Mar 05, 2010 at 08:52:56AM +0100, Hans de Goede wrote: One size does still not fit all, although this is a great idea for most packages in Fedora for packages in certain niches this is a bad idea. I've said this before (and got 0 response), I believe there should be some divide made between core packages (with core being quite big, not the bare essentials, but also most of all desktop environments, etc.) and non core packages. I kind of agree with you, but without completely describing these core packages, it is kind of hard to know what to agree to. Imho it is not only core-ness, but also package complexity that should be taken into account (and upstream QA, but you mentioned this in your other mail). But as you wrote their, it mostly boils down to a maintainer decision and cannot easily be written into a Policy, but more into a guideline for maintainers to help them decide. Regards Till pgpQqdD79EfGc.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On Fri, Mar 05, 2010 at 08:52:56AM +0100, Hans de Goede wrote: Make rawhide consumable as a semi-rolling release itself. We already have this it is called early branching of the next release. I would fully agree with you if it were not for the early branching feature, which means we effectively already have such a release, except the first 2 months or so after a release, at which time rawhide tends to be very volatile in general (*), so a stabilized rawhide would pretty much boil down to the latest release anyways. This is something I've been toying with in my mind myself. I like the idea of using it better than using rawhide for the same reasons that Till Mass mentions in his email. I haven't been able to completely satisfy myself on two points though: 1) This new tree is supposedly leading towards a new release. As such, there are other pressures on it than simply being a consumable tree. (For instance, updates will be frozen for periods in this tree as alpha and beta are spun; the freezes lead to a huge number of updates on release day). How disruptive these pieces are to using this tree as a consumable product in its own right is the worry. However, we haven't yet run a full release of no-frozen-rawhide so we don't know precisely how annoying the problems would be or how easily solvable. 2) The new tree is not consumable all the time because there are times when it does not exist. As you also mention, there is the period right after release when the unrelease tree does not exist. There's only F-current (which would be taking less updates under a policy that anticipates people who want semi-rolling to be consuming the F-current+1 tree) and rawhide (which, as you say, is very volatile at this time.) In fact, this is probably the time period in which a semi-rolling tree would be most useful to people (as at other times of the cycle, maintainer laziness would make rawhide closer to the current+1 tree and therefore more consumable) Do you have any ideas on what we could do here? -Toshio pgpcz2JdtmKgs.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On 03/05/2010 04:49 AM, Kevin Kofler wrote: Doug Ledford wrote: So, I'm going to reiterate my policy suggestion: Make Fedora releases (all of them) stable in nature, not semi-rolling. Make rawhide consumable as a semi-rolling release itself. And let me reiterate my objections, because you asked for it. :-) Reasons: 1) Most importantly, it's a means of satisfying both groups of people in the Fedora developer community and leaving none behind. 2) It saves developers time and effort. This is actually the least burdensome method of satisfying both groups of people inside Fedora. The whole 2 update stream approach increases the load on maintainers, while this approach reduces the load on maintainers as a maintainer gets to consider a Fedora release done when it goes out the door with the exception of fixing any security or important bugs in that release. Yet it is the only solution which really satisfies both groups of people. You should always be more clear when writing emails such as this. The Yet it is above is unclear. Are you referring to a stable rawhide, or the two update stream model for satisfying both parties? Stabilizing Rawhide is pretty much a lost cause. No, it's not. If we care about rawhide being stable, then it's up to us to make it so. Since we don't currently care, developers do sloppy, lazy things in rawhide and don't worry about it. What your proposal further down does is actually introducing an intermediate stream à la Debian testing, which comes with a lot of extra maintainer work as well. It comes with less extra work than doing two update streams. Face it, there is *no* solution to this problem that both solves the issue for both parties involved and does not include at least *some* extra work for you. 3) It conserves resources (drastically so) on the fedora infrastructure and mirrors and reduces bandwidth needs greatly. How so? You're still adding an extra repo. The updates repos will be *drastically* smaller, enough so that the new repo won't even come close to using up the space savings. We have established, beyond any doubt whatsoever, that there are users of Fedora that expect, demand, and need a stable update stream. Ignoring those user's needs by saying why can't we just stay as we are is tantamount to blatantly ignoring their cries for change/help, trivializing their issues, and pushing them aside as unimportant. Uh no. It's just saying that there are dozens of distributions out there already doing exactly that, they should just pick one. Fedora has never worked that way, why should we do so now? They are fedora users today, ignoring them is exceedingly rude. And given the fact that more people have stood up requesting a more stable update stream than those that have stood up for the semi-rolling update stream, it is both very cavalier and risky on your part to throw around the we like it the way it is, go away card. In the end, the one going away may not be the one you want. It would be to everyone's benefit if we didn't have either group go away. And I also dislike the usage of the term stable for this purpose, as it implies our current updates are unstable, which to most users means crashy. Did you miss the KDE update thread yesterday? I stand by the name stable, and I raise you an it *was* crashy. We have established, beyond any doubt whatsoever, that there are users of Fedora that expect, demand, and use fedora *because* they get major updates in the middle of a release. Ignoring these users needs by saying rawhide is == way without first doing what is necessary to make rawhide usable and consumable is tantamount to blatantly ignoring the untenable nature of your suggestion, trivializing their wants and issues, and pushing them aside as unimportant. Sure, but making Rawhide consumable is a lost cause (and is in fact not what your proposal is actually trying to do). My proposal *is* trying to do that. Whether or not it is a lost cause is a function of *us*. We make rawhide consumable, or we make it a minefield. Either way, it's our actions at work. All we have to do is care. Technical implementation proposal: 1) Make rawhide consumable. A) Create rawhide-unstable. Any time a known disruptive change is being worked on, it should be built here by the developer. In addition, add rpmdiff checks to all builds from devel into dist-rawhide and if any of certain rpmdiff checks trip positive, move the package from rawhide to rawhide-unstable. Additional checks can be added as AutoQA gets into full swing, and so we can add more ways to catch breakage and move things to rawhide-unstable as needed. At this point you've created 2 rawhides, which already means 2 repositories etc. So? Rawhide-unstable is just a temporary holding area. Unlike the GA/updates repo combinations for releases, rawhide is not frozen and doesn't contain a static
Re: To semi-rolling or not to semi-rolling, that is the question...
On 03/05/2010 02:52 AM, Hans de Goede wrote: One size does still not fit all, although this is a great idea for most packages in Fedora for packages in certain niches this is a bad idea. I've said this before (and got 0 response), I believe there should be some divide made between core packages (with core being quite big, not the bare essentials, but also most of all desktop environments, etc.) and non core packages. Well, the reason I didn't respond before is because I thought I had no disagreement with what you wrote. It seems obvious to me that even if we made a policy that Fedora was primarily stable once released, that there would always be exceptions to that rule and things that should be updated more aggressively. So I would not advocate for any policy that was absolute and inflexible. There should be room for human judgment to play a role. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
Doug Ledford wrote: On 03/05/2010 04:49 AM, Kevin Kofler wrote: Yet it is the only solution which really satisfies both groups of people. You should always be more clear when writing emails such as this. The Yet it is above is unclear. Are you referring to a stable rawhide, or the two update stream model for satisfying both parties? The 2 update stream model. Stabilizing Rawhide is pretty much a lost cause. No, it's not. If we care about rawhide being stable, then it's up to us to make it so. Since we don't currently care, developers do sloppy, lazy things in rawhide and don't worry about it. I think there's a lot more to it than just that, as I explained further down. What your proposal further down does is actually introducing an intermediate stream à la Debian testing, which comes with a lot of extra maintainer work as well. It comes with less extra work than doing two update streams. Face it, there is *no* solution to this problem that both solves the issue for both parties involved and does not include at least *some* extra work for you. Sure, but will yours be *less* extra work? And do we also really need to target both groups in the first place (as opposed to focusing on the fast- moving solution which distinguishes us from other distros), seeing how many existing distributions already fill the need for the conservative variant? [re users who expect a conservative update policy] They are fedora users today, ignoring them is exceedingly rude. It was their choice to use Fedora. They may have made the wrong choice, but that doesn't mean they get to turn Fedora into something different. As an analogy, if I walk into a gay bar, I don't get to convert it into a bar for heterosexuals just because I happen not to be attracted by men. And given the fact that more people have stood up requesting a more stable update stream than those that have stood up for the semi-rolling update stream, Really? That's not the impression I got. And I also dislike the usage of the term stable for this purpose, as it implies our current updates are unstable, which to most users means crashy. Did you miss the KDE update thread yesterday? I stand by the name stable, and I raise you an it *was* crashy. I actually believe the KDE updates are an example of things done right. They aren't perfect, but no software is. My proposal *is* trying to do that. Whether or not it is a lost cause is a function of *us*. We make rawhide consumable, or we make it a minefield. Either way, it's our actions at work. All we have to do is care. No matter what we do, we can't achieve the impossible. So? Rawhide-unstable is just a temporary holding area. Unlike the GA/updates repo combinations for releases, rawhide is not frozen and doesn't contain a static set of packages. That means that rawhide-unstable is not like an updates repo where it holds the updates forever. Instead, packages move from rawhide-unstable to rawhide on a regular basis, and when that happens, the rawhide-unstable repo shrinks (preferably to 0, but if some of the packages aren't ready for the move yet then they stay in rawhide-unstable). I understood that just fine the first time, there was no need to repeat it. You don't test locally before you build into the build system? That would be one way. The other way is to actually know and understand the code you are submitting. Many disruptive changes you simply *know* are disruptive because you know and understand the code. This would be determined on a maintainer by maintainer basis. Then you would also have the AutoQA/rpmdiff checks on the packages post-build that would attempt to catch disruptive changes that the maintainer missed. Of course all this is possible and will be done, but that doesn't obviate the need for a testing repository. If it did, then why would we have updates-testing in the first place? But, really, this boils down to what I said earlier. If we care about rawhide, then we make an effort not to break it, it will be better. If we don't, then it will stay the same. There's no great technical hurdle here, only a social one. The fact that packages cannot be tested adequately without a testing repository is just common sense, and it is a technical issue, not a social one. I for one don't see untested or poorly-tested stuff as a viable alternative to our current updates. I wouldn't want to use stuff which had no testing (except for trivial fixes or urgent stuff such as security fixes, but we've had that part of the discussion already; and stuff which is not trivial and/or urgent definitely DOES need testing, and there needs to be infrastructure for that). And an unstable repository also containing disruptive changes which are known to be disruptive is a poor substitute for a testing repository. It's hard to test something when everything else is broken. There's a reason we have updates-testing and
Re: To semi-rolling or not to semi-rolling, that is the question...
On Fri, 05 Mar 2010 12:56:11 -0500, Doug wrote: It seems obvious to me that even if we made a policy that Fedora was primarily stable once released, that there would always be exceptions to that rule and things that should be updated more aggressively. So I would not advocate for any policy that was absolute and inflexible. There should be room for human judgment to play a role. Hear, hear! I'd like to sign that. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
To semi-rolling or not to semi-rolling, that is the question...
Obviously, some people want this and some don't. It isn't appropriate to simply hand down an edict that things will be one way or the other if we truly consider Fedora a community run project. It must be a community decision. That means, as Adam Williamson pointed out, this is likely a decision to be made by the board, based upon what the community wants and what's best for Fedora. But let's be clear. That's a *policy* decision. One of the things that got very confusing in the previous thread(s) was the intermixing of policy decisions and technical issues. For instance, Kevin's response to my proposal was all about technical issues he saw. Technical issues are almost always solvable if you have a specific policy you are trying to implement. So one thing I think people should keep in mind is that policy decisions should always lead to technical decisions, *not* the other way around. We should decide what we want ourselves to be and what our policies are, and then that should guide our infrastructure, our tools, our work flows, and our processes. We should never allow things to flow in the reverse direction. We should never allow a current tooling limitation to set our policy, modulo that our policy should acknowledge and accommodate for the time necessary for tooling changes to take place or for the limitations of our resources. So, I'm going to reiterate my policy suggestion: Make Fedora releases (all of them) stable in nature, not semi-rolling. Make rawhide consumable as a semi-rolling release itself. Reasons: 1) Most importantly, it's a means of satisfying both groups of people in the Fedora developer community and leaving none behind. 2) It saves developers time and effort. This is actually the least burdensome method of satisfying both groups of people inside Fedora. The whole 2 update stream approach increases the load on maintainers, while this approach reduces the load on maintainers as a maintainer gets to consider a Fedora release done when it goes out the door with the exception of fixing any security or important bugs in that release. For many packages, this means the maintainer will actually not have anything to do in those releases and they can concentrate solely on devel (this is obviously not true for big and highly used packages like thunderbird, firefox, gnome, kde, they will always have bugs and updates needed, but many other packages, especially smaller leaf packages, may never need a single update during a stable release lifetime). 3) It conserves resources (drastically so) on the fedora infrastructure and mirrors and reduces bandwidth needs greatly. 4) It helps Fedora scale (from a repo resource consumption standpoint at least, but also in terms of developer time for the packages that won't need lots of stable updates). 5) It would be less labor intensive to implement and work flows would be simpler than the two update stream, ala Mandriva, approach to satisfying everyone. Before I get into the technical implementation part of the proposal, I want to take a moment to step back and mention something to both sides of the camp that have been involved in this discussion so far: We have established, beyond any doubt whatsoever, that there are users of Fedora that expect, demand, and need a stable update stream. Ignoring those user's needs by saying why can't we just stay as we are is tantamount to blatantly ignoring their cries for change/help, trivializing their issues, and pushing them aside as unimportant. We have established, beyond any doubt whatsoever, that there are users of Fedora that expect, demand, and use fedora *because* they get major updates in the middle of a release. Ignoring these users needs by saying rawhide is == way without first doing what is necessary to make rawhide usable and consumable is tantamount to blatantly ignoring the untenable nature of your suggestion, trivializing their wants and issues, and pushing them aside as unimportant. So maybe we can choose to put the rhetoric that has flung around in this discussion down, stop trivializing our fellow Fedora community member's issues, and work together. After all, that's what being part of a community is all about. If we can agree on a policy that satisfies both groups of people, then that can inform and guide our technical implementation discussions, keeping in mind that a failure of the first draft of the technical implementation is not grounds to throw out the policy, merely grounds for more work on the technical implementation ;-) Technical implementation proposal: 1) Make rawhide consumable. A) Create rawhide-unstable. Any time a known disruptive change is being worked on, it should be built here by the developer. In addition, add rpmdiff checks to all builds from devel into dist-rawhide and if any of certain rpmdiff checks trip positive, move the package from rawhide to rawhide-unstable. Additional checks can be added as AutoQA gets into full swing,
Re: To semi-rolling or not to semi-rolling, that is the question...
On Thu, Mar 04, 2010 at 03:59:16PM -0500, Doug Ledford wrote: But let's be clear. That's a *policy* decision. One of the things that got very confusing in the previous thread(s) was the intermixing of policy decisions and technical issues. For instance, Kevin's response So, I'm going to reiterate my policy suggestion: Make Fedora releases (all of them) stable in nature, not semi-rolling. Make rawhide consumable as a semi-rolling release itself. Imho this semi-rolling is too blurry, because only the technical implementation give a grasp of what is rolling. I thought I was pro semi-rolling, but the technical proposal did not seem to fit my expectation. Here is what I would like to have as semi-rolling. What I would like to have semi rolling is as many updates that fix known bugs, even if only upstream knows them, as long as they fit these criteria: http://lists.fedoraproject.org/pipermail/devel/2010-February/131570.html And they must pass all AutoQA tests, which is not a big issue currently, but will be if AutoQA becomes what I would like it to be. All other updates should only be mandatory like every six to twelve months. Also I would like to have packages that are required to fix broken updates be taken a little bit more care of. Afaik this is what is happening with the critical path nowadays. These are my desires as mostly a user. As a packager I would like to also have some staging instance (like updates-testing), where I can stage updates that are supposed to match the criteria, but can be easily used to verify this. So I can tell users to verify that a bug is fixed, if there is no easy way to reproduce it myself, e.g. if it requires a complex setup. Since there also needs to be a repo where things can be arbitrarily broken to develop fixes, to satisfy this semi rolling approach and the stable one is to create two more repos per stable release, so we would have these: fedora - initial release updates-testing- testing updates targeted at updates updates- semi rolling wrt. to above policy outline updates-stable-staging - used to test updates for updates-stable updates-stable - only gets updates wrt. to a stable policy To lower the demand of resources, the updates/updates-testing could not be provided for the full lifetime of the release, but e.g. only for around nine months. A Milestone could be the Alpha release of the Rawhide branch of N+2, e.g. F13 updates will be supported until F15 Alpha is created, so everyone has a about a three month update window to get from FN-updates to F(N+1)-updates or F(N+1)-updates-stable. Regards Till pgp9oJd2zq5Mg.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
Doug Ledford wrote: [the whole nine yards] I like this idea. As a user of fedora updates- testing and kde-redhat, in order to get the latest software the soonest onto my computer, without having the burden of reinstalling my system twice a year on 2 computers, x86_64 desktop and i686/PAE laptop, I like the idea of being able to run rawhide perpetually (and not having it get into such a conundrum that one day it will no longer even boot, due to some bad yum update). I had flippantly proposed 'fedora infinity' a year or two back on this list and it was shot down :-( I was told that I should look into sidux or arch linux, I think. Sorry, I'm not into building my system from scratch and appreciate the fantastic work here and, were rawhide to always work while always being on the edge of the latest development, and no longer having to reinstall twice a year, i.e., having rawhide become a de facto rolling release, I would be thrilled with it!!! -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
On 03/04/2010 06:27 PM, Kevin Kofler wrote: Doug Ledford wrote: But let's be clear. That's a *policy* decision. One of the things that got very confusing in the previous thread(s) was the intermixing of policy decisions and technical issues. For instance, Kevin's response to my proposal was all about technical issues he saw. Technical issues are almost always solvable if you have a specific policy you are trying to implement. So one thing I think people should keep in mind is that policy decisions should always lead to technical decisions, *not* the other way around. We should decide what we want ourselves to be and what our policies are, and then that should guide our infrastructure, our tools, our work flows, and our processes. We should never allow things to flow in the reverse direction. We should never allow a current tooling limitation to set our policy, modulo that our policy should acknowledge and accommodate for the time necessary for tooling changes to take place or for the limitations of our resources. I heavily disagree with that assertion. We need to always keep the technical and practical limitations in mind when deciding on a policy. Limitations, yes. Current state, no. You can't make a policy to do the impossible and expect it to just happen. But you *can* make a policy to do the very hard and seemingly impossible and make it happen. To that end I reference the fact that man has in fact been to the moon and it was a policy mandate by two competing governments that caused us (as a species) to do the work to get there. Deciding that a policy is good is the first step to being willing to commit to the work to implement it. It doesn't make sense to enact a policy which cannot be realized due to technical limitations, or whose realization causes unsolvable problems. The technical details are essential. You only need enough details to know that it isn't impossible, not enough to know the exact route to get to the end goal. Only a policy with a good technical implementation can be a good policy. In the end, this is true. In the beginning, it is not. [ snip the remainder of your non-reply ] Please, if you have objections to the policy, then state them. What I got from your email is you objected to me saying set policy first, technical implementation later, and then you never once said anything about the policy nor the technical implementation. Should I then take your silence on those issues to mean that you are OK with the policy and OK with the proposed technical implementation? -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
Doug Ledford dledf...@redhat.com writes: Limitations, yes. Current state, no. You can't make a policy to do the impossible and expect it to just happen. But you *can* make a policy to do the very hard and seemingly impossible and make it happen. To that end I reference the fact that man has in fact been to the moon and it was a policy mandate by two competing governments that caused us (as a species) to do the work to get there. Correction: it was a policy mandate plus the expenditure of a lot of billions of dollars that got us to the moon. It doesn't make sense to enact a policy which cannot be realized due to technical limitations, or whose realization causes unsolvable problems. The technical details are essential. You only need enough details to know that it isn't impossible, not enough to know the exact route to get to the end goal. You also need the resources to make it happen. A mandate from FESCO is not worth diddly-squat unless FESCO is prepared to do the work. regards, tom lane -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: To semi-rolling or not to semi-rolling, that is the question...
Hi, On 03/04/2010 09:59 PM, Doug Ledford wrote: Obviously, some people want this and some don't. It isn't appropriate to simply hand down an edict that things will be one way or the other if we truly consider Fedora a community run project. It must be a community decision. That means, as Adam Williamson pointed out, this is likely a decision to be made by the board, based upon what the community wants and what's best for Fedora. But let's be clear. That's a *policy* decision. One of the things that got very confusing in the previous thread(s) was the intermixing of policy decisions and technical issues. For instance, Kevin's response to my proposal was all about technical issues he saw. Technical issues are almost always solvable if you have a specific policy you are trying to implement. So one thing I think people should keep in mind is that policy decisions should always lead to technical decisions, *not* the other way around. We should decide what we want ourselves to be and what our policies are, and then that should guide our infrastructure, our tools, our work flows, and our processes. We should never allow things to flow in the reverse direction. We should never allow a current tooling limitation to set our policy, modulo that our policy should acknowledge and accommodate for the time necessary for tooling changes to take place or for the limitations of our resources. So, I'm going to reiterate my policy suggestion: Make Fedora releases (all of them) stable in nature, not semi-rolling. One size does still not fit all, although this is a great idea for most packages in Fedora for packages in certain niches this is a bad idea. I've said this before (and got 0 response), I believe there should be some divide made between core packages (with core being quite big, not the bare essentials, but also most of all desktop environments, etc.) and non core packages. For example I really see no reason not to upgrade some EDA tool to the latest and greatest if the maintainer thinks that there are good reasons to do this, because the group of EDA users bitten by potential regressions is small and EDA users are highly technical skilled, so know how to downgrade if needed. Another example is multiplayer games. In quite a few online gaming communities, most of the community moves over to the latest release once it is sanctioned stable by upstream. If the client - server version need to be in sync (which they often do because they need the exact same maps), then this means that our stable version in F-released might become worthless pretty quickly as there are no or very little stable version servers available to play on. I strongly urge FESco to come up with a policy which is flexible enough to handle these kind of special cases, without requiring going to rel-eng for an exception each and every time. And to mandate that the tools are flexible enough too. Also I cannot get rid of the collective punishment feeling here, because a few people do crazy things, ALL of us loose the right to apply common sense and have to adhere to strict policy and jump to even more hoops to get updates out there. How about a FAS flag which decides if a maintainer can skip updates-testing, which default to yes, and take this away from people who show to little restraint in skipping updates-testing ? Make rawhide consumable as a semi-rolling release itself. We already have this it is called early branching of the next release. I would fully agree with you if it were not for the early branching feature, which means we effectively already have such a release, except the first 2 months or so after a release, at which time rawhide tends to be very volatile in general (*), so a stabilized rawhide would pretty much boil down to the latest release anyways. Given that we pretty much already have this my reaction to this is please not another tree! * (we still need to see if no frozen rawhide changes this) Regards, Hans -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
