I think Davids idea makes a lot of sense. If a bug is detected 2 days before the next release, it makes no sense to do a .1 release. If we discover an important bug directly after a release, I don't see a good reason *not* do a followup .1 release. That doesn't mean we have to switch back to "as-needed" release schedule for new major releases.
Regards, Michael 2015-12-18 16:39 GMT+01:00 David Lang <[email protected]>: > On Fri, 18 Dec 2015, Rainer Gerhards wrote: > >> Date: Fri, 18 Dec 2015 14:46:34 +0100 >> From: Rainer Gerhards <[email protected]> >> Reply-To: rsyslog-users <[email protected]> >> To: rsyslog-users <[email protected]> >> Subject: [rsyslog] release cycle - was: rsyslog 8.15.0 (v8-stable) >> released >> >> >> 2015-12-17 22:52 GMT+01:00 Michael Biebl <[email protected]>: >>> >>> 2015-12-17 16:51 GMT+01:00 Rainer Gerhards <[email protected]>: >>>> >>>> 2015-12-17 16:47 GMT+01:00 Thomas D. <[email protected]>: >>>>> >>>>> Hi, >>>>> >>>>> I agree with Michael. >>>>> >>>>> While I understand Rainers concerns in general this is different: For >>>>> you there are only test files missing. But for distributions there is >>>>> no >>>>> working v8.15 release (tests are really important for us). >>>> >>>> >>>> Can't you apply a patch? I remeber well in that long discussion over a >>>> year ago that you were on of the strong proponents of "it's easy to >>>> patch if something is a small nit"? >>>> >>>> I just want to understand that change of position, if it is one. >>> >>> >>> We run the test-suite as part of the Debian build (that's why I >>> noticed the failure). So this is a serious issue for the Debian build. >>> But sure, if it's too much of a hassle, I'll just add the missing >>> files as a downstream patch. >> >> >> I agree that it would be almost no work for me to do a re-release. >> It's more a policy argument. Let me explain, especially as I consider >> changing policy: >> >> Up until ~15 month ago, we released when there was need to. Need was >> defined as >> >> - important enough (set of bugfixes) >> - new functionality >> >> This resulted in various releases. We had the stable/devel releases. >> Stable releases were rare, devel frequent. >> >> Now, we have scheduled releases. Actually, a release is triggered when >> we hit a certain calender date, irrelevant of whether or not there is >> need to release (there is always one or two minor fixes, so we will >> probably never exprience a totally blank release). We also have >> switched to stable releases only, and done so without grief (basically >> because a) we have improved testing and b) users didn't use devel at >> all). >> >> I just dug into the old discussion. A good entry point is probably >> this here, where we talk about patches: >> >> http://lists.adiscon.net/pipermail/rsyslog/2014-October/038796.html >> >> The new system works reasonably well. It has it's quircks, though. >> Let's look at a concrete example: >> >> 8.14.0, to me, was an absolutely horrible release. The worst we have >> done in the past 2 to 3 years. I worked hard on fixing some real bad >> race issues with JSON variables. Friday before the release I was ready >> to release that work, which would be really useful for folks that make >> heavy use of those variables. Then, over the weekend and Monday, it >> turned out that we may get unwanted regressions that weren't detected >> earlier (NO testbench can mimic a heavy-used production system, so >> let's not get into "we need better tests" blurb). The end result was >> that I pulled the plug on release day, and what we finally released >> was 8.13.0 plus a few small things. All problems with variables >> persisted. If I had have half a week to a week (don't remember >> exactly) more, we could have done a real release instead of the 8.13 >> re-incarnation. But, hey, we run on a schedule. >> >> Now 8.15.0 fixes these problems (except for the json-c induced >> segfault, which we cannot fix in rsyslog). I also has all other "8.14" >> enhancements and fixes and so is actually worth 3 month of work. It is >> a *very heavy* release. Usually, I'd never released such a fat release >> shortly before the holiday period. Not that I distrust it, and we >> really got some new testing capabilites (really, really much better), >> so it is probably the most solid release for a longer time (besides >> the small quirk with the missing testbench files). But in general I >> don't like to do releases when I know there is very limited resources >> available to deal with problems. That's the old datacenter guy in me. >> But, again, hey, we run on a schedule. >> >> There have similiar occasions in the past 14 month. That's the >> downside. And due to the 6-week cycle things usually do not get really >> bad. >> >> The scheduled model has a lot of good things as well. First of all, >> everyone (users and contributors) know when the next release will be. >> This also means you can promise to include something into a specific >> release. However, usually users know when the release happens, but not >> what will be part of it, so in a sense it's not much better than >> before IMO. The new model has advantages for me: less releases mean >> less work. Also, I do not longer really need to think about when to do >> a release, which feature is important engouh and so. I just look at >> the calender and know that, for example, in 2016, November 15th we >> will have a release, no matter if I am present, no matter what is done >> code-wise etc (we actually had, for the first time everm a release >> while I was in vacation and it went really well as I learned later). >> That really eases my task. > > > the other big change was the change in mindset from "Ok, we made a stable > release, we can break things now and fix them over time before the next > stable release" to "master should be kept in a releasable state at all > times" > > The big adantage I see of the schedules release is that it (almost) > eliminates the pressure to get something that's not quite ready into _this_ > release because if it doesn't make this release, you know that the next one > isn't far away. > > I see 8.14 as an example of the new approach working perfectly. Instead of > agonizing over things for weeks as we tried to work things out (and it did > take weeks to finally track things down in this case), you reverted the > problem (just about everything, since we couldn't identify the problem) made > a release and we kept going working on the problem. > >> All of this bases on the "we release every 6 weeks, interim releases >> happen only for emergencies and anything else may be pulled as >> patches" policy. If we now begin to say "this problem is inconvenient >> to ..{pick somebody}", we need to do a re-release we get into trouble. >> I wonder which groups of "sombody" are important enough to grant >> non-emergency releases. Are only distro maintainers important enough? >> Probably not. So enterprise users? Mmmm.. maybe small enterprises as >> well? Who judges this? So let's assume every user is as important as >> every other (an idea I really like). If I then look at my change logs, >> I think I would need to release more frequent. In essence, I would >> need to release again when it is needed, which is, surprise, the >> as-needed schedule). >> >> Rsyslog is not a project big enough to do an even more complex release >> schedule. To keep things managable to me, I need to release either >> >> a) as-needed >> b) on schedule (except for *true* emergencies) >> >> And *that all* is the reason for my reluctance to break the release >> policy because this time distro maintainers experience the bug versus >> end users. >> >> I am currently tempted to switch back to "as-needed" mode, even though >> this means more work for me. > > > I would not like to see us move back to the "as-needed" mode the way things > were before, it was WAY too long between releases. Every 6 weeks is great, > if anything we could probably bump this out a couple weeks if it made a > noticable difference to your workload. > > That said, there are two slight tweaks I think we should consider. > > If we are on 8 week schedule, at week 7 we should look at outstanding bug, > especially on new features, and try agressively reverting changes. most of > the time this will just be 'remove the new feature', but once in a great > while it will end up with the 8.14 situation where you revert just about > everything, keeping only 'obviously correct, trivial bugfixes'. It wouldn't > hurt to have a standard e-mail go out to the list at the start of week 7 > reminding people that the release is only a week away, test the master > branch. > > If there is a regression (including build problems) reported within the > first couple days after a new release, and the fix is available within a day > or two, and the fix is either small (<10 lines), or the fix is reverting > patches applied during this release cycle , then I think releasing a .1 > within week 9 is reasonable. After about the midpoint of week 9, only > critical things would trigger a new release. So a .1, .2, etc would only > come out during week 9 except for a critical bug. > > If this makes releases enough more complicated that we need to push out to a > 8 week, or even 10 week release cycle, I think it would be worth it. > > I'd rather do a 'brown paper bag' release every year or so than have a > release with a fix that is trivial to fix out there. Based on how things > have gone over the last year, I think such releases would be pretty rare. > > I think the releases over the last year have worked out well. The pace of > releases has drastically increased, the effort of releases has decreased, > and the reliability of releases and rsyslog overall has increased. > > going to a pure 'as needed' release is a problem, a 'small' bugfix that > isn't worth a release to most people, is very much needed by someone > tripping over that bug in production. Having a guarantee that the bug will > be in a release withing a known period is worth a lot. > > David Lang > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
