On Mon, Sep 23, 2013 at 1:55 PM, Animesh Chaturvedi <animesh.chaturv...@citrix.com> wrote: > > >> -----Original Message----- >> From: Marcus Sorensen [mailto:shadow...@gmail.com] >> Sent: Monday, September 23, 2013 12:25 PM >> To: dev@cloudstack.apache.org >> Subject: RE: [PROPOSAL] move away from time-based releases and/or revamp >> release process >> >> On Sep 23, 2013 1:03 PM, "Animesh Chaturvedi" >> <animesh.chaturv...@citrix.com> >> wrote: >> > >> > >> > >> > > -----Original Message----- >> > > From: Marcus Sorensen [mailto:shadow...@gmail.com] >> > > Sent: Monday, September 23, 2013 11:38 AM >> > > To: dev@cloudstack.apache.org >> > > Subject: [PROPOSAL] move away from time-based releases and/or revamp >> > > release process >> > > >> > > Guys, I think we are not currently in a state to handle time-based >> > > releases. Until we can cut master at any time and have it >> > > releasable, or at least at a reasonable RC-level matching minimum >> > > tested requirements, it's just going to continue to be an exercise >> > > in frustration to cut RCs simply because we hit a deadline. >> > [Animesh>] David is going to propose Release Criterion up for >> > discussion >> as per his thread [1] >> >> I see that thread more about defining what minimum bar we should always >> have master at in order to meet time-based releases. Its where we want >> to go, but not what to do in the meantime. > [Animesh>] His proposal is not just for master, but also for deciding the > release exit criterion and IMO is something we should follow for 4.3.0 and > onwards
Yes, I know. What I meant was that it will be a step toward stabilizing master, until we do that I'm not convinced we can adhere to any time-based expectation). It still doesn't fix our issue if we're going to insist on time-based releases, it just (from my undertanding) sets a bar for what is acceptable and what isn't, for any release. It stops the argument of "should we release with this bug". >> >> > > >> > > Maybe we can get away with sticking to time-based if we revamp our >> > > schedule and procedures, I don't know, but in light of how 4.1 >> > > (dragged on so long that some were seriously considering >> > > skipping/not releasing it with 4.2 on its heels) and 4.2 (six rounds >> > > of votes so >> > > far) have worked it's probably worth discussing. >> > > >> > > Any suggestions on what might be better? It's been mentioned in the >> > > past that it's a chicken-egg thing, many really don't try it until >> > > we hit an RC, which causes multiple iterations. I do agree that many >> > > don't take it seriously until we start cutting artifacts, but maybe >> > > we do this in a more deliberate fashion instead of jumping right to >> > > the vote. After feature/code freeze, cut some alpha artifacts, wait >> > > a week, cut alpha2 or some beta artifacts, etc, and then at some >> > > point anyone can propose that certain artifacts (or a new set of >> > > artifacts) be put up for a vote as an RC. This gives us a way to >> > > signal that we're gearing up for release and gives plenty of time >> > > for people to test their components, or see the [PROPOSAL] and say >> > > 'oh crap, I had better test my stuff', prior to cutting an RC. >> > > Maybe this wouldn't help in practice, but I think right now we go >> > > from telling the community "code is frozen, don't check anything in >> > > unless its a bug fix" to "here's our RC, try it out", without a >> formal testing window. >> > > I realize the whole thing should be a testing window, but I don't >> > > think it's conveyed well. >> > >> > [Animesh>] After the code freeze is all the stabilization and >> > integration >> testing phase and has been documented at [2]. No one should be waiting >> until the RC to test their components for the first time. It should be >> happening after code freeze. >> >> > >> > [1] http://markmail.org/thread/wlaq4zg36xnpgsjm >> > [2] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Releases >> > >> >> Got it. As mentioned I realize that the whole time there is supposed to >> be testing, but its not really working that way in practice. People are >> volunteers, they forget where things are, or they dont want to mess with >> it unless there is an indication that its semi-stable, and then suddenly >> an RC is thrown over the fence and we go through iterations of RC. By >> the time the RC comes through we should be done testing and just verify >> that someone's last minute bug fix didn't cause a regression or >> something. > [Animesh>] RC is not thrown in it is discussed as part of the release > schedule. After the code-freeze date everyone is expected to complete their > integration testing by RC date. In fact I had sent numerous reminders prior > to the first RC starting from 2 weeks before the proposed RC date. That's not the point. The code is changing at a rapid pace. Mike, for example, commented on tons of critical fixes going in right up until the RC is cut. Then we cut some artifacts and give people 72 hours to test and buy off. What I'm advocating is to lengthen the process, and not tie it to a timeline until we have better testing that stabilizes our master. At that time, when people can trust master remotely, then maybe individuals will take the time to poke at it prior to RC. Maybe that's a horrible idea, but let's at least talk about doing something until we're stable... or do we think we can accomplish that in a timely fashion? I think there are a few subgroups in our team here. 1) people whose job it is to develop on cloudstack, but don't really use it, 2) people who use cloudstack daily, and only do development to bugfix and/or add a pet feature. There may be some overlap for some individuals. This process might work great for individuals whose job it is to focus on cloudstack every single day and are tightly integrated with the massive changes, but the rest of us who consume cloudstack don't always have time to look at the big picture and focus on the unstable branches. We use the releases and focus on making the stable ones better and/or fixing/adding our pet features, until the next stable one comes around. Until the development branches stabilize I don't believe it will work for the users, they won't get involved until the end. For me, personally, it's a waste of time to even look at a branch that probably won't work due to sweeping changes that tend to occur between releases. Make your core changes, add spring, replace the storage subsystem, whatever it is, and then I'll go back and see what it broke after the bugs are worked out in all of that. That's how group #2 thinks, in general. And right now the only indicator that we're to that point is when we start talking RC, at which point I have a 3 day window that I hopefully catch and have time to play with it. > > My impression from your responses Animesh is that you feel everything is fine as-is. I don't know how anyone could think that given what we've seen over the last two releases, especially you who had to cut six RCs. We're blowing past our "time based releases", and trying to push through buggy releases (for some reason). My intent was to sum up and focus on some of the comments I've seen over the past few weeks about low/sporadic RC participation, major changes going on at the last minute, etc. I guess I'm in the minority though, since we're the only ones discussing it.
