On 06/09/2012 03:39 AM, Vincent Massol wrote: > > On Jun 9, 2012, at 6:28 AM, Sergiu Dumitriu wrote: > >> On 06/08/2012 05:07 AM, Caleb James DeLisle wrote: >>> >>> It seems that if we want to shorten a release cycle, we have 2 options: >>> >>> #1 No RC release >>> #2 No staging >>> >>> I think it would be a shame to scrap staging over this, especially since >>> my understanding is we want to move toward frequent releases with no >>> release candidates or milestones so staging would be a requirement. >>> >>> This is an interesting topic which can be discussed. >>> >>> Right now for 4.1 I think we need a coherent proposal rather than an adhoc >>> chop-and-slice of the agreed upon schedule and dates. >>> Finally, I'm concerned about changing our release process in the middle >>> of a release which is behind it's normal schedule. I'm not completely >>> opposed to a change but IMO if we want to change it we need to proceed with >>> extra caution. >> >> The purpose of the RC releases was as a kind of staging, without actually >> using real staging. So I'd say that for the next releases we should only >> have milestones, and staged final releases. >> >> However, for 4.1 we should keep the rc-1 release in the plan, since our >> staging process is new and not thoroughly tested yet. >> >> The way I simulated "staging releases" when I had the RM hat was to leave a >> window between uploading the artifacts in our maven repository and pushing >> them further, and in this window I did quick smoke tests to see that >> everything works (with the help of Sorin and the rest of the QA team at >> XWiki SAS). >> >> So, I propose this as the strategy for our next releases: >> - Each release goes to a staging repository first >> - Milestone releases spend a little time in staging, enough for a few quick >> smoke tests; depending on the time of day, this could last from an hour to >> at most a day >> - There are no RC releases, there are only RC staging repositories. >> Currently the staging repositories have random names, but the RM should >> rename each repository with the proper RC name: 4.2-rc-1, 4.2-rc-2 if the >> first build was busted, and so on. The version written inside the packages >> is the final one, like 4.2, without a rc-x suffix. While in staging, a RC >> release goes through more intensive tests, including smoke testing, the full >> MTR, and developers that contributed to the release should test their >> committed features. Maybe even a Jenkins job could be executed for that >> release, with all the automated tests. >> >> Timing? Since we don't know how many times we have to stage releases, I'd >> say that we can start building RCs one week before the planned final release >> date, and if bugs are found, a new RC is build the next day (to let issues >> accumulate), hoping that there will be a stable successful build before the >> final release date, without a 72h mandatory waiting period. > > I have one big problem with this strategy that I've already explained several > past emails. > > The current way of doing staging with RC is very powerful because it's a real > release and as such is advertised everywhere: on xwiki.org in download, in > blog post, on twitter, on wikipedia, freshmeat, etc. Which means we have a > lot of users who can download it and try it out. This will not happen with > staging. It won't be advertised significantly (it cannot since it's not a > real release) thus leading to a less good solution than what we have.
I've heard you mention this but I've not noticed any example of a user finding an issue in an RC and reporting it, causing it to be fixed for the final. I've only been around for 2 years though. Do you have such an example? > > Thus the only valid solution for me to reduce release cycles is to do 2 > things at the same time: > - work on the quality of our releases (i.e. increase our automated test suite > and test coverage) This is good no matter how we release. > - do short release cycle so that if a user finds a problem in say version > 4.45 then we release 4.46 the week after or 2 days after if it's critical > (this is basically what Jenkins is doing) > > Right now our cycle is 3 months for each release which means that if some dev > cannot make the date for ex he has to wait 3 months for his stuff to be > available to end users, which means devs are pushing stuff even if they are > half baked (missing tests for example). > > Thus, with the current situation, I'd very much prefer to not do staging (in > the sense of Nexus staging) and instead continue to use milestones and RCs. > Now we can still do staging as a quick check for the RM before pushing to the > main repo but it shouldn't last more than 1 day, as otherwise it adds up with > the RC time and thus leads to even longer release cycles. This would make sense if "final" was a clone of rc-X. What we do now is build an RC, fix whatever issues we have with it then build "final" and release it without manual testing. I do not like releasing final without testing, especially since we use the name rc-1, leading users to believe that we have a candidate is tested then selected for release. > > <brainstorming> > Moving forward: > > what we could do would be to reduce the whole cycle from 3 months to 3 weeks: > * 1 week milestone 1 > * 1 week milestone 2 > * 1 week final (we need less time for stabilization over what we have now > since there's less work done in m1/m2 than when they last 4 weeks each) I like the general thinking but from 12 weeks to 3 weeks is a huge step, I think we should at least try 6 week cycle and then do post-release brainstorming to see what issues come up with shortened release times. > > If a developer doesn't have time to push his stuff in m1 or m2 then he'll > have to wait only 1 week. > > Of course we should have Roadmap that last several releases so that we don't > do roadmap preparation all the time and so that we can put large features in > the roadmap that span more than 1 release. Actually the roadmap would list > priority stuff we want to work on but will not say in what exact version > we'll have it. This is good, currently we encourage developers to delay the release. > > The advantage of this is we'll get even more testing/feedback from community > since we'll have real releases and users will get > features/improvements/bugfixes pushed to them faster. The challenge is to > keep good quality but I think we can do that and in any case there's only one > way to know if it works: to try it ;) If we do manual testing on final before publicizing it then we're already ahead of the current process. Thanks, Caleb > > I think that without Sergiu's automated script for releases doing a release > was too costly to do this. This means doing 3 releases in 3 weeks (ie. one > per week). Is the automated script good enough to allow for this. > > Also we're going to have automated upgrades very soon now so it seems like > upgrading versions is going to be less an issue for users (note that they > don't have to upgrade at each version and they can skip releases). > > WDYT? Do you think the pace is sustainable for developers and that we won't > spend too much time in "bureaucracies"? > > IMO this would help us automate our release process even further so that it > becomes a nobrainer and will help us improve ourselves wrt quality/automated > tests even further. > > There's one problem maybe with APIs. They need time to mature but with the > young-api strategy that could be ok. > <brainstorming> > > Thanks > -Vincent > >>> Thanks, >>> >>> Caleb >>> >>> >>> On 06/08/2012 03:15 AM, Vincent Massol wrote: >>>> Hi guys, >>>> >>>> I'd like to bring an issue with this VOTE below. >>>> >>>> When I initially read it I didn't realize that this was about doing >>>> double-staging: >>>> * once with nexus staging >>>> * another one with the RC release >>>> >>>> So it increases the time we spend for doing releases instead of reducing >>>> it which is the direction we would like to go. >>>> >>>> The increase is bad because we're already spending too much time just on >>>> the release itself while we should reduce it to a minimum so that we can >>>> focus on developing new features/improvements/fixing bugs. >>>> >>>> So IMO if we really want to go with staging we need to remove the RC phase >>>> and go from M2 to Final directly. However if we were to do this we would >>>> need to find a way to advertise it as a release on all channels because >>>> this is the time when we need to most testers. Right now it seems to me >>>> that an official RC is much more powerful than staging >>>> >>>> Thus I'd like to retract my vote on this (if it's not possible I'll send a >>>> new vote to not do double staging). >>>> >>>> Thanks >>>> -Vincent >>>> >>>> PS: Sorry for not realizing this earlier... >>>> >>>> On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote: >>>> >>>>> Hi, >>>>> >>>>> I'd like to add staging to our official release process. >>>>> For milestone releases, I propose the staging cycle be for "0 time" (this >>>>> may be revisited later). >>>>> For RC or finals, we place the release in staging and immediately call a >>>>> VOTE to publish the release, this gives our testing team (everybody!) 72 >>>>> hours to raise a potential issue. >>>>> >>>>> Why: >>>>> >>>>> #1. After some chat on IRC I decided that it is advantageous to move >>>>> toward a faster release cycle and begin moving away from milestone >>>>> releases in favor of staging. This will set the stage for the release >>>>> method we will need. >>>>> >>>>> #2. Staging is easy, I've modified the release script to include staging >>>>> and with the script, it is a simple matter of about 5 clicks on nexus to >>>>> "login", "close repository", "release repository". >>>>> >>>>> #3. Staging is safe, the RM need not worry about fat fingers breaking the >>>>> release, all it costs is time. >>>>> >>>>> #4. The release process should be as close to the same as possible for >>>>> milestone and RC/final releases. This simplifies scripting of the >>>>> process, decreases the amount the RM must remember and makes every >>>>> milestone release a rehearsal. >>>>> >>>>> #5. Everybody else is doing it (is that even a reason?!) >>>>> >>>>> >>>>> Mandatory? >>>>> I would rather impress the RM with how easy and helpful staging can be >>>>> than bind him with rules. >>>>> If I had followed the existing process to the letter, I would not have >>>>> had any experience with staging to begin with. >>>>> In the interest of continuous improvement I would like to make this a >>>>> strong recommendation, not a strict rule. >>>>> >>>>> >>>>> Here's my +1 >>>>> >>>>> Caleb >> >> >> -- >> Sergiu Dumitriu >> http://purl.org/net/sergiu/ >> >> >> _______________________________________________ >> devs mailing list >> [email protected] >> http://lists.xwiki.org/mailman/listinfo/devs > > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs > _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
