Hi Graeme, Finally -- I have a moment to clarify my response without having to type on a tiny phone.
>>>>>> My view is that the connector components tend to be far more 'dynamic' and we should enable regular updates to them. As such, should we restrict ourselves to a single point release between minor releases (recent history seems to be that the first point release tends to fix major bugs in the previous release (I might be wrong though))? I realise that potentially this would probably require changes to be made against the existing release branch AND trunk. <<<<<< The policy heretofore has been that point releases are only for critical bugs; new features are basically not the reason we'd do a point release. There are limitations as to what you can indeed *do* in a point release, as I've alluded to earlier. Specifically, our current main rule is that point releases *cannot* make schema changes; this is in accordance with good standard industry practice, where people expect to not be able to back up from a major release to an earlier major release, but *may* expect to be able to back up from one point release to an earlier point release, should the need arise. So my proposal is that we don't relax the schema rule, but just the policy about what goes in a point release, and plan on having a point release roughly 2 months after the main release, if warranted, that will include hot new features pulled from trunk. Which reminds me -- seeing that you are new here it's good up front to note another long-standing standard development rule for MCF is that when you make changes, they get done as follows: - first, a ticket is created, with the "fix in" field set to the release(s) the change is targeted to go into - second, the change is committed to trunk in as few commits as possible, preferably just one (back to this point in a moment), and the svn revs are noted in the ticket - third, it is pulled up to zero or more release branches, as needed, using svn merge (which can be done by the person who is handling a release, and not the original developer), and the svn revs of these commits are also noted in the ticket There are, of course, limits to this; you can't reasonably pull up a feature to a release branch that requires an earlier feature that isn't pulled up. So the release engineer in charge of the release branch has the responsibility of making sure it all works out. And, because actual development of a feature is often messy, we try to limit any churn on trunk by creating development branches, whenever it looks like a change will have a broad enough impact. That's why you will see a lot of branches named after tickets, e.g. branches/CONNECTORS-123. Hope this helps; Karl On Sun, Mar 16, 2014 at 7:16 AM, Karl Wright <[email protected]> wrote: > Hi Graeme, > > Just to clarify, I am not proposing a single point release between > minor releases. We can do as many releases as needed. It is just that > the criteria for whether to build a point release would now include > features needed by clients, rather than just critical bug fixes. > > As for separating framework and connectors -- there are several issues > with this. First, voting on individual packages in apache is time > consuming, and you need a committer quorum. Committers who do not use a > connector do not usually evaluate it for release. So, such a move would > have the effect of orphaning most of our lightly-used connectors. > > Second, most of the features of the framework are currently exercised > via integration tests involving connectors which use those features. > Before such a separation could occur, new tests would need to be > written which would remove that dependency. > > Karl > > Sent from my Windows Phone > From: Graeme Seaton > Sent: 3/16/2014 5:02 AM > To: [email protected] > Subject: Re: Reminder -- MCF 1.6 development must complete in 4 weeks > Karl, > > Sounds fair enough. > > My view is that the connector components tend to be far more 'dynamic' > and we should enable regular updates to them. As such, should we > restrict ourselves to a single point release between minor releases > (recent history seems to be that the first point release tends to fix > major bugs in the previous release (I might be wrong though))? I > realise that potentially this would probably require changes to be made > against the existing release branch AND trunk. > > On a related train of thought, would it be worth separating out the > builds for the 'core' components and the connectors? ATM we have a > separation between open and proprietary connectors, would it be worth > doing the same for the main infrastructure? (I have to admit that I > currently have a separate project in eclipse which points to the main > Manifold build but adds a couple of internal connectors to the project > for deployment). > > Regards, > > Graeme > > On 15/03/14 08:14, Karl Wright wrote: > > Thinking about release cycles still further... > > > > Right now there is a "scheduled" release type, and an "as needed" > > release type. The "scheduled" release corresponds to the creation of a > > release branch, and the as-needed releases correspond to a release from > > the release branch. Graeme, I can't visualize how the release > > engineering would work for two kinds of scheduled release and > > additional as-needed releases for each type. Two levels of release > > branch? What if you fix a ticket that should go out in a minor changes > > point release? Would you need to pull up the change into two other > > branches? > > > > I guess the concern is that there would need to be a lot of releng > > complexity needed to manage releases in this way. It might be worth it > > if we had two separate teams involved in the project, e.g. a team > > handling architectural changes and another responsible for minor > > releases. But we definitely don't have that kind of setup in this > > apache project. The same people have to do everything here.... > > > > So here is a counter-proposal: let us lengthen the main scheduled > > release cycle to four months. Then we leave the option open of a point > > release at two months if it seems to be warranted and there have not > > been any other point releases since the last scheduled release. We will > > hold a formal vote on whether to have such a point release, and/or the > > parties with the most interest in seeing it go out the door can take > > the release management responsibility for it. Thoughts? > > > > Karl > > > > Sent from my Windows Phone > > From: Karl Wright > > Sent: 3/14/2014 6:48 PM > > To: Graeme Seaton; [email protected] > > Subject: RE: Reminder -- MCF 1.6 development must complete in 4 weeks > > Hi Graeme, > > > > Usually we don't get much of a choice about minor features. These are > > largely user driven. But I think true releases could well work more > > or less in the manner you describe. > > > > Basically, though, releases have the philosophy of "the train leaving > > the station". They go out on a schedule and contain everything > > developed that fits in the schedule. In many ways this is like agile > > development, but there are no scrums yet and the cycle is longer. > > > > A tick-tock release plan on its face requires a level of planning we > > don't usually have the luxury of. Using point releases to get > > software released while we work on architectural changes, though, is > > perfectly consistent with how we have been viewing things to date. > > > > Karl > > > > Sent from my Windows Phone > > > > -----Original Message----- > > From: Graeme Seaton > > Sent: 3/14/2014 5:25 PM > > To: [email protected] > > Subject: Re: Reminder -- MCF 1.6 development must complete in 4 weeks > > > > Or an alternative is to do a 3 monthly releases where even minor > > releases involve architectural changes and odd's only include bug > > fixes/new connectors. > > > > Graeme > > > > On 14/03/14 21:18, Graeme Seaton wrote: > > > > Karl, > > > > Thinking about it some more, guess I'm arguing for a 6 month minor > > release number update with a defined 3 monthly point release in > > between (with potential other minor releases intervening). > > > > Regards, > > > > Graeme > > > > On 14/03/14 21:11, Graeme Seaton wrote: > > > > Hi Karl, > > > > Don't disagree but was thinking that Tick releases would be labelled > > 1.6.0 (for example) and the Tock would be 1.6.5 to conform with the > > conventions. If we ran into significant issues then interim releases > > would be (for example) 1.6.1 and 1.6.6 (and if we need more space in > > the numbering scheme then we probably have some major other issues). > > > > Regards, > > > > Graeme > > > > On 14/03/14 18:41, Karl Wright wrote: > > > > Hi Graeme, > > > > The release numbers are pretty well defined according to industry > standard. > > > > Specifically, in MCF's case, this is the arrangement: > > > > <major>.<minor>.<point> > > - Major release increments only when backwards compatibility is broken > > - Minor release increments for significant tri-monthly periodic updates > > - Point release means a patch or bug-fix release > > > > We're not allowed to do schema changes in point releases, for instance, > but > > minor releases we can. But it is also allowed to put straightforward > > feature improvements out in point releases; we just haven't had the > need. > > > > There are also no plans at the time to bump the major release number; the > > kind of thing that would require such a bump might be reimplementing > > ManifoldCF on top of Voldemort, for instance. > > > > Karl > > > > > > > > > > > > On Fri, Mar 14, 2014 at 2:32 PM, Graeme Seaton <[email protected]> > wrote: > > > > > > To quickly follow up - we could also only increment the 'minor' version > ie > > 1.x every 6 months which would allow an amount of time for deployment > > planning - the last thing I think any of us want is for versions in the > > field to fall behind significantly due to 'upgrade' fatigue. > > > > Graeme > > > > > > On 14/03/14 18:26, Graeme Seaton wrote: > > > > > > Thanks to everyone for the greetings. > > > > IMHO, I'd still like to see 3 monthly releases (because it provides the > > project with a level of momentum) but perhaps we could structure them > in a > > similar way to Intel's Tick/Tock approach - every other release is > focussed > > on architectural/infrastructure related items (and potentially new > > connector functionality) while the other is a > stabilisation/connectors-focussed > > release. > > > > (and yes I would love you to have the time to update the book :-D) > > > > Regards, > > > > Graeme > > > > On 14/03/14 11:53, Karl Wright wrote: > > > > > > Also, FWIW, I would like to propose that we consider slowing down the > > release cycle, maybe going from 3 months to 4 or 6 months per release. > > While we've still been getting strong bursts of activity, we've also > been > > seeing relative lulls now, and the project is definitely getting more > > mature. This might allow me time to update the book. ;-) > > > > Of course, this is a tradeoff. Getting new features out is important > > both > > for the development of the features and for the utility of the project > to > > others. Other projects release software based on some notion of how > many > > features/changes they've put into it. We too could do that -- but I've > > found in my professional life that such a way of releasing software > often > > either leads to chronic release stress or releases that are so long > > people > > even forget what's in them. > > > > Thoughts welcome on how we should manage release cycles going forward. > > > > Karl > > > > > > > > On Wed, Mar 12, 2014 at 8:56 PM, Karl Wright <[email protected]> wrote: > > > > Hi folks, > > > > MCF 1.6 is scheduled to be released on April 30, and should be completed > > on April 15th, if we stick to the standard three-month release cycle > for > > this project. > > > > There are a number of tickets I'd love to get resolved in this > > timeframe. > > I've listed the key ones below: > > > > CONNECTORS-909: Upgrade to the latest ElasticSearch (Abe Shinichiro) > > CONNECTORS-856: Get a mock testing framework working (Alessandro > > Benedetti, largely) > > CONNECTORS-565: SharePoint 2013 support (Karl Wright) > > > > Please let me know if there will be any problem finishing these tickets > > in > > a month. > > > > Thanks! > > > > Karl >
