If the problem is to not destabilize trunk, one solution is to create a branch and do all the release work in that branch. That way, people are not impacted by the releases in progress.
On Fri, Jul 6, 2012 at 3:24 PM, Holly Cummins < [email protected]> wrote: > Re-opening a slightly old conversation, just to ensure we have a > record of the advantages and disadvantages of each approach to > releasing groups of bundles. (I'm also updating the 'Releasing Aries' > web page with the pros and cons of each way of doing it.) > > On Thu, Jun 28, 2012 at 6:12 PM, Guillaume Nodet <[email protected]> wrote: > > I think you made a wrong assumption which is that the staging repo has > > to be backed by a single svn revision (which is btw not really the > > case because it comes from several tags anyway). > > What I'd suggest is doing the api bundles release and have them > > uploaded to nexus, then upgrade the implementations and all other > > components to use those releases, commit, release those other > > projects, THEN, close the staging repo and call for a vote. > > So we'd have a single staging repo / vote with a set of testable bundles > in it. > > This approach will break trunk unless we do an extra step. > > Say we release an API bundle, at version 1.0.0. The release plugin > will change the version of the API bundle in trunk to be > 1.0.1-SNAPSHOT. Normally, you'd just upgrade everything to depend on > 1.0.1-SNAPSHOT of the API bundle (and if we weren't doing release by > bundle, the release plugin would automatically update the dependencies > for us). However, it's important that things which depend on the API > bundle continue depending on 1.0.0, unless there's a good reason to > change (like they use a new method). Otherwise, we lose a lot of the > benefits of semantic versioning. Why? If we add a new method to the > API, the package version changes to 1.1, and anything compiling > against 1.0.1-SNAPSHOT won't resolve against 1.0.0, even if it would > actually *work* fine against 1.0.0. In order to avoid restricting our > modularity, we need to depend on 1.0.0 unless the 1.0.1-SNAPSHOT > methods are actually being used. > > So, continuing the example, in trunk, as part of the release, we've > switched to depending on 1.0.0 of the API. This compiles in the tagged > release, as long as bundles are compiled in a specific order. In > trunk, it won't compile at all, because nothing in trunk is building > API-1.0.0, and version 1.0.0 isn't in a maven repository yet. Instead, > trunk is building 1.0.1-SNAPSHOT. So to patch trunk up, we need to add > an extra step in which we switch back to depending on 1.0.0-SNAPSHOT, > and then a second extra step once the release is promoted, to depend > on 1.0.0. > > I think this is sensible to do occasionally, but it's error prone and > makes more work for the release manager. Obviously, the big advantage > of it is that the release gets voted through over a period of a few > days, rather than a month or two, for a big release. :) > > Holly > > > > > On Thu, Jun 28, 2012 at 6:45 PM, Holly Cummins > > <[email protected]> wrote: > >> Hi Guillaume, > >> > >> Thanks for your comments. Here are some of my thoughts ... > >> > >> On Thu, Jun 28, 2012 at 3:19 PM, Guillaume Nodet <[email protected]> > wrote: > >>> I think one problem, considering the current votes, is that it's > >>> really difficult to test anything. Releasing api bundles with no > >>> implementation to test is definitely not helping imo. > >> > >> I know what you mean about the testing, and I'm not totally sure what > >> the best answer is. I know what I'm releasing comes from trunk, and is > >> being tested by the Jenkins builds, so I'm pretty confident it works > >> in a real system. However being tested in more environments and by > >> more systems is obviously a Good Thing. I think the best way to test > >> the API bundles is with the current -SNAPSHOT bundles of the > >> implementation, either in something like the blog sample or some other > >> working system. If we weren't moving from 0.x to 1.0 you could also > >> test micro releases alongside existing impl bundles to ensure > >> everything resolves and works as claimed. > >> > >>> Holly, just a question: is there a specific reason why are you doing > >>> the release in multiple votes ? It would be simpler to just release > >>> everything in one go and wait for a longer time because there are more > >>> things to check, or at least, release the api + implementation so that > >>> we can actually try something. Just my 2 cents. > >> > >> I agree that this sort of 'extended incremental' release is a bit > >> awkward, and I was wondering when someone would ask what on earth I > >> was doing :). IMO it's the cleanest way to work with with > >> release-by-bundle (which I know you disagree with). If I release > >> everything in one go, there's a problem with the dependencies between > >> bundles. At the moment in trunk, almost every dependency is a SNAPSHOT > >> dependency. In the past we've updated all bundles to use non-SNAPSHOT > >> (but not yet released) versions in a branch, and I could even do > >> something similar without using a branch by briefly having the trunk > >> builds produce 1.0.0 artefacts. However, I think this creates a > >> greater burden for testers. If there are compilation-order > >> dependencies between parts of a release which don't share a top-level > >> pom, everyone verifying a script has to compile them in the right > >> order. I count 118 bundles to release, so that's a lot of bundles to > >> get in the right order, and I didn't think any PMC member would want > >> to try. :) I guess this could be automated with a verification script > >> which either hardcodes or calculates the dependency graph, but it > >> seemed to me like more work for everyone and more risk for the > >> release. My hope was that if verifying individual mini-releases was > >> easy enough, doing multiple ones wouldn't be a problem (and in fact > >> would nicely distribute any effort, making it easier to vote). > >> > >> I know at this stage some of you are thinking "and *this* is why > >> release by bundle is a bad idea!", and that's not really a debate I > >> want to re-open. Among other things, I think any re-engineering of our > >> poms at this stage will further delay the release. > >> > >> The good news is I believe this problem will almost entirely go away > >> for 1.0.x and 1.x releases, because the impl bundle will, in most > >> cases, depend on an *already* released version of its API bundle or > >> another Aries component. This means a bunch of related bundles could > >> be released at the same time, without compile issues, or a meaningful > >> release really could consist of just a single bundle. That's true > >> modularity and it should give both us and our users big benefits. > >> > >> > >>> > >>> On Mon, Jun 25, 2012 at 8:07 PM, Daniel Kulp <[email protected]> wrote: > >>>> > >>>> Honestly, with the change to using Nexus, the SHA1 and MD5 checks are > >>>> completely pointless. Nexus generates them itself based on what's > >>>> uploaded. The "is it a valid signature" part of the GPG testing is > also > >>>> pointless as Nexus won't let you close the repo unless the signatures > are > >>>> valid. The only check you really need to do is to make sure the key > that > >>>> was used is "trusted" by you. (aka: was it really Holly who > deployed those > >>>> artifacts) So the monontonous parts of checking that stuff is > really > >>>> irrelevant at this point. (providing we trust that infra has Nexus > >>>> sufficiently locked down and secure) > >>>> > >>>> > >>>> I actually don't have a big issue with the difficulting in getting > votes. > >>>> I'm involved in another community that has a PMC that is easily 4 > times the > >>>> size of this one, yet we still have difficulting getting votes there. > >>>> While not ideal, life events can cause priority shifts and such so > people > >>>> may not be able to be as responsive. > >>>> > >>>> My bigger problem is that the entire per bundle release process and > symantic > >>>> versioning crap has put a HUGE burden on the release manager. That > makes > >>>> it much harder to get quality releases out and makes it less likely > that > >>>> anyone will step up to get "minor fixes" released. The only reason I > >>>> stepped up with the 0.3.1 bp stuff is that *MY* customers are being > >>>> affected by it. Like wise for the proxy stuff. If *my* customers > were > >>>> not affected, I don't think I would have spent the time and effort. > If > >>>> the process for getting fixes and releases out to users was smaller > and > >>>> easier, I have no problem doing them. For CXF, we do full releases > on 3 > >>>> branches every other month or so. But that's because it's EASY to > do. > >>>> > >>>> If it was up to me, I'd toss out the entire versioning thing with 1.0 > and go > >>>> back to per module versioning thing. So my fix to proxy would have > >>>> involved checking out all of "proxy", fixing it, and releasing all of > proxy > >>>> as a proxy "0.3.1", even the modules that haven't changed. It's > just a > >>>> huge hassle to track down which bundles have changed, which haven't > which > >>>> version numbers need to be updated, etc.... If it's not quick and > easy to > >>>> do releases as a release manager, very few people are going to step > up to do > >>>> it. It may not be 100% "proper OSGi", but IMO, getting fixes and > such to > >>>> the users is more important than that. But that's my opinion. > >>>> > >>>> > >>>> Dan > >>>> > >>>> > >>>> > >>>> On Saturday, June 23, 2012 03:27:07 PM Holly Cummins wrote: > >>>>> Hi all, > >>>>> > >>>>> Now that Jeremy's taken the time to write up our release verification > >>>>> process, I'd like to propose we change it. :) I think it's too > onerous > >>>>> on the pmc, which therefore also inhibits our ability to be > responsive > >>>>> to our users. > >>>>> > >>>>> > >>>>> ------------------------------- Why what we have isn't working for > the > >>>>> community ------------------------------- > >>>>> > >>>>> I believe our users would like more frequent releases. We've had > >>>>> several keen requests and tweets and comments on the aries-user > >>>>> mailing list wishing we'd release more often. For example: > >>>>> > >>>>> * "Desperately waiting for an Aries release after loooong time.." > >>>>> * "The problem with Aries is they seem to be too busy coding to > >>>>> release anything." > >>>>> * "Compared to other projects (like Karaf and Camel) Aries releases > >>>>> tend to take quite some time." > >>>>> * "It's 2012 now and Aries 0.3 is almost a year old. Is there any > >>>>> chance of a new Aries JPA release any time soon? " > >>>>> * "Looks like Apache Aries has made no visible progress since Jan > >>>>> 2011, if the time stamps on the maven central artefacts are to be > >>>>> believed." > >>>>> > >>>>> ------------------------------- Why what we have isn't working for us > >>>>> ------------------------------- > >>>>> > >>>>> Both Dan and I are trying to do releases at the moment, and > struggling > >>>>> to get enough PMC votes. Dan's release is to back port a show-stopper > >>>>> proxy fix, so a release there is particularly pressing - he's got a > >>>>> non-binding +infinity vote, but that's all. My test support release > >>>>> vote has been open for about 64 hours, and only got one vote so far > >>>>> (thanks David B!). Obviously testsupport is less exciting than proxy, > >>>>> but that bundle does block more interesting releases. > >>>>> > >>>>> Why aren't people voting? My guess is that it's too much work to do > >>>>> the full set of verifications described at > >>>>> http://aries.apache.org/development/verifyingrelease.html. There are > >>>>> seven steps, and while they don't actually take that long to > complete, > >>>>> it's enough of a burden that we tend to leave the voting to someone > >>>>> else unless we really care about a release. I'm as guilty of this as > >>>>> anyone - I think a release is a good idea, but I'm spending enough > >>>>> time working on the 1.0.0 release that I don't want to take time out > >>>>> to vote on another release. I suspect Dan might feel exactly the same > >>>>> about my 1.0.0 bundles. :) > >>>>> > >>>>> With release-by-bundle, there's a lot of verifications. Excluding the > >>>>> sandbox code, we have 123 bundles to release in 1.0.0. At three votes > >>>>> per bundle, that means the PMC need to do 369 MD5 checks, 369 PGP > >>>>> checks, 369 RAT checks, and so on, just to get 1.0.0 out the door. > >>>>> This just doesn't seem like it scales. Batching the bundle releases > >>>>> together eases some of this burden, but not all. > >>>>> > >>>>> ------------------------------- What I propose > >>>>> ------------------------------- > >>>>> > >>>>> I suggest we move to a more trust-based system, where PMC members > >>>>> carefully check releases if they want, but where in general they're > >>>>> voting on the principle of the release, rather than the mechanics of > >>>>> the archives. In particular, they don't feel compelled to do checks > >>>>> before voting. If PMC members could say "Our users need this > function, > >>>>> so +1", or "I know Holly has done sensible things in the past, so +1" > >>>>> or even "Do I want to check the SHAs on a test support bundle? > Really? > >>>>> +1" it would get our releases moving better, and also save work for > >>>>> all of us. > >>>>> > >>>>> (At the moment I think what's happening is people are thinking "Do I > >>>>> want to check the SHAs on a test support bundle? Really?" and then > >>>>> skipping the +1 bit. :) ) > >>>>> > >>>>> To ensure that at least *someone* has run the checks, the release > >>>>> manager could include the output of the seven checks in an email to > >>>>> the list. I think this level of checking is perfectly compatible with > >>>>> the minimum Apache process, which is that the release manager signs > >>>>> the artefacts and three PMC members vote +1 > >>>>> (http://www.apache.org/dev/release-publishing.html#voted). > >>>>> > >>>>> What do people think? > >>>>> > >>>>> Holly > >>>> -- > >>>> Daniel Kulp > >>>> [email protected] - http://dankulp.com/blog > >>>> Talend Community Coder - http://coders.talend.com > >>> > >>> > >>> > >>> -- > >>> ------------------------ > >>> Guillaume Nodet > >>> ------------------------ > >>> Blog: http://gnodet.blogspot.com/ > >>> ------------------------ > >>> FuseSource, Integration everywhere > >>> http://fusesource.com > > > > > > > > -- > > ------------------------ > > Guillaume Nodet > > ------------------------ > > Blog: http://gnodet.blogspot.com/ > > ------------------------ > > FuseSource, Integration everywhere > > http://fusesource.com > -- ------------------------ Guillaume Nodet ------------------------ Blog: http://gnodet.blogspot.com/ ------------------------ FuseSource, Integration everywhere http://fusesource.com
