Re: [DISCUSS] Release process change
I don't think so. You don't have to use a single tree with a unique version. You can separate modules and release them independantly. I agree that each separate set of modules should be released at once, which is not the case currently with the specs. But we could easily split the modules in its own tree and separate them from the configs / assemblies. What's wrong with that ? On 12/23/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: This is NOT how mvn works folks... I've been trying to explain that to you for the past weeks. You just don get it :-/ Good luck --jason -Original Message- From: Matt Hogstrom <[EMAIL PROTECTED]> Date: Sat, 23 Dec 2006 00:15:39 To:[email protected] Subject: Re: [DISCUSS] Release process change On Dec 22, 2006, at 7:01 PM, Jason Dillon wrote: > On Dec 22, 2006, at 2:57 PM, Guillaume Nodet wrote: >> From what I understand, the problem you are focusing on >> is just a naming problem: you want the GA release to be >> named 2.0. That's fine. But it only concerns the final >> downloadables archives, right ? This is a good point. For the assemblies, from a "marketing" standpoint, we can version them any way we want. > > Nope, since mvn releases each component's artifact separately, its > not just the zip/tgz that one traditionally thinks of as the > release... its every single artifact that gets spit out of the > release. We should deal with the module versions separately from the assemblies. I'll hop back on this next week...I'm taking a few days of R & R. Cheers Matt Hogstrom [EMAIL PROTECTED] -- Cheers, Guillaume Nodet
Re: [DISCUSS] Release process change
This is NOT how mvn works folks... I've been trying to explain that to you for the past weeks. You just don get it :-/ Good luck --jason -Original Message- From: Matt Hogstrom <[EMAIL PROTECTED]> Date: Sat, 23 Dec 2006 00:15:39 To:[email protected] Subject: Re: [DISCUSS] Release process change On Dec 22, 2006, at 7:01 PM, Jason Dillon wrote: > On Dec 22, 2006, at 2:57 PM, Guillaume Nodet wrote: >> From what I understand, the problem you are focusing on >> is just a naming problem: you want the GA release to be >> named 2.0. That's fine. But it only concerns the final >> downloadables archives, right ? This is a good point. For the assemblies, from a "marketing" standpoint, we can version them any way we want. > > Nope, since mvn releases each component's artifact separately, its > not just the zip/tgz that one traditionally thinks of as the > release... its every single artifact that gets spit out of the > release. We should deal with the module versions separately from the assemblies. I'll hop back on this next week...I'm taking a few days of R & R. Cheers Matt Hogstrom [EMAIL PROTECTED]
Re: [DISCUSS] Release process change
On Dec 22, 2006, at 7:01 PM, Jason Dillon wrote: On Dec 22, 2006, at 2:57 PM, Guillaume Nodet wrote: From what I understand, the problem you are focusing on is just a naming problem: you want the GA release to be named 2.0. That's fine. But it only concerns the final downloadables archives, right ? This is a good point. For the assemblies, from a "marketing" standpoint, we can version them any way we want. Nope, since mvn releases each component's artifact separately, its not just the zip/tgz that one traditionally thinks of as the release... its every single artifact that gets spit out of the release. We should deal with the module versions separately from the assemblies. I'll hop back on this next week...I'm taking a few days of R & R. Cheers Matt Hogstrom [EMAIL PROTECTED]
Re: [DISCUSS] Release process change
On Dec 22, 2006, at 2:57 PM, Guillaume Nodet wrote: From what I understand, the problem you are focusing on is just a naming problem: you want the GA release to be named 2.0. That's fine. But it only concerns the final downloadables archives, right ? Nope, since mvn releases each component's artifact separately, its not just the zip/tgz that one traditionally thinks of as the release... its every single artifact that gets spit out of the release. For this to work in mvn, the version of everything needs to change together, so that you can be sure that one release is using the right set of binaries vs. another. Otherwise you have to set the version of the project to 2.0.0 and then you could go through many iterations which all use the same version, which means deploying them over previous version and the end result is that you can never be certain which version of 2.0.0 you end up including into your build. To avoid publishing all of these binaries over and over the mvn way is to make the project as a SNAPSHOT (like 2.0.0-SNAPSHOT) which will deploy new artifacts using timestamps to avoid clobbering previous version... BUT, when it comes time to actually make one of those a GA release, then you have to update source code, rebuild and the end result might be something quite different than the previous release that every one has certified and voted upon. Its far to easy to make a small mistake in the build process to dramatically alter the build outputs... especially since you need to make source code changes (and lots of them) to flip the version. This would lead to using maven release to perform the first release, named 2.0.0.0 and start a QA cycle. If a new version is to be made, you can call it 2.0.0.1 and so on, until we are satisfied. Yup, this means that 2.0.0.1 - 2.0.0.x will end up into the central sync repo and pushed out to mirrors. Not ideal. This can be avoided by using a profile to set the distribution URL to something else (and hope that people don't forget to enable that profile)... and then you are forced to use mvn repo copy stuff (which was just added) to perform the official deploy from a staged repo. Which... well, hopefully will just work, if not then we risk corrupting metadata on releases, which would be very bad. That is a bunch of intermediate steps we must go through to work around deficiencies in a mvn projects releasability. The more steps, the more chances that something will go wrong or that someone will miss something. This is really a huge PITA from a build release managers perspective... massive hoops to jump through to get mvn closer to (yet still far off from) a simple release process. IMO, many of these problems go away when you don't need to deploy/ publish every single intermediate artifact, or are lessened if you could deploy a previously build artifact with out needing to rebuild it. The whole need to rebuild artifacts to deploy is a huge hole... you can't be sure that artifacts built with `mvn install` will be the same artifacts used for `mvn deploy`. In most cases the artifacts will change. So you can't really run ` mvn install` to ensure that your build completes and then `mvn deploy` to simply publish what was recently built. And you can't safely just `mvn deploy` and if that fails halfway through, then you end up with a half deployed release. And for a large project like Geronimo server, cleaning up all those release turds is a PITA... and really, re-releasing the same version over it just causes more potential problems, as others might have already started to consume parts of the newly deployed partial released artifacts. When the release is voted, we keep these binaries, renamed then to 2.0 and upload them to the mirroring system. At the same time, you move the individual artifacts from the staging repo to the official repo. You could do that, really only for the zip/tgz bits... though that adds more manual steps, as well as introduces some inconsistence with the versions of internal artifacts. If internally we did not use the same mvn style repo with version information all over everything, then I think this would work okay, but we have version muck all over everything inside of Geronimo. But that is not the case, so you will download 2.0 and find that its really 2.0.0.3 and then download 2.1 and find that its really 2.1.0.5, not really what one might expect. It might be more of a cosmetic problem, but we've already see requests from Hernan for screen shots to strip out #rev numbers and such. What I mean, is that when Geronimo releases a 2.0 version, this is a "marketing" version number. All the artifacts inside are not in version 2.0: openejb, activemq and all other external dependencies have their own lifecycle. So I don't really see a big problem if modules and configurations have a different version
Re: [DISCUSS] Release process change
Just some thoughts on the QA stuff. From what I understand, the problem you are focusing on is just a naming problem: you want the GA release to be named 2.0. That's fine. But it only concerns the final downloadables archives, right ? These are usually downloaded from the Apache mirroring system, and uploaded manually (not by maven). So we can freely rename them before uploading. This would lead to using maven release to perform the first release, named 2.0.0.0 and start a QA cycle. If a new version is to be made, you can call it 2.0.0.1 and so on, until we are satisfied. When the release is voted, we keep these binaries, renamed then to 2.0 and upload them to the mirroring system. At the same time, you move the individual artifacts from the staging repo to the official repo. What I mean, is that when Geronimo releases a 2.0 version, this is a "marketing" version number. All the artifacts inside are not in version 2.0: openejb, activemq and all other external dependencies have their own lifecycle. So I don't really see a big problem if modules and configurations have a different version number from the final Geronimo release. If you find it too cumbersome, let's say we revert back the modules to 1.3, and we use them to release a 2.0. It would become more obvious I think. I don't imply this is the best solution, but it may be one. Thoughts ? On 12/22/06, Jason Dillon <[EMAIL PROTECTED]> wrote: On Dec 22, 2006, at 11:14 AM, David Jencks wrote: > I thought from looking at the maven dev list that maven now or in a > couple days is going to support staging releases. I'm considerably > less than thrilled by the idea of changing what we vote on, > especially just when it looks like hope is on the horizon. I'm not convinced that the recent work from the mvn team to improve the release process for mvn projects is going to solve the release problem... I do believe that it is a step forward, but there are still a few major issues which I don't see being addressed yet. One of the big problems is certification of binaries, and release build iteration. For example, many shops generally like to produce a set of release artifacts, promote them to QA, let QA perform some battery of tests and then bless the binaries. To accomplish this with mvn (today and even tomorrow or whenever the new release support plugins are finished) means that a "real" release must be made for each iteration to QA, and when QA comes back and rejects a release, code must be altered and a new release must be made and the cycle continues until one of blessed. In mvn terms that means that version 1.0.0 might get passed to QA, rejected, and then 1.0.1 is spun and handled to QA, rejected... etc... maybe the *actual* release is 1.0.3, instead of 1.0.0 which users are expecting. IMO, the problem with mvn releases, is that there is so much version encoding in each artifact (name, embedded poms and dependency configuration) that it is not possible to take a set of binaries and promote them from a DEV release, to QA release to a final blessed release. You must in each case rebuild to change versions, and by rebuilding some of the certification is essentially lost. You and I might both know that only version number have changed... but it is possible that when changing versions to make a release build that something else in your environment has changed (say different JDK was used), or perhaps the previously downloaded artifact cache was lost, and the build must download new dependencies, which could potentially be altered, producing output artifacts which are different. Unfortunately mvn can not even take a previously build set of binaries in a projects target directory and simply install or deploy them... but in order for it to deploy it must go through and run the previous phases and will re-generate jars and other artifacts. So, you can't even really be 100% sure that in a simple local environment, that building artifacts, performing some set of test, and then deploying will actually deploy the same set of artifacts that you tested with. Unfortunately mvn differs greatly from ant projects in that it tends to redo work instead of detecting that no changes are needed and skipping steps. While it will only recompile changed sources (for java at least), it will always at the end regenerate a jar and install it into the local repo. So far I do not see the mvn folks making any progress on solving this problem... and IMO, this is required if you are going to certify binaries. > I'm definitely in favor of releasing the specs now, but I'd be more > comfortable voting on binaries as we have in the past. Unfortunately, to do this... if there is a problem with one of the binaries, they must be rebuilt using the same version numbers, so its not easy to tell which artifacts are from which iteration. Only way around this IMO is to use a separate staged deploy directory for each iteration... which isn't something that can be easily
Re: [DISCUSS] Release process change
On Dec 22, 2006, at 11:14 AM, David Jencks wrote: I thought from looking at the maven dev list that maven now or in a couple days is going to support staging releases. I'm considerably less than thrilled by the idea of changing what we vote on, especially just when it looks like hope is on the horizon. I'm not convinced that the recent work from the mvn team to improve the release process for mvn projects is going to solve the release problem... I do believe that it is a step forward, but there are still a few major issues which I don't see being addressed yet. One of the big problems is certification of binaries, and release build iteration. For example, many shops generally like to produce a set of release artifacts, promote them to QA, let QA perform some battery of tests and then bless the binaries. To accomplish this with mvn (today and even tomorrow or whenever the new release support plugins are finished) means that a "real" release must be made for each iteration to QA, and when QA comes back and rejects a release, code must be altered and a new release must be made and the cycle continues until one of blessed. In mvn terms that means that version 1.0.0 might get passed to QA, rejected, and then 1.0.1 is spun and handled to QA, rejected... etc... maybe the *actual* release is 1.0.3, instead of 1.0.0 which users are expecting. IMO, the problem with mvn releases, is that there is so much version encoding in each artifact (name, embedded poms and dependency configuration) that it is not possible to take a set of binaries and promote them from a DEV release, to QA release to a final blessed release. You must in each case rebuild to change versions, and by rebuilding some of the certification is essentially lost. You and I might both know that only version number have changed... but it is possible that when changing versions to make a release build that something else in your environment has changed (say different JDK was used), or perhaps the previously downloaded artifact cache was lost, and the build must download new dependencies, which could potentially be altered, producing output artifacts which are different. Unfortunately mvn can not even take a previously build set of binaries in a projects target directory and simply install or deploy them... but in order for it to deploy it must go through and run the previous phases and will re-generate jars and other artifacts. So, you can't even really be 100% sure that in a simple local environment, that building artifacts, performing some set of test, and then deploying will actually deploy the same set of artifacts that you tested with. Unfortunately mvn differs greatly from ant projects in that it tends to redo work instead of detecting that no changes are needed and skipping steps. While it will only recompile changed sources (for java at least), it will always at the end regenerate a jar and install it into the local repo. So far I do not see the mvn folks making any progress on solving this problem... and IMO, this is required if you are going to certify binaries. I'm definitely in favor of releasing the specs now, but I'd be more comfortable voting on binaries as we have in the past. Unfortunately, to do this... if there is a problem with one of the binaries, they must be rebuilt using the same version numbers, so its not easy to tell which artifacts are from which iteration. Only way around this IMO is to use a separate staged deploy directory for each iteration... which isn't something that can be easily enforced/ automated by mvn itself, so this ends up being a potential problem when running a release, to make sure you configure the iteration when publishing for certification. * * * I don't think that projects using mvn should really need to use mvn to make releases... but unfortunately due to the complexity of actually making a release for a mvn 2.x project it is a necessity. With all those versions and scm bits to update... users are kinda forced to use the release plugin. And... I don't think that every single artifact which is included into a server build should be "released". For the server, the release is the assembly zip/tgz... mvn forces us to release *everything* that is used to make those assemblies and IMO is a huge mistake. * * * When it comes to building a project, mvn does a pretty good job, better job with out snapshots. Its plugin system is really getting to be solid and functional, though some of the bugs are forcing people to implement potentially dangerous workarounds. Actually building artifacts is reasonable, though most plugins tend to always redo work, so re-running a build that is already built will redo a bunch of stuff, which just slows things down and introduces unwanted side-effects. Site generation is novel, though most mvn sites are basically
Re: [DISCUSS] Release process change
On Dec 22, 2006, at 1:06 PM, Dain Sundstrom wrote: On Dec 22, 2006, at 6:27 AM, Hernan Cunico wrote: I'm OK with the change but would like to understand the proposal a little bit better. Not knowing Maven enough I do have a few questions. What would be the equivalent to branches? How would we manage the "cut" for stabilizing and releasing? Maven only handles the "tag" creation setp. If I understand Matt's proposal correctly, the release manager would still do the branching and once the code is stable, the community votes to release the code as it without further changes. They you run the maven tools to create the tags and the binaries. I thought from looking at the maven dev list that maven now or in a couple days is going to support staging releases. I'm considerably less than thrilled by the idea of changing what we vote on, especially just when it looks like hope is on the horizon. I'm definitely in favor of releasing the specs now, but I'd be more comfortable voting on binaries as we have in the past. ATM I'm in an internet constrained environment and haven't been able to do a review of what's up for voting. Hopefully this will be fixed in a day or two. thanks david jencks How would we manage *TCK*? Again, if I understand Matt's proposal, this would not apply to Geronimo server, so TCK is not an issue. -dain
Re: [DISCUSS] Release process change
On Dec 22, 2006, at 6:27 AM, Hernan Cunico wrote: I'm OK with the change but would like to understand the proposal a little bit better. Not knowing Maven enough I do have a few questions. What would be the equivalent to branches? How would we manage the "cut" for stabilizing and releasing? Maven only handles the "tag" creation setp. If I understand Matt's proposal correctly, the release manager would still do the branching and once the code is stable, the community votes to release the code as it without further changes. They you run the maven tools to create the tags and the binaries. How would we manage *TCK*? Again, if I understand Matt's proposal, this would not apply to Geronimo server, so TCK is not an issue. -dain
Re: [DISCUSS] Release process change
I meant things like DayTrader, XBean, DevTools, or broken out modules like Tx mgr which we've been talking about for some time. I guess it was a little ambiguous...basically everything except the big G On Dec 22, 2006, at 2:51 AM, Guillaume Nodet wrote: On 12/22/06, Matt Hogstrom <[EMAIL PROTECTED]> wrote: Vote: Allow individual modules to be released based on source content in SVN as indicated by the release manager and use Maven to produce and release the appropriate artifacts. What do you mean by individual modules ? This could apply to the whole spec tree, or any other subproject as a whole ? -- Cheers, Guillaume Nodet Matt Hogstrom [EMAIL PROTECTED]
Re: [DISCUSS] Release process change
I'm OK with the change but would like to understand the proposal a little bit better. Not knowing Maven enough I do have a few questions. What would be the equivalent to branches? How would we manage the "cut" for stabilizing and releasing? How would we manage *TCK*? Cheers! Hernan Matt Hogstrom wrote: David brought up a proposed change to the release process around some specs that he is currently working through. As you all know the release process has been a challenge for quite some time. Branching, building, voting, building, voting, building voting, etc. Which is quite time consuming. It may be that we've gone too far in the release process in terms of how were structured. We need a better way that ensures we are releasing good binaries in terms of code quality as well as meeting the legal requirements. I think David's suggestion is a step in the right direction. We elect someone to be the release manager which in essence empowers them to follow through the process of getting software out the door. In David's proposal (currently for specs and this is a great starting point) is to propose a release and use Maven to release:prepare / release:perform to create the final binaries and place them into the Maven repo. In essence, the release manager (or whoever is working to release some component) indicates their desire and points to the branch to be released. This effectively defines the content of the release. Folks on the project would then build from that branch and satisfy themselves that they are satisfied with the content and vote. The release manager would then release using the Maven plugin and be responsible for the actual distribution. Based on the way we can release with Maven 2 we are significantly closer to release early / release often. The content of the vote would be something like Vote: Allow individual modules to be released based on source content in SVN as indicated by the release manager and use Maven to produce and release the appropriate artifacts. I'd like to leave this open for discussion through next Friday and bring this up for a vote first week in January. This discussion thread is the place to get ideas on the table. I suspect that this will not garner 100% support so this vote will be a majority consensus. Remember, we can always change things later if they aren't working out. If you like the proposal as is just simply give a +1 so people can know your opinion. Its not a vote in this thread but simply a quick determination of opinion. For those who haven't released before, this will really simplify the process and make it less tedious and allow more people to be involved. Matt Hogstrom [EMAIL PROTECTED]
Re: [DISCUSS] Release process change
On 12/22/06, Matt Hogstrom <[EMAIL PROTECTED]> wrote: Vote: Allow individual modules to be released based on source content in SVN as indicated by the release manager and use Maven to produce and release the appropriate artifacts. What do you mean by individual modules ? This could apply to the whole spec tree, or any other subproject as a whole ? -- Cheers, Guillaume Nodet
Re: [DISCUSS] Release process change
+1 Jacek On 12/22/06, Matt Hogstrom <[EMAIL PROTECTED]> wrote: David brought up a proposed change to the release process around some specs that he is currently working through. As you all know the release process has been a challenge for quite some time. Branching, building, voting, building, voting, building voting, etc. Which is quite time consuming. It may be that we've gone too far in the release process in terms of how were structured. We need a better way that ensures we are releasing good binaries in terms of code quality as well as meeting the legal requirements. I think David's suggestion is a step in the right direction. We elect someone to be the release manager which in essence empowers them to follow through the process of getting software out the door. In David's proposal (currently for specs and this is a great starting point) is to propose a release and use Maven to release:prepare / release:perform to create the final binaries and place them into the Maven repo. In essence, the release manager (or whoever is working to release some component) indicates their desire and points to the branch to be released. This effectively defines the content of the release. Folks on the project would then build from that branch and satisfy themselves that they are satisfied with the content and vote. The release manager would then release using the Maven plugin and be responsible for the actual distribution. Based on the way we can release with Maven 2 we are significantly closer to release early / release often. The content of the vote would be something like Vote: Allow individual modules to be released based on source content in SVN as indicated by the release manager and use Maven to produce and release the appropriate artifacts. I'd like to leave this open for discussion through next Friday and bring this up for a vote first week in January. This discussion thread is the place to get ideas on the table. I suspect that this will not garner 100% support so this vote will be a majority consensus. Remember, we can always change things later if they aren't working out. If you like the proposal as is just simply give a +1 so people can know your opinion. Its not a vote in this thread but simply a quick determination of opinion. For those who haven't released before, this will really simplify the process and make it less tedious and allow more people to be involved. Matt Hogstrom [EMAIL PROTECTED] -- Jacek Laskowski http://www.jaceklaskowski.pl
Re: [DISCUSS] Release process change
I like David's idea and I think it's really the right way to go about this process. Jeff Matt Hogstrom wrote: > David brought up a proposed change to the release process around some > specs that he is currently working through. As you all know the release > process has been a challenge for quite some time. Branching, building, > voting, building, voting, building voting, etc. Which is quite time > consuming. It may be that we've gone too far in the release process in > terms of how were structured. We need a better way that ensures we are > releasing good binaries in terms of code quality as well as meeting the > legal requirements. I think David's suggestion is a step in the right > direction. > > We elect someone to be the release manager which in essence empowers > them to follow through the process of getting software out the door. In > David's proposal (currently for specs and this is a great starting > point) is to propose a release and use Maven to release:prepare / > release:perform to create the final binaries and place them into the > Maven repo. > > In essence, the release manager (or whoever is working to release some > component) indicates their desire and points to the branch to be > released. This effectively defines the content of the release. Folks > on the project would then build from that branch and satisfy themselves > that they are satisfied with the content and vote. The release manager > would then release using the Maven plugin and be responsible for the > actual distribution. > > Based on the way we can release with Maven 2 we are significantly closer > to release early / release often. > > The content of the vote would be something like > > Vote: Allow individual modules to be released based on source content > in SVN as indicated by the release manager and use Maven to produce and > release the appropriate artifacts. > > I'd like to leave this open for discussion through next Friday and bring > this up for a vote first week in January. This discussion thread is the > place to get ideas on the table. I suspect that this will not garner > 100% support so this vote will be a majority consensus. Remember, we > can always change things later if they aren't working out. > > If you like the proposal as is just simply give a +1 so people can know > your opinion. Its not a vote in this thread but simply a quick > determination of opinion. > > For those who haven't released before, this will really simplify the > process and make it less tedious and allow more people to be involved. > > Matt Hogstrom > [EMAIL PROTECTED] >
