Re: An idea I had for one way of doing Continuous Delivery with Maven
Of course, we tag the MANIFEST of all jars with the git commit. We actually go a bit farther and print out all dependency git SHA1s during the build process, so you only have to refer to the jenkins build log to do a bisect and not dig through jars. I describe how we did that here: http://liveramp.com/engineering/identifying-maven-snapshot-artifacts-by-git-revision/ On Thu, May 5, 2016 at 6:32 PM, Barrie Treloarwrote: > On 6 May 2016 at 09:48, Ben Podgursky wrote: > > > I see these discussions often, and I wanted to jump in and mention how we > > handle continuous deploys, because I feel that it avoids many of the > > downsides mentioned here, albeit with some (IMO) minor costs: > > > > - all of our internal dependencies are SNAPSHOT. we do not do releases, > > ever, or ever depend on a fixed version > > > > - we attach to the package phase of our builds an assembly which packages > > the artifact and all of the dependencies it tested against into a tar.gz > > > > - we deploy the tar.gz to our artifact server with a snapshot version > (ex, > > MY_PROJECT-1.0-deploy.tar.gz) > > > > - when deploying we simply take the last snapshot deploy artifact and > unzip > > it in production (+ some config files etc). > > > > so while you use SNAPSHOT versions, you are guaranteed that you are > > deploying against tested dependencies. and you never have to commit > > updated dependency versions, do releases, etc. the only real cost is a > bit > > of artifact server disk space… but disk space is incredibly cheap. > > > You left out tagging the source code in the built jars. > > How are you supposed to pull up the code in production? > Or git bisect, or diff between versions to track down when a defect was > introduced? >
Re: An idea I had for one way of doing Continuous Delivery with Maven
On 6 May 2016 at 09:48, Ben Podgurskywrote: > I see these discussions often, and I wanted to jump in and mention how we > handle continuous deploys, because I feel that it avoids many of the > downsides mentioned here, albeit with some (IMO) minor costs: > > - all of our internal dependencies are SNAPSHOT. we do not do releases, > ever, or ever depend on a fixed version > > - we attach to the package phase of our builds an assembly which packages > the artifact and all of the dependencies it tested against into a tar.gz > > - we deploy the tar.gz to our artifact server with a snapshot version (ex, > MY_PROJECT-1.0-deploy.tar.gz) > > - when deploying we simply take the last snapshot deploy artifact and unzip > it in production (+ some config files etc). > > so while you use SNAPSHOT versions, you are guaranteed that you are > deploying against tested dependencies. and you never have to commit > updated dependency versions, do releases, etc. the only real cost is a bit > of artifact server disk space… but disk space is incredibly cheap. You left out tagging the source code in the built jars. How are you supposed to pull up the code in production? Or git bisect, or diff between versions to track down when a defect was introduced?
Re: An idea I had for one way of doing Continuous Delivery with Maven
sorry, meant " MY_PROJECT-1.0-SNAPSHOT-deploy.tar.gz" On Thu, May 5, 2016 at 5:18 PM, Ben Podgurskywrote: > I see these discussions often, and I wanted to jump in and mention how we > handle continuous deploys, because I feel that it avoids many of the > downsides mentioned here, albeit with some (IMO) minor costs: > > - all of our internal dependencies are SNAPSHOT. we do not do releases, > ever, or ever depend on a fixed version > > - we attach to the package phase of our builds an assembly which packages > the artifact and all of the dependencies it tested against into a tar.gz > > - we deploy the tar.gz to our artifact server with a snapshot version (ex, > MY_PROJECT-1.0-deploy.tar.gz) > > - when deploying we simply take the last snapshot deploy artifact and > unzip it in production (+ some config files etc). > > so while you use SNAPSHOT versions, you are guaranteed that you are > deploying against tested dependencies. and you never have to commit > updated dependency versions, do releases, etc. the only real cost is a bit > of artifact server disk space… but disk space is incredibly cheap. > > On Thu, May 5, 2016 at 2:13 AM, Stephen Connolly < > stephen.alan.conno...@gmail.com> wrote: > >> On 5 May 2016 at 10:00, Robert Scholte wrote: >> >> > Hi Stephen, nice blog. >> > >> > There are a few things I'd like to share: >> > - The maven-release-plugin does several checks which ensures that you >> have >> > a reproducible artifact, where checking on SNAPSHOT dependencies is one >> of >> > the most important ones. (you can find a lot patterns based on the >> > versions-maven-plugin on the web, but they forget these checks) >> > >> >> This is one of the reasons why my suggestion is to actually cut a release >> for every commit (just only push the tag - and just the tag - if the >> release goes anywhere) >> >> >> > - I still have this wish to have support a couple of strategies within >> the >> > maven-release-plugin, this could be one as well. >> > >> >> Yes, I think it would be a valuable one to have in the default plugin, for >> sure... and it may need some assistance from the scm plugin too >> >> >> > >> > In a continuous environment is becomes more and more important to be >> able >> > to reproduce issues, which can be hard when things keep changing. This >> > might even work with SNAPSHOT dependencies if your can get the >> > distributable (either the deployable or executable artifact, assuming >> they >> > contain all the required dependencies) and have a testing setup which >> can >> > test with this artifact. If you can reproduce it here, you can >> (re)write it >> > as an appropriate test and fix it. >> > So next to all the current tests you need to do regression tests on the >> > distributable as well. >> > As long as this is not possible or secured in your pipeline, you should >> > NEVER EVER want to depend on SNAPSHOT-dependencies. >> > >> >> +1 >> >> One that we gain with the approach I suggest is that we can actually use >> versions:resolve-ranges as we know that the pom will not be pushed back to >> master. >> >> Thus developers can keep version ranges in the pom and the CI build will >> resolve the ranges for each "release" >> >> I'd like to find a way to allow resolve-ranges to work with the current >> release plugin flow... my working idea has been to inject e.g. XML PI >> entities into the pom so that we know what range to restore, e.g. >> pre-release pom has >> >> >> ... >> [1.2,2.0) >> >> >> which would get resolved into the release pom as >> >> >> ... >> 1.5 >> >> >> so that when we want to un-pin we can then see the original range and >> up-lift it so that the development pom becomes >> >> >> ... >> [1.5,2.0) >> >> >> >> >> > >> > thanks, >> > Robert >> > >> > >> > On Wed, 04 May 2016 21:20:01 +0200, Stephen Connolly < >> > stephen.alan.conno...@gmail.com> wrote: >> > >> > Here's an embryonic idea I had for doing Continuous Delivery with Maven >> >> >> >> >> >> >> https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline >> >> >> >> Probably needs a little fleshing out. >> >> >> >> -Stephen >> >> >> > >> > - >> > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org >> > For additional commands, e-mail: users-h...@maven.apache.org >> > >> > >> > >
Re: An idea I had for one way of doing Continuous Delivery with Maven
I see these discussions often, and I wanted to jump in and mention how we handle continuous deploys, because I feel that it avoids many of the downsides mentioned here, albeit with some (IMO) minor costs: - all of our internal dependencies are SNAPSHOT. we do not do releases, ever, or ever depend on a fixed version - we attach to the package phase of our builds an assembly which packages the artifact and all of the dependencies it tested against into a tar.gz - we deploy the tar.gz to our artifact server with a snapshot version (ex, MY_PROJECT-1.0-deploy.tar.gz) - when deploying we simply take the last snapshot deploy artifact and unzip it in production (+ some config files etc). so while you use SNAPSHOT versions, you are guaranteed that you are deploying against tested dependencies. and you never have to commit updated dependency versions, do releases, etc. the only real cost is a bit of artifact server disk space… but disk space is incredibly cheap. On Thu, May 5, 2016 at 2:13 AM, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > On 5 May 2016 at 10:00, Robert Scholtewrote: > > > Hi Stephen, nice blog. > > > > There are a few things I'd like to share: > > - The maven-release-plugin does several checks which ensures that you > have > > a reproducible artifact, where checking on SNAPSHOT dependencies is one > of > > the most important ones. (you can find a lot patterns based on the > > versions-maven-plugin on the web, but they forget these checks) > > > > This is one of the reasons why my suggestion is to actually cut a release > for every commit (just only push the tag - and just the tag - if the > release goes anywhere) > > > > - I still have this wish to have support a couple of strategies within > the > > maven-release-plugin, this could be one as well. > > > > Yes, I think it would be a valuable one to have in the default plugin, for > sure... and it may need some assistance from the scm plugin too > > > > > > In a continuous environment is becomes more and more important to be able > > to reproduce issues, which can be hard when things keep changing. This > > might even work with SNAPSHOT dependencies if your can get the > > distributable (either the deployable or executable artifact, assuming > they > > contain all the required dependencies) and have a testing setup which can > > test with this artifact. If you can reproduce it here, you can (re)write > it > > as an appropriate test and fix it. > > So next to all the current tests you need to do regression tests on the > > distributable as well. > > As long as this is not possible or secured in your pipeline, you should > > NEVER EVER want to depend on SNAPSHOT-dependencies. > > > > +1 > > One that we gain with the approach I suggest is that we can actually use > versions:resolve-ranges as we know that the pom will not be pushed back to > master. > > Thus developers can keep version ranges in the pom and the CI build will > resolve the ranges for each "release" > > I'd like to find a way to allow resolve-ranges to work with the current > release plugin flow... my working idea has been to inject e.g. XML PI > entities into the pom so that we know what range to restore, e.g. > pre-release pom has > > > ... > [1.2,2.0) > > > which would get resolved into the release pom as > > > ... > 1.5 > > > so that when we want to un-pin we can then see the original range and > up-lift it so that the development pom becomes > > > ... > [1.5,2.0) > > > > > > > > thanks, > > Robert > > > > > > On Wed, 04 May 2016 21:20:01 +0200, Stephen Connolly < > > stephen.alan.conno...@gmail.com> wrote: > > > > Here's an embryonic idea I had for doing Continuous Delivery with Maven > >> > >> > >> > https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline > >> > >> Probably needs a little fleshing out. > >> > >> -Stephen > >> > > > > - > > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > > For additional commands, e-mail: users-h...@maven.apache.org > > > > >
Re: An idea I had for one way of doing Continuous Delivery with Maven
On 5 May 2016 at 10:00, Robert Scholtewrote: > Hi Stephen, nice blog. > > There are a few things I'd like to share: > - The maven-release-plugin does several checks which ensures that you have > a reproducible artifact, where checking on SNAPSHOT dependencies is one of > the most important ones. (you can find a lot patterns based on the > versions-maven-plugin on the web, but they forget these checks) > This is one of the reasons why my suggestion is to actually cut a release for every commit (just only push the tag - and just the tag - if the release goes anywhere) > - I still have this wish to have support a couple of strategies within the > maven-release-plugin, this could be one as well. > Yes, I think it would be a valuable one to have in the default plugin, for sure... and it may need some assistance from the scm plugin too > > In a continuous environment is becomes more and more important to be able > to reproduce issues, which can be hard when things keep changing. This > might even work with SNAPSHOT dependencies if your can get the > distributable (either the deployable or executable artifact, assuming they > contain all the required dependencies) and have a testing setup which can > test with this artifact. If you can reproduce it here, you can (re)write it > as an appropriate test and fix it. > So next to all the current tests you need to do regression tests on the > distributable as well. > As long as this is not possible or secured in your pipeline, you should > NEVER EVER want to depend on SNAPSHOT-dependencies. > +1 One that we gain with the approach I suggest is that we can actually use versions:resolve-ranges as we know that the pom will not be pushed back to master. Thus developers can keep version ranges in the pom and the CI build will resolve the ranges for each "release" I'd like to find a way to allow resolve-ranges to work with the current release plugin flow... my working idea has been to inject e.g. XML PI entities into the pom so that we know what range to restore, e.g. pre-release pom has ... [1.2,2.0) which would get resolved into the release pom as ... 1.5 so that when we want to un-pin we can then see the original range and up-lift it so that the development pom becomes ... [1.5,2.0) > > thanks, > Robert > > > On Wed, 04 May 2016 21:20:01 +0200, Stephen Connolly < > stephen.alan.conno...@gmail.com> wrote: > > Here's an embryonic idea I had for doing Continuous Delivery with Maven >> >> >> https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline >> >> Probably needs a little fleshing out. >> >> -Stephen >> > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > >
Re: An idea I had for one way of doing Continuous Delivery with Maven
Hi Stephen, nice blog. There are a few things I'd like to share: - The maven-release-plugin does several checks which ensures that you have a reproducible artifact, where checking on SNAPSHOT dependencies is one of the most important ones. (you can find a lot patterns based on the versions-maven-plugin on the web, but they forget these checks) - I still have this wish to have support a couple of strategies within the maven-release-plugin, this could be one as well. In a continuous environment is becomes more and more important to be able to reproduce issues, which can be hard when things keep changing. This might even work with SNAPSHOT dependencies if your can get the distributable (either the deployable or executable artifact, assuming they contain all the required dependencies) and have a testing setup which can test with this artifact. If you can reproduce it here, you can (re)write it as an appropriate test and fix it. So next to all the current tests you need to do regression tests on the distributable as well. As long as this is not possible or secured in your pipeline, you should NEVER EVER want to depend on SNAPSHOT-dependencies. thanks, Robert On Wed, 04 May 2016 21:20:01 +0200, Stephen Connollywrote: Here's an embryonic idea I had for doing Continuous Delivery with Maven https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline Probably needs a little fleshing out. -Stephen - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: An idea I had for one way of doing Continuous Delivery with Maven
Very promising, may need to do some scm provider work to support other SCM like Perforce? Thanks -D On Wed, May 4, 2016 at 12:20 PM, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > Here's an embryonic idea I had for doing Continuous Delivery with Maven > > > https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline > > Probably needs a little fleshing out. > > -Stephen >
An idea I had for one way of doing Continuous Delivery with Maven
Here's an embryonic idea I had for doing Continuous Delivery with Maven https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline Probably needs a little fleshing out. -Stephen