Re: fixing an almost Maven project
It sounds like this would be a good solutions for us, until we fully move towards the correct versioning, described by Paul B. and you in the other mail. Thank You all! Mark. On 23 July 2015 at 21:44, Barrie Treloar baerr...@gmail.com wrote: On 24 July 2015 at 13:33, V. Mark Lehky mark.le...@gmail.com wrote: Our release process is definitely: everything gets released at the same time (with the same version number). How do you create per-project .m2 repo via Jenkins? Is it just a case of deleting the repo before each build, or are there actually separate copies of repos on disk? I won't claim that doing this is best practice. You can't just delete the repo before each build, your machine has multiple build threads right? So that would cause things to break. There is an option in the jenkins config under Build Use private Maven repository Help for feature: Use private Maven repository Normally, Jenkins uses the local Maven repository as determined by Maven — the exact process seems to be undocumented, but it's ~/.m2/repository and can be overridden by localRepository in ~/.m2/settings.xml (see the reference for more details.) This normally means that all the jobs that are executed on the same node shares a single Maven repository. The upside of this is that you can save the disk space, but the downside of this is that sometimes those builds could interfere with each other. For example, you might end up having builds incorrectly succeed, just because your have all the dependencies in your local repository, despite that fact that none of the repositories in POM might have them. There are also some reported problems regarding having concurrent Maven processes trying to use the same local repository. When this option is checked, Jenkins will tell Maven to use $WORKSPACE/.repository as the local Maven repository. This means each job will get its own isolated Maven repository just for itself. It fixes the above problems, at the expense of additional disk space consumption. When using this option, consider setting up a Maven artifact manager so that you don't have to hit remote Maven repositories too often. If you'd prefer to activate this mode in all the Maven jobs executed on Jenkins, refer to the technique described here. --- Note: we use the evil Maven build job. Read http://javaadventure.blogspot.com.au/2013/11/jenkins-maven-job-type-considered-evil.html for more on this. We got everything working - and haven't spent much effort refactoring this stuff to best practices. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: fixing an almost Maven project
Thanx for the vote of confidence. Good to know that I am on the right path. And yes: mailing this list, I am all in drinking the kool-aid. ;) On 23 July 2015 at 21:35, Barrie Treloar baerr...@gmail.com wrote: On 24 July 2015 at 13:39, V. Mark Lehky mark.le...@gmail.com wrote: I should have added: Before I came along, everything got build as jar-with-dependencies. Test code did not live in src/test, but in src/main; and was, obviously, not run with 'mvn test', but with 'java -cp ...jar-with-dependencies.jar org.junit.runner.JUnitCore ...AllTests'. All of which I thought was rather brute. Bloatware, but it did work. Or was I wrong? If your mailing to the Maven User list, be prepared to drink the kool-aid :) I will say that what you have done has moved your project onto The Right Path(tm). Keep going, you'll get there. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: fixing an almost Maven project
On 24 July 2015 at 12:18, Paul Benedict pbened...@apache.org wrote: It sounds like both your projects are snapshots. So when you build B-SNAPSHOT you have no idea what's inside of A-SNAPSHOT. If this is bothersome to you, you can think about releasing milestone versions of A so that B-SNAPSHOT always has a known/reliable codebase to work with. Perhaps you want weekly snapshots to be released, for example, but you definitely want to lock down A's version for B. Alternatively you can have Jenkin's use a per-project ~/.m2/repository. Before each build we delete the per-project .m2/repo so that previous builds don't pollute it. This does mean that each build requires every released version to be seeded into your .m2/repo - but this is retrieved from a network local repository server, so its mostly fast enough. Now you can build your jenkins projects for A and B knowing that they have their own .m2/repo and will build everything from source control. However Paul's comment would be a better approach. You should also think closely about release lifecylces, as the way you have it setup now you will need to release project A and B at the same time, as child projects should be released together with the parent. I think the advise for git is to make the relationship 1 git repo : 1 release cycle. That is, dont put A and B in the same git repo if you indent to release A and B on different cycles.
Re: fixing an almost Maven project
So you're suggesting controlling this via version numbers? Have projectB depend on projectA:1.0.1-SNAP, while projectA is actively being worked on the next version, say 1.0.2-SNAP? Or something like that? On 07/23/2015 07:48 PM, Paul Benedict wrote: It sounds like both your projects are snapshots. So when you build B-SNAPSHOT you have no idea what's inside of A-SNAPSHOT. If this is bothersome to you, you can think about releasing milestone versions of A so that B-SNAPSHOT always has a known/reliable codebase to work with. Perhaps you want weekly snapshots to be released, for example, but you definitely want to lock down A's version for B. Cheers, Paul On Thu, Jul 23, 2015 at 9:30 PM, V. Mark Lehky mark.le...@gmail.com wrote: Hello. I have taken it upon myself to fix our Maven project. :) Our current situation is that we have about dozen child projects, all tied together with one parent pom. Everything lives in git (Bitbucket) and is build by Jenkins. There are several (ugly?) python scripts that package and deploy the finished product. Further some of the tests need to be build into a jar and run from command line. Specifically there are two projects, projectB is dependent on projectA, and both of these have a lot of activity right now. The team decided to create one branch in git for work on projectA and a separate branch for work on projectB; let us call them branchA and branchB. Jenkins builds both of these branches with 'mvn clean install'. Subsequently tests are run separately with 'mvn -f projectB/pom.xml test'. The problem is: during test, I do not know which jars I am getting from the local m2 repo. Occasionally branchA is build and jars are pushed to the local repo. Then branchB test phase runs, but grabs jars from the local repo (branchA) which have different code. Would appreciate some advice how to move forward so as to minimize or outright eliminate grabbing the wrong jars from the local repo during the test phase. TIA for any advice. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: fixing an almost Maven project
I should have added: Before I came along, everything got build as jar-with-dependencies. Test code did not live in src/test, but in src/main; and was, obviously, not run with 'mvn test', but with 'java -cp ...jar-with-dependencies.jar org.junit.runner.JUnitCore ...AllTests'. All of which I thought was rather brute. Bloatware, but it did work. Or was I wrong? On 07/23/2015 07:30 PM, V. Mark Lehky wrote: Hello. I have taken it upon myself to fix our Maven project. :) Our current situation is that we have about dozen child projects, all tied together with one parent pom. Everything lives in git (Bitbucket) and is build by Jenkins. There are several (ugly?) python scripts that package and deploy the finished product. Further some of the tests need to be build into a jar and run from command line. Specifically there are two projects, projectB is dependent on projectA, and both of these have a lot of activity right now. The team decided to create one branch in git for work on projectA and a separate branch for work on projectB; let us call them branchA and branchB. Jenkins builds both of these branches with 'mvn clean install'. Subsequently tests are run separately with 'mvn -f projectB/pom.xml test'. The problem is: during test, I do not know which jars I am getting from the local m2 repo. Occasionally branchA is build and jars are pushed to the local repo. Then branchB test phase runs, but grabs jars from the local repo (branchA) which have different code. Would appreciate some advice how to move forward so as to minimize or outright eliminate grabbing the wrong jars from the local repo during the test phase. TIA for any advice. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: fixing an almost Maven project
On 24 July 2015 at 13:33, V. Mark Lehky mark.le...@gmail.com wrote: Our release process is definitely: everything gets released at the same time (with the same version number). How do you create per-project .m2 repo via Jenkins? Is it just a case of deleting the repo before each build, or are there actually separate copies of repos on disk? I won't claim that doing this is best practice. You can't just delete the repo before each build, your machine has multiple build threads right? So that would cause things to break. There is an option in the jenkins config under Build Use private Maven repository Help for feature: Use private Maven repository Normally, Jenkins uses the local Maven repository as determined by Maven — the exact process seems to be undocumented, but it's ~/.m2/repository and can be overridden by localRepository in ~/.m2/settings.xml (see the reference for more details.) This normally means that all the jobs that are executed on the same node shares a single Maven repository. The upside of this is that you can save the disk space, but the downside of this is that sometimes those builds could interfere with each other. For example, you might end up having builds incorrectly succeed, just because your have all the dependencies in your local repository, despite that fact that none of the repositories in POM might have them. There are also some reported problems regarding having concurrent Maven processes trying to use the same local repository. When this option is checked, Jenkins will tell Maven to use $WORKSPACE/.repository as the local Maven repository. This means each job will get its own isolated Maven repository just for itself. It fixes the above problems, at the expense of additional disk space consumption. When using this option, consider setting up a Maven artifact manager so that you don't have to hit remote Maven repositories too often. If you'd prefer to activate this mode in all the Maven jobs executed on Jenkins, refer to the technique described here. --- Note: we use the evil Maven build job. Read http://javaadventure.blogspot.com.au/2013/11/jenkins-maven-job-type-considered-evil.html for more on this. We got everything working - and haven't spent much effort refactoring this stuff to best practices.
Re: fixing an almost Maven project
Our release process is definitely: everything gets released at the same time (with the same version number). How do you create per-project .m2 repo via Jenkins? Is it just a case of deleting the repo before each build, or are there actually separate copies of repos on disk? On 07/23/2015 08:19 PM, Barrie Treloar wrote: On 24 July 2015 at 12:18, Paul Benedict pbened...@apache.org wrote: It sounds like both your projects are snapshots. So when you build B-SNAPSHOT you have no idea what's inside of A-SNAPSHOT. If this is bothersome to you, you can think about releasing milestone versions of A so that B-SNAPSHOT always has a known/reliable codebase to work with. Perhaps you want weekly snapshots to be released, for example, but you definitely want to lock down A's version for B. Alternatively you can have Jenkin's use a per-project ~/.m2/repository. Before each build we delete the per-project .m2/repo so that previous builds don't pollute it. This does mean that each build requires every released version to be seeded into your .m2/repo - but this is retrieved from a network local repository server, so its mostly fast enough. Now you can build your jenkins projects for A and B knowing that they have their own .m2/repo and will build everything from source control. However Paul's comment would be a better approach. You should also think closely about release lifecylces, as the way you have it setup now you will need to release project A and B at the same time, as child projects should be released together with the parent. I think the advise for git is to make the relationship 1 git repo : 1 release cycle. That is, dont put A and B in the same git repo if you indent to release A and B on different cycles. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
fixing an almost Maven project
Hello. I have taken it upon myself to fix our Maven project. :) Our current situation is that we have about dozen child projects, all tied together with one parent pom. Everything lives in git (Bitbucket) and is build by Jenkins. There are several (ugly?) python scripts that package and deploy the finished product. Further some of the tests need to be build into a jar and run from command line. Specifically there are two projects, projectB is dependent on projectA, and both of these have a lot of activity right now. The team decided to create one branch in git for work on projectA and a separate branch for work on projectB; let us call them branchA and branchB. Jenkins builds both of these branches with 'mvn clean install'. Subsequently tests are run separately with 'mvn -f projectB/pom.xml test'. The problem is: during test, I do not know which jars I am getting from the local m2 repo. Occasionally branchA is build and jars are pushed to the local repo. Then branchB test phase runs, but grabs jars from the local repo (branchA) which have different code. Would appreciate some advice how to move forward so as to minimize or outright eliminate grabbing the wrong jars from the local repo during the test phase. TIA for any advice. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: fixing an almost Maven project
It sounds like both your projects are snapshots. So when you build B-SNAPSHOT you have no idea what's inside of A-SNAPSHOT. If this is bothersome to you, you can think about releasing milestone versions of A so that B-SNAPSHOT always has a known/reliable codebase to work with. Perhaps you want weekly snapshots to be released, for example, but you definitely want to lock down A's version for B. Cheers, Paul On Thu, Jul 23, 2015 at 9:30 PM, V. Mark Lehky mark.le...@gmail.com wrote: Hello. I have taken it upon myself to fix our Maven project. :) Our current situation is that we have about dozen child projects, all tied together with one parent pom. Everything lives in git (Bitbucket) and is build by Jenkins. There are several (ugly?) python scripts that package and deploy the finished product. Further some of the tests need to be build into a jar and run from command line. Specifically there are two projects, projectB is dependent on projectA, and both of these have a lot of activity right now. The team decided to create one branch in git for work on projectA and a separate branch for work on projectB; let us call them branchA and branchB. Jenkins builds both of these branches with 'mvn clean install'. Subsequently tests are run separately with 'mvn -f projectB/pom.xml test'. The problem is: during test, I do not know which jars I am getting from the local m2 repo. Occasionally branchA is build and jars are pushed to the local repo. Then branchB test phase runs, but grabs jars from the local repo (branchA) which have different code. Would appreciate some advice how to move forward so as to minimize or outright eliminate grabbing the wrong jars from the local repo during the test phase. TIA for any advice. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: fixing an almost Maven project
On 24 July 2015 at 13:39, V. Mark Lehky mark.le...@gmail.com wrote: I should have added: Before I came along, everything got build as jar-with-dependencies. Test code did not live in src/test, but in src/main; and was, obviously, not run with 'mvn test', but with 'java -cp ...jar-with-dependencies.jar org.junit.runner.JUnitCore ...AllTests'. All of which I thought was rather brute. Bloatware, but it did work. Or was I wrong? If your mailing to the Maven User list, be prepared to drink the kool-aid :) I will say that what you have done has moved your project onto The Right Path(tm). Keep going, you'll get there.
Re: fixing an almost Maven project
On 24 July 2015 at 13:30, V. Mark Lehky mark.le...@gmail.com wrote: So you're suggesting controlling this via version numbers? Have projectB depend on projectA:1.0.1-SNAP, while projectA is actively being worked on the next version, say 1.0.2-SNAP? Or something like that? On 07/23/2015 07:48 PM, Paul Benedict wrote: It sounds like both your projects are snapshots. So when you build B-SNAPSHOT you have no idea what's inside of A-SNAPSHOT. If this is bothersome to you, you can think about releasing milestone versions of A so that B-SNAPSHOT always has a known/reliable codebase to work with. Perhaps you want weekly snapshots to be released, for example, but you definitely want to lock down A's version for B. He is saying: projectB depends on projectA:1.0.1 (Note the lack of -SNAPSHOT). and projectA dependon on projectB:1.0.2 (again no snapshot). So at regular, and determined intervals, you merge your work back onto HEAD. When you do, cut a release - they are free and cheap, you can just create as many as you need. Then your branch projects can decide when they want to move to the point releases. That way they are isolated from each other, have a known release state, and can easily switch back to previous versions when a release is deficient. You won't be able to do this with the SNAPSHOT approach.
