Regression: Reactor memory consumption/leak
Hi folks, we have a single build with currently ~400 projects (incl. builders i.e. POMs having modules only). We are already used to increase the provided memory in MAVEN_OPTS, but lately we have troubles to build at all because of OOMEs (heap). Look at following numbers building with the different Maven versions (latest Oracle JDK 7 on 64-bit Linux): == % === $ MAVEN_OPTS=-Xmx640m mvn-3.0.5 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 25.392s [INFO] Finished at: Thu Oct 16 10:02:56 CEST 2014 [INFO] Final Memory: 276M/511M [INFO] - == % === 3.0.5 fails with 512 only. == % === MAVEN_OPTS=-Xmx1100m mvn-3.1.1 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 37.004s [INFO] Finished at: Thu Oct 16 10:13:36 CEST 2014 [INFO] Final Memory: 431M/978M [INFO] - == % === 3.1.1 fails with 1024m only. == % === MAVEN_OPTS=-Xmx2500m mvn-3.2.3 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 55.465 s [INFO] Finished at: 2014-10-16T10:27:42+02:00 [INFO] Final Memory: 1488M/M [INFO] - == % === 3.2.3 fails with 2400m only. The required memory increases dramatically from Maven version to version just to calculate the build order list (the OOME occurs before this list is printed). - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Regression: Reactor memory consumption/leak
Igor and I have been moving a vast project with hundreds of modules (200 to 400 to 500) from 3.1.0 through 3.2.4-SNAPSHOT and don't observe this drastic change. We would definitely notice. Are all your plugin versions locked down in that they don't vary even though the version of Maven does? Also are you running reporting in here? On Oct 16, 2014, at 5:12 AM, Jörg Schaible joerg.schai...@swisspost.com wrote: Hi folks, we have a single build with currently ~400 projects (incl. builders i.e. POMs having modules only). We are already used to increase the provided memory in MAVEN_OPTS, but lately we have troubles to build at all because of OOMEs (heap). Look at following numbers building with the different Maven versions (latest Oracle JDK 7 on 64-bit Linux): == % === $ MAVEN_OPTS=-Xmx640m mvn-3.0.5 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 25.392s [INFO] Finished at: Thu Oct 16 10:02:56 CEST 2014 [INFO] Final Memory: 276M/511M [INFO] - == % === 3.0.5 fails with 512 only. == % === MAVEN_OPTS=-Xmx1100m mvn-3.1.1 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 37.004s [INFO] Finished at: Thu Oct 16 10:13:36 CEST 2014 [INFO] Final Memory: 431M/978M [INFO] - == % === 3.1.1 fails with 1024m only. == % === MAVEN_OPTS=-Xmx2500m mvn-3.2.3 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 55.465 s [INFO] Finished at: 2014-10-16T10:27:42+02:00 [INFO] Final Memory: 1488M/M [INFO] - == % === 3.2.3 fails with 2400m only. The required memory increases dramatically from Maven version to version just to calculate the build order list (the OOME occurs before this list is printed). - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org Thanks, Jason -- Jason van Zyl Founder, Apache Maven http://twitter.com/jvanzyl http://twitter.com/takari_io - There's no sense in being precise when you don't even know what you're talking about. -- John von Neumann
Re: Regression: Reactor memory consumption/leak
Can you provide an example project we can use to reproduce the problem locally? You may be able to strip your source tree from everything bun pom.xml files, for example. -- Regards, Igor On 2014-10-16, 5:12, Jörg Schaible wrote: Hi folks, we have a single build with currently ~400 projects (incl. builders i.e. POMs having modules only). We are already used to increase the provided memory in MAVEN_OPTS, but lately we have troubles to build at all because of OOMEs (heap). Look at following numbers building with the different Maven versions (latest Oracle JDK 7 on 64-bit Linux): == % === $ MAVEN_OPTS=-Xmx640m mvn-3.0.5 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 25.392s [INFO] Finished at: Thu Oct 16 10:02:56 CEST 2014 [INFO] Final Memory: 276M/511M [INFO] - == % === 3.0.5 fails with 512 only. == % === MAVEN_OPTS=-Xmx1100m mvn-3.1.1 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 37.004s [INFO] Finished at: Thu Oct 16 10:13:36 CEST 2014 [INFO] Final Memory: 431M/978M [INFO] - == % === 3.1.1 fails with 1024m only. == % === MAVEN_OPTS=-Xmx2500m mvn-3.2.3 validate ... [INFO] - [INFO] BUILD SUCCESS [INFO] - [INFO] Total time: 55.465 s [INFO] Finished at: 2014-10-16T10:27:42+02:00 [INFO] Final Memory: 1488M/M [INFO] - == % === 3.2.3 fails with 2400m only. The required memory increases dramatically from Maven version to version just to calculate the build order list (the OOME occurs before this list is printed). - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Regression: Reactor memory consumption/leak
Hi Jason, Jason van Zyl wrote: Igor and I have been moving a vast project with hundreds of modules (200 to 400 to 500) from 3.1.0 through 3.2.4-SNAPSHOT and don't observe this drastic change. We would definitely notice. Are all your plugin versions locked down in that they don't vary even though the version of Maven does? Except eclipse and help plugin - yes. Also are you running reporting in here? In validate? No. However, it does not reach this point at all: == % == $ mvn-3.2.3 validate [INFO] Scanning for projects... Downloading: http://es3.elsag.de/~maven/repo-m2-snapshot/com/scalaris/buildsystem/maven2/master/x-SNAPSHOT/maven-metadata.xml Downloaded: http://es3.elsag.de/~maven/repo-m2-snapshot/com/scalaris/buildsystem/maven2/master/x-SNAPSHOT/maven-metadata.xml (813 B at 10.7 KB/sec) Downloading: http://es3.elsag.de/~maven/repo-m2-snapshot/ip/ip-master/5.0.x-SNAPSHOT/maven-metadata.xml Downloading: http://es3.elsag.de/~maven/repo-m2-snapshot/ip/ip-master/4.4.x-SNAPSHOT/maven-metadata.xml Downloading: http://es3.elsag.de/~maven/repo-m2-snapshot/com/essencio/components/mobile/master/5.0.x-SNAPSHOT/maven-metadata.xml [ERROR] GC overhead limit exceeded - [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError == % === - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Regression: Reactor memory consumption/leak
Hi Igor, Igor Fedorenko wrote: Can you provide an example project we can use to reproduce the problem locally? You may be able to strip your source tree from everything bun pom.xml files, for example. Interesting idea. I'll try if this works out. - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Regression: Reactor memory consumption/leak
You could take some heap dumps and share them with me on google disk. A baseline from the best one and one from the worst one. It would be nice if you could get them at approx the same time in the build Kristian 2014-10-16 15:20 GMT+02:00 Jörg Schaible joerg.schai...@swisspost.com: Hi Igor, Igor Fedorenko wrote: Can you provide an example project we can use to reproduce the problem locally? You may be able to strip your source tree from everything bun pom.xml files, for example. Interesting idea. I'll try if this works out. - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
[GitHub] maven-surefire pull request: [SUREFIRE-1095] NPE in JUnit 4.x RunL...
Github user asfgit closed the pull request at: https://github.com/apache/maven-surefire/pull/58 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Regression: Reactor memory consumption/leak
Hi Igor, Igor Fedorenko wrote: Can you provide an example project we can use to reproduce the problem locally? You may be able to strip your source tree from everything bun pom.xml files, for example. OK, this works out. I have now such a transportable setup. It's zipped about 1.1MB with an empty local repository or 38MB with a filled one. I can run MAVEN_OPTS=-Xmx3000m mvn validate -s settings.xml successfully and will fail with OOME calling MAVEN_OPTS=-Xmx2000m mvn validate -s settings.xml using M323/Java7 on Linux x64. The project tree contains the pom.xml and anything that triggers profiles automatically. Any non-public artifact is in a remote repo (included in the 1.1MB), so you should be able to reproduce this. Which do you want and what's the most useful place to put this? Cheers, Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Regression: Reactor memory consumption/leak
You can zip and email it to me directly or share it on github, dropbox or google drive and send me the link. I am flexible :-) -- Regards, Igor On 2014-10-16, 11:41, Jörg Schaible wrote: Hi Igor, Igor Fedorenko wrote: Can you provide an example project we can use to reproduce the problem locally? You may be able to strip your source tree from everything bun pom.xml files, for example. OK, this works out. I have now such a transportable setup. It's zipped about 1.1MB with an empty local repository or 38MB with a filled one. I can run MAVEN_OPTS=-Xmx3000m mvn validate -s settings.xml successfully and will fail with OOME calling MAVEN_OPTS=-Xmx2000m mvn validate -s settings.xml using M323/Java7 on Linux x64. The project tree contains the pom.xml and anything that triggers profiles automatically. Any non-public artifact is in a remote repo (included in the 1.1MB), so you should be able to reproduce this. Which do you want and what's the most useful place to put this? Cheers, Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Is it possible to have file system access in Jenkins?
Our IT tests fail in maven-surefire project. More specifically I need to access files under module surefire-integration-tests/target for analysis. Is this possible? - BR, tibor17 -- View this message in context: http://maven.40175.n5.nabble.com/Is-it-possible-to-have-file-system-access-in-Jenkins-tp5809566.html Sent from the Maven Developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Regression: Reactor memory consumption/leak
Hi Igor, Igor Fedorenko wrote: You can zip and email it to me directly or share it on github, dropbox or google drive and send me the link. I am flexible :-) I've sent the small one directly ... Cheers, Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Is it possible to have file system access in Jenkins?
You *should* be able to login with your asf credentials and then the workspace link will let you browse or download a zip of the last build workspace... (Unless Andrew has gone into even more security lock down mode than I was aware of previously) On Thursday, October 16, 2014, tibor17 tibo...@lycos.com wrote: Our IT tests fail in maven-surefire project. More specifically I need to access files under module surefire-integration-tests/target for analysis. Is this possible? - BR, tibor17 -- View this message in context: http://maven.40175.n5.nabble.com/Is-it-possible-to-have-file-system-access-in-Jenkins-tp5809566.html Sent from the Maven Developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org javascript:; For additional commands, e-mail: dev-h...@maven.apache.org javascript:; -- Sent from my phone
Re: PlexusTestCase ?
More links? Okay: https://jira.codehaus.org/browse/PLX-446 I've been in situations where I needed the combination of 2 TestCase extension (I think it was springtest + dbunit) Since you can only extend one class, you'll run into a huge challenge. You really want to avoid such situations, so if there's a need rewrite/move PlexusTestCase, let's do it good so we won't block other testing extensions. Things like Mockito and PowerMock are very easy to include when using JUnit4 Annotations. Robert Op Thu, 16 Oct 2014 00:28:38 +0200 schreef Barrie Treloar baerr...@gmail.com: On 16 October 2014 03:49, Robert Scholte rfscho...@apache.org wrote: TestCase is so 20th century... *If* you want to do it, please make a TestRule of it. See also https://jira.codehaus.org/browse/MPLUGINTESTING-28 Alright, I'll bite. Send me more links on why this is better? - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: PlexusTestCase ?
Well ... trying to make the case that you cannot use jUnit 4 because your testcase extends another is simply not correct. It might be more modern to implement a runner in true jUnit 4 style - implying I agree with the sentiment of PLX-446 - but it is an overstatement to claim it cannot be used with jUnit 4. 2014-10-16 21:11 GMT+02:00 Robert Scholte rfscho...@apache.org: More links? Okay: https://jira.codehaus.org/browse/PLX-446 I've been in situations where I needed the combination of 2 TestCase extension (I think it was springtest + dbunit) Since you can only extend one class, you'll run into a huge challenge. You really want to avoid such situations, so if there's a need rewrite/move PlexusTestCase, let's do it good so we won't block other testing extensions. Things like Mockito and PowerMock are very easy to include when using JUnit4 Annotations. Robert Op Thu, 16 Oct 2014 00:28:38 +0200 schreef Barrie Treloar baerr...@gmail.com: On 16 October 2014 03:49, Robert Scholte rfscho...@apache.org wrote: TestCase is so 20th century... *If* you want to do it, please make a TestRule of it. See also https://jira.codehaus.org/browse/MPLUGINTESTING-28 Alright, I'll bite. Send me more links on why this is better? - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org -- -- +==+ | Bästa hälsningar, | [sw. Best regards] | | Lennart Jörelid | EAI Architect Integrator | | jGuru Europe AB | Mölnlycke - Kista | | Email: l...@jguru.se | URL: www.jguru.se | Phone | (skype):jgurueurope | (intl): +46 708 507 603 | (domestic): 0708 - 507 603 +==+
Re: PlexusTestCase ?
...and even though dbUnit has a base test class for extension, you can use composition of it instead. Same for Spring Test - can configure it with annotations and use a test runner. No base class needed in either case. On Thu, Oct 16, 2014 at 3:19 PM, Lennart Jörelid lennart.jore...@gmail.com wrote: Well ... trying to make the case that you cannot use jUnit 4 because your testcase extends another is simply not correct. It might be more modern to implement a runner in true jUnit 4 style - implying I agree with the sentiment of PLX-446 - but it is an overstatement to claim it cannot be used with jUnit 4. 2014-10-16 21:11 GMT+02:00 Robert Scholte rfscho...@apache.org: More links? Okay: https://jira.codehaus.org/browse/PLX-446 I've been in situations where I needed the combination of 2 TestCase extension (I think it was springtest + dbunit) Since you can only extend one class, you'll run into a huge challenge. You really want to avoid such situations, so if there's a need rewrite/move PlexusTestCase, let's do it good so we won't block other testing extensions. Things like Mockito and PowerMock are very easy to include when using JUnit4 Annotations. Robert Op Thu, 16 Oct 2014 00:28:38 +0200 schreef Barrie Treloar baerr...@gmail.com: On 16 October 2014 03:49, Robert Scholte rfscho...@apache.org wrote: TestCase is so 20th century... *If* you want to do it, please make a TestRule of it. See also https://jira.codehaus.org/browse/MPLUGINTESTING-28 Alright, I'll bite. Send me more links on why this is better? - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org -- -- +==+ | Bästa hälsningar, | [sw. Best regards] | | Lennart Jörelid | EAI Architect Integrator | | jGuru Europe AB | Mölnlycke - Kista | | Email: l...@jguru.se | URL: www.jguru.se | Phone | (skype):jgurueurope | (intl): +46 708 507 603 | (domestic): 0708 - 507 603 +==+
[GitHub] maven-surefire pull request: [SUREFIRE-817] JUnit 4.7+ Provider an...
GitHub user Tibor17 opened a pull request: https://github.com/apache/maven-surefire/pull/59 [SUREFIRE-817] JUnit 4.7+ Provider and behavior of System Exit proved by ITs Surefire817SystemExitIT.java Test.java pom.xml You can merge this pull request into a Git repository by running: $ git pull https://github.com/Tibor17/maven-surefire s1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/maven-surefire/pull/59.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #59 commit 40b4b48a323c46e1b2ace3d5acb4add183b12d02 Author: tibordigana tibo...@lycos.com Date: 2014-10-16T21:18:51Z [SUREFIRE-817] JUnit 4.7+ Provider and System Exit proved by ITs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Is it possible to have file system access in Jenkins?
Unlike Workspace menu item in CloudBees, here in https://builds.apache.org/job/maven-surefire/1325/ I don't see such menu item. Is it older Jenkins? - BR, tibor17 -- View this message in context: http://maven.40175.n5.nabble.com/Is-it-possible-to-have-file-system-access-in-Jenkins-tp5809566p5809618.html Sent from the Maven Developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
[GitHub] maven-surefire pull request: [SUREFIRE-817] JUnit 4.7+ Provider an...
Github user asfgit closed the pull request at: https://github.com/apache/maven-surefire/pull/59 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Is it possible to have file system access in Jenkins?
Hi Stephen, On 10/16/14 9:06 PM, Stephen Connolly wrote: You *should* be able to login with your asf credentials and then the workspace link will let you browse or download a zip of the last build workspace... (Unless Andrew has gone into even more security lock down mode than I was aware of previously) It looks like he hasas i can't see the workspace link as well... Kind regards Karl Heinz Marbaise On Thursday, October 16, 2014, tibor17 tibo...@lycos.com wrote: Our IT tests fail in maven-surefire project. More specifically I need to access files under module surefire-integration-tests/target for analysis. Is this possible? - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Is it possible to have file system access in Jenkins?
It would seem Andrew has removed the Job.WORKSPACE permission. I don't know if he has any suggested alternatives On 16 October 2014 22:40, tibor17 tibo...@lycos.com wrote: Unlike Workspace menu item in CloudBees, here in https://builds.apache.org/job/maven-surefire/1325/ I don't see such menu item. Is it older Jenkins? - BR, tibor17 -- View this message in context: http://maven.40175.n5.nabble.com/Is-it-possible-to-have-file-system-access-in-Jenkins-tp5809566p5809618.html Sent from the Maven Developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Is it possible to have file system access in Jenkins?
I think we should complain about this. It makes Jenkins much much harder to work with ! 17. Okt. 2014 07:14 skrev Stephen Connolly stephen.alan.conno...@gmail.com følgende: It would seem Andrew has removed the Job.WORKSPACE permission. I don't know if he has any suggested alternatives On 16 October 2014 22:40, tibor17 tibo...@lycos.com wrote: Unlike Workspace menu item in CloudBees, here in https://builds.apache.org/job/maven-surefire/1325/ I don't see such menu item. Is it older Jenkins? - BR, tibor17 -- View this message in context: http://maven.40175.n5.nabble.com/Is-it-possible-to-have-file-system-access-in-Jenkins-tp5809566p5809618.html Sent from the Maven Developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Is it possible to have file system access in Jenkins?
/me waiting for Stephen complaining about Maven plugin :-) (weird I didn't see that yet in this thread) /me sorry but Friday afternoon joke :-) On 17 October 2014 16:39, Kristian Rosenvold kristian.rosenv...@gmail.com wrote: I think we should complain about this. It makes Jenkins much much harder to work with ! 17. Okt. 2014 07:14 skrev Stephen Connolly stephen.alan.conno...@gmail.com følgende: It would seem Andrew has removed the Job.WORKSPACE permission. I don't know if he has any suggested alternatives On 16 October 2014 22:40, tibor17 tibo...@lycos.com wrote: Unlike Workspace menu item in CloudBees, here in https://builds.apache.org/job/maven-surefire/1325/ I don't see such menu item. Is it older Jenkins? - BR, tibor17 -- View this message in context: http://maven.40175.n5.nabble.com/Is-it-possible-to-have-file-system-access-in-Jenkins-tp5809566p5809618.html Sent from the Maven Developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org -- Olivier Lamy http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
