Apparently "includes" support fairly complex syntax, such as regexes: http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#includes
I'm thinking of a clunky approach of including tests with names starting with [A-Ma-m] in one build configuration, and those with names starting with [^A-Ma-m] in another build configuration. These build configuration could have most of their actual configuration in a build template, and only specify the differences separately, which would be the regex. My organization is also using TeamCity and Maven, and is about to embark on using (again) build chains in TeamCity, which would allow to define these two (or whichever many you desire) build configurations, and pull them together somehow. We would be using this approach to split up, and parallelize unit tests vs. (module) integration tests vs. some end to end tests (cross-module integration tests). Some of these tests live in a separate Maven module already, which I'd call a best practice for cross-module integration tests. The simpler (module) integration tests live among our unit tests, and we've done a decent job at categorizing these (JUnit @Category annotation). (Actually, we have a unit test to make sure that all unit tests have said annotation.) We're using this mixed bag of approaches already to define subsequent build steps within the same build configuration, so it should be trivial to break them out into a build chain. I've burned myself in the past with build chains in TeamCity, as I didn't get the feeling that they roll up very well, if some part of the chain gets stuck (fails). But this might be a bad observation of mine, and I'll be revisiting this shortly. And this also seems out of scope for this mailing list. I'm curious to hear what you come up with and/or settle on. Please report back! Sander Verhagen [ [email protected] ] NOTICE: my e-mail address has changed. Please remove [email protected] now and start using [email protected] from now on. Please update your address book. Thank you! -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Kevin Burton Sent: Saturday, November 5, 2016 14:34 To: Maven Users List <[email protected]> Subject: Using maven profiles for easily parallel testing? We use TeamCity internally (which is great btw) for Maven testing. We have about 2000 tests which we continually integrate on every commit. The problem is that testing takes about 15 minutes from start to end. We use -T 16 on our tests and our boxes have 8 cores so this allows some tests to block on IO while others execute. However, I'm not happy with this. I want our tests to finish in 2 minutes. The only way this could happen is for us to hand parallelize everything (no fun), or do it automatically, or buy REALLY expensive hardware (no fun). I think one could do this with JUnit categories and maven profiles. I think what one could do is hash the name of the test and then based on the hash prefix, create N buckets. Right now we have 4 CI boxes so we would run 4 profiles, one per box. Then TeamCity could integrate the results. I think this is technically possible now but I think a lot of plugins would need to cooperate here. It would be better if maven didn't need to explicitly have to split these out. Maybe update surefire to have a way to only run 1/Nth of the tests and make the set determinstic? This way you can use two build boxes and each have two sets of tests with no intersection. Thoughts on this approach? I think it would be pretty awesome. I'm sure someone here knows the gang at TeamCity and could recommend this to the proper decision makers. -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts>
