One thing that might help with the port numbers is the Maven Build Helper plugin which can reserve a set of unused ports for tests to use, see http://mojo.codehaus.org/build-helper-maven-plugin/reserve-network-port-mojo.html. I'm not sure how we'd hook that in to how the tests and Tuscany runtime are using ports so just pointing it out as something to be aware of.
...ant On Tue, Jan 19, 2010 at 5:40 PM, Raymond Feng <[email protected]> wrote: > Sounds good. We probably need to use JUNIT 4.8.1 to take advantage of it. > The main issue is to find out a solution for the TCP/IP port numbers per > testcase to avoid conflicts. > > Thanks, > Raymond > -------------------------------------------------- > From: "ant elder" <[email protected]> > Sent: Monday, January 18, 2010 8:31 AM > To: <[email protected]> > Subject: Running tests in parallel in the build (was:Re: [DISCUSS] 2.x build > structure > >> On Thu, Nov 19, 2009 at 12:20 PM, Mike Edwards >> <[email protected]> wrote: >>> >>> Luciano Resende wrote: >>>> >>>> On Tue, Nov 17, 2009 at 7:20 AM, Simon Laws <[email protected]> >>>> wrote: >>>>> >>>>> I raise this topic with some trepidation as we've been here before but >>>>> here goes. We have been building (and releasing) all of the code in >>>>> the 2.x code base which has worked well to date. However two things >>>>> make me pause for thought: >>>>> >>>>> - we are not running the otests as part of the main build and we are >>>>> already seeing regressions because of this (this will happen more >>>>> often as more otests are brought in). This is not good. >>>>> - my 2.x build (without otests) is already up to 45 minutes and there >>>>> are still more extensions to bring back in >>>>> >>>> >>>> Here is some numbers that can help in case we want to optimize the >>>> current build as a first step. >>>> http://people.apache.org/~lresende/tuscany/Tuscany%20Build.pdf >>>> >>>> This is based on my build machine, which is taking 12 mins for a full >>>> 2.x >>>> build >>>> >>>> [INFO] Total time: 12 minutes 10 seconds >>>> [INFO] Finished at: Tue Nov 17 08:07:47 PST 2009 >>>> [INFO] Final Memory: 102M/210M >>>> [INFO] >>>> ------------------------------------------------------------------------ >>>> >>>> >>> Folks, >>> >>> I can see that Luciano has access to the Los Alamos supercomputer that >>> does >>> the nuclear simulations ;-) Can we have a go please? >>> >>> >>> The BPEL integration tests in 6 milliseconds is astoundingly good - how >>> do >>> you get that Luciano? My figures running the BPEL iTest on its own give >>> this: >>> >>> Apache Tuscany SCA BPEL Integration Tests ............. SUCCESS [1.687s] >>> Apache Tuscany SCA iTest HelloWorld BPEL .............. SUCCESS [18.328s] >>> >>> This is on a relatively speedy Lenovo T61p with 3Gb RAM under Win XP. >>> Even >>> the iTest HelloWorld BPEL is 3x faster on Luciano's system. I am using >>> Sun >>> JDK 1.6.0_13. >>> >>> >>> One thing I note about the build is that we build all the Samples and run >>> all the iTests. >>> >>> I think that we should not build and test Samples by default. Let them >>> be >>> built on demand only. >>> >>> No problem with running tests, but I note that even on Luciano's system, >>> the >>> tests seem to dominate the larger times in the list. Are we running too >>> many tests - and are the tests poorly structured? We should give some >>> thought to this. >>> >>> >>> Yours, Mike. >>> >> >> (replying to this email as its most directly related to the build time) >> >> The surefire plugin has just had a new release and one of the new >> features is parallel execution of tests. I gave it a quick try with >> the Tuscany build, its not totally straight forward as a bunch of >> tuscany tests fail or hang due to conflict's with things like http >> ports getting used at the same time so i had to fudge a few things to >> get a build through but it does look like it might reduce the build >> time by about 50%. The Java CAA otests dont do much with http ports so >> trying that I managed a test run with no fails which went from 1 >> minute 48 secs single thread down to just 39 seconds with parallel >> tests. Its probably a bit too fragile with the tuscany build to check >> this in just yet but it seems like a big enough improvement to be >> worth looking at more, there's a blog post that describes more about >> it if anyone else wants to have a go - >> >> http://incodewetrustinc.blogspot.com/2010/01/run-your-junit-tests-concurrently-with.html >> >> ...ant > >
