Sounds good. We probably need to use JUNIT 4.8.1 to take advantage of it.
The main issue is to find out a solution for the TCP/IP port numbers per
testcase to avoid conflicts.
Thanks,
Raymond
--------------------------------------------------
From: "ant elder" <[email protected]>
Sent: Monday, January 18, 2010 8:31 AM
To: <[email protected]>
Subject: Running tests in parallel in the build (was:Re: [DISCUSS] 2.x build
structure
On Thu, Nov 19, 2009 at 12:20 PM, Mike Edwards
<[email protected]> wrote:
Luciano Resende wrote:
On Tue, Nov 17, 2009 at 7:20 AM, Simon Laws <[email protected]>
wrote:
I raise this topic with some trepidation as we've been here before but
here goes. We have been building (and releasing) all of the code in
the 2.x code base which has worked well to date. However two things
make me pause for thought:
- we are not running the otests as part of the main build and we are
already seeing regressions because of this (this will happen more
often as more otests are brought in). This is not good.
- my 2.x build (without otests) is already up to 45 minutes and there
are still more extensions to bring back in
Here is some numbers that can help in case we want to optimize the
current build as a first step.
http://people.apache.org/~lresende/tuscany/Tuscany%20Build.pdf
This is based on my build machine, which is taking 12 mins for a full
2.x
build
[INFO] Total time: 12 minutes 10 seconds
[INFO] Finished at: Tue Nov 17 08:07:47 PST 2009
[INFO] Final Memory: 102M/210M
[INFO]
------------------------------------------------------------------------
Folks,
I can see that Luciano has access to the Los Alamos supercomputer that
does
the nuclear simulations ;-) Can we have a go please?
The BPEL integration tests in 6 milliseconds is astoundingly good - how
do
you get that Luciano? My figures running the BPEL iTest on its own give
this:
Apache Tuscany SCA BPEL Integration Tests ............. SUCCESS [1.687s]
Apache Tuscany SCA iTest HelloWorld BPEL .............. SUCCESS [18.328s]
This is on a relatively speedy Lenovo T61p with 3Gb RAM under Win XP.
Even
the iTest HelloWorld BPEL is 3x faster on Luciano's system. I am using
Sun
JDK 1.6.0_13.
One thing I note about the build is that we build all the Samples and run
all the iTests.
I think that we should not build and test Samples by default. Let them
be
built on demand only.
No problem with running tests, but I note that even on Luciano's system,
the
tests seem to dominate the larger times in the list. Are we running too
many tests - and are the tests poorly structured? We should give some
thought to this.
Yours, Mike.
(replying to this email as its most directly related to the build time)
The surefire plugin has just had a new release and one of the new
features is parallel execution of tests. I gave it a quick try with
the Tuscany build, its not totally straight forward as a bunch of
tuscany tests fail or hang due to conflict's with things like http
ports getting used at the same time so i had to fudge a few things to
get a build through but it does look like it might reduce the build
time by about 50%. The Java CAA otests dont do much with http ports so
trying that I managed a test run with no fails which went from 1
minute 48 secs single thread down to just 39 seconds with parallel
tests. Its probably a bit too fragile with the tuscany build to check
this in just yet but it seems like a big enough improvement to be
worth looking at more, there's a blog post that describes more about
it if anyone else wants to have a go -
http://incodewetrustinc.blogspot.com/2010/01/run-your-junit-tests-concurrently-with.html
...ant
