On 15/06/2015 13:02, Mark Thomas wrote: > I have been experimenting with the free Azure credits that come with the > MSDN subscription Microsoft kindly offers to all Apache committers to > use for their ASF work. > > I have been looking at options for making the unit tests run faster. > > All the figures below are for running the trunk unit tests on a fully > updated Ubuntu 14.04 LTS instance. > > > A2 Basic 233:53 tests on hdd, with code coverage, 1 thread > D2 120:57 tests on hdd, with code coverage, 1 thread > D2 119:53 tests on ssd, with code coverage, 1 thread > D2 32:16 tests on hdd, no code coverage, 2 threads > D2 23:24 tests on hdd, no code coverage, 4 threads
Any higher than 2 threads per core there is a risk that tests experience IO timeouts. I'm going to recommend 1 thread per core. The other factor here is that multi-threaded tests requires ant 1.9.5 onwards. Mark > > (Both A2 and D2 boxes have 2 cores. D2 have 60% faster processors). > > I'll be testing larger instance with more cores later. > > So far, I think it is safe to draw the following conclusions: > - code coverage is expensive > - code coverage (as currently configured) requires single thread > execution (more on this below) > - 1 test thread per core definitely gives better performance > - 2 test threads per core gives even better performance > > Where the limit is for threads per core is TBD. > > I've already fixed the unit tests (I think) so parallel running is > possible. I'll be adding a threads option to build.xml shortly. It will > default to 1 and I'll add a comment to build.properties.default not to > increase it above 1 if code coverage is enabled (I might try and detect > and handle that case). Once I have data on threads vs cores I'll add > that too. > > The reason code coverage doesn't work with the junit threads option is > that cobertura serialises the coverage data between tests. If we > partitioned the tests (e.g. by name) and configured separated coverage > data files for each partition (merging them at the end) then cobertura > would be OK. Sensibly partitioning the tests is more effort than I have > time for at the moment so I am going with the simple option. > > Mark > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
