On 18/01/2013, at 9:38 AM, Rene Groeschke wrote: > Hi devs, > > As part of story to make the hmtl report generation more > efficient(https://github.com/gradle/gradle/blob/master/design-docs/testing-improvements.md#story-html-test-report-generation-is-efficient), > I've spiked porting the test html report generation in gradle to use jatl > (http://code.google.com/p/jatl/). > > Especially when dealing with verbose tests (lot of stdout and stderr) gradle > should benefit from that. > > I've pushed this spike on a seperate branch for the moment > (https://github.com/gradle/gradle/tree/jatl-testreports). > > Here are the performance test results: > > Speed results: > Test | 1.4-rc-1| JATL | Difference > --------------------------------------------------- > withTestNG | 9.842s | 10.087s | +2.49 % > withJUnit | 8.042s | 8.364s | +4.01 % > withVerboseTestNG | 20.222s | 21.334s | +5.50 % > withVerboseJUnit | 18.487s | 17.726s | -4.12 % > withVerboseTestNG*| 1.803m | 1.793m | -0.58 % > withVerboseJUnit* | 1.829m | 1.601m | -12.51 % > > Memory results: > Test | 1.4-rc1 | JATL | Difference > ---------------------------------------------------- > withTestNG | 22.657MB | 26.108MB| +15.23 % > withJUnit | 25.854MB | 25.386MB| -1.81 % > withVerboseTestNG | 22.738MB | 25.048MB| +10.16 % > withVerboseJUnit | 25.175MB | 23.927MB| -4.96 % > withVerboseTestNG*| 24.263MB | 24.8MB | +2.21% % > withVerboseJUnit* | 24.646MB | 23.869MB| -3.15 % > > * modified to output 10000 lines of stdout and stderr per Test > > As you can see especially verbose junit tests do benefit from the change, but > I wonder where the weak results for testng comes from. From my point of view, > I havn't seen any other change in the current master that explains the > difference, since we use the same report infrastructure for generating the > html reports for junit and testng.
Perhaps run the tests again, against a distribution built from the git revision just before you made the change, to isolate your changes from other changes in master since 1.4-rc-1. -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
