SimpleXmlWriter is definitely faster than java dom (xml generation has proved it).
Cheers! On Sat, Jan 19, 2013 at 9:18 AM, Luke Daley <[email protected]> wrote: > Another option would be to improve jatl and push the changes upstream. > > On 19/01/2013, at 0:34, Rene Groeschke <[email protected]> wrote: > >> I've run the tests against with just the cherry picked commit on top of the >> 1.4-rc-2. >> >> I think the difference between the testng and junit test results are caused >> by the different template setup. the junit template creates just 10 test >> classes and the testng one creates 100. >> >> I changed the template for my tests to use 1000 testclasses in the withJunit >> and withTestNg template and 100 Test classes in the verboseJunit and >> verboseTestNg. >> >> See the results below: >> >> Test | 1.4-rc1 | JATL | Difference >> --------------------------------------------------- >> withTestNG | 41.524s | 41.914s | +0.94 % >> withJUnit | 26.554s | 26.33s | -0.85 % >> withVerboseTestNG | 2.248m | 2.36m | +5.01 % >> withVerboseJUnit | 2.276m | 2.29m | +0.61 % >> >> Test | 1.4-rc1 | JATL | Difference >> ---------------------------------------------------- >> withTestNG | 23.647MB | 24.447MB| +3.38 % >> withJUnit | 20.461MB | 19.689MB| -3.77 % >> withVerboseTestNG | 20.009MB | 23.405MB| +16.97% >> withVerboseJUnit | 20.969MB | 24.584MB| +17.24% >> >> I can't see a real benefit of porting our current solution to jatl. I'm >> suprised to see, that the verbose test output with the jatl based solution >> needs more memory as the former dom based one. Maybe I can tweak the stderr >> and stdout a bit more, but I don't really see a potential for optimisation >> here atm. >> >> My next option is to spike the usage of our SimpleXmlWriter to generate the >> test reports. >> >> >> cheers! >> René >> -- >> Principal Engineer, >> Gradleware Inc. - Gradle Training, Support, Consulting >> [email protected] >> http://gradleware.com >> >> >> >> >> Adam Murdoch wrote: >>> >>> On 18/01/2013, at 9:38 AM, Rene Groeschke wrote: >>> >>>> Hi devs, >>>> >>>> As part of story to make the hmtl report generation more >>>> efficient(https://github.com/gradle/gradle/blob/master/design-docs/testing-improvements.md#story-html-test-report-generation-is-efficient), >>>> I've spiked porting the test html report generation in gradle to use >>>> jatl (http://code.google.com/p/jatl/). >>>> >>>> Especially when dealing with verbose tests (lot of stdout and stderr) >>>> gradle should benefit from that. >>>> >>>> I've pushed this spike on a seperate branch for the moment >>>> (https://github.com/gradle/gradle/tree/jatl-testreports). >>>> >>>> Here are the performance test results: >>>> >>>> Speed results: >>>> Test | 1.4-rc-1| JATL | Difference >>>> --------------------------------------------------- >>>> withTestNG | 9.842s | 10.087s | +2.49 % >>>> withJUnit | 8.042s | 8.364s | +4.01 % >>>> withVerboseTestNG | 20.222s | 21.334s | +5.50 % >>>> withVerboseJUnit | 18.487s | 17.726s | -4.12 % >>>> withVerboseTestNG*| 1.803m | 1.793m | -0.58 % >>>> withVerboseJUnit* | 1.829m | 1.601m | -12.51 % >>>> >>>> Memory results: >>>> Test | 1.4-rc1 | JATL | Difference >>>> ---------------------------------------------------- >>>> withTestNG | 22.657MB | 26.108MB| +15.23 % >>>> withJUnit | 25.854MB | 25.386MB| -1.81 % >>>> withVerboseTestNG | 22.738MB | 25.048MB| +10.16 % >>>> withVerboseJUnit | 25.175MB | 23.927MB| -4.96 % >>>> withVerboseTestNG*| 24.263MB | 24.8MB | +2.21% % >>>> withVerboseJUnit* | 24.646MB | 23.869MB| -3.15 % >>>> >>>> * modified to output 10000 lines of stdout and stderr per Test >>>> >>>> As you can see especially verbose junit tests do benefit from the >>>> change, but I wonder where the weak results for testng comes from. >>>> From my point of view, I havn't seen any other change in the current >>>> master that explains the difference, since we use the same report >>>> infrastructure for generating the html reports for junit and testng. >>> >>> Perhaps run the tests again, against a distribution built from the git >>> revision just before you made the change, to isolate your changes from >>> other changes in master since 1.4-rc-1. >>> >>> >>> -- >>> Adam Murdoch >>> Gradle Co-founder >>> http://www.gradle.org >>> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >>> http://www.gradleware.com >> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > -- Szczepan Faber Principal engineer@gradleware Lead@mockito --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
