Hi folks, The performance subproject in our build was designed to check on (among other things) compiler performance. The 'performanceTests' task compiles a set of source files across numerous Groovy versions. Over time, the number of source files it checks against has been diluted. The exact history isn't too important, but the summary is something like this:
* We have tried to keep the tests to just relying on the core subproject. * We have tried to not rely on too many external dependencies. * We have tried not to include any classes which have been impacted by package changes, e.g. using XmlParser would require groovy.util.XmlParser in early versions and groovy.xml.XmlParser in later versions. * We haven't included functionality included in later versions. * Our approach when dropping further impacted files has been to just remove them from the list of files compiled. The end result is that the current performance test reveals very little information. I think we need to add in numerous additional files. I have also recently pruned down the versions we try to check against. I don't think there is a lot of value in knowing performance compared to Groovy 1.8.x. I have just created a PR which adds a bunch of additional files - I used a whole bunch of examples I created some years ago for PLEAC. I believe the licensing is all legit. There are around 10000 lines of Groovy source in those files. The alternative would be to pick our way through the current source base and add selected files which meet all of the criteria we have above. Incidentally, results on Windows were: > Task :performance:performanceTests Groovy 2.4.21 Average 437.965ms ± 16.07ms Groovy 4.0.5-SNAPSHOT Average 649.8199999999999ms ± 89.79ms (48.37% slower) Groovy 4.0.4 Average 654.635ms ± 85.86ms (49.47% slower) Groovy 2.5.18 Average 676.224ms ± 8.38ms (54.4% slower) Groovy 3.0.12 Average 792.625ms ± 192.19ms (80.98% slower) On Linux, the results were: > Task :performance:performanceTests Groovy 2.4.21 Average 473.17699999999996ms ± 24.47ms Groovy 4.0.4 Average 648.592ms ± 129.15ms (37.07% slower) Groovy current Average 649.4449999999999ms ± 126.79ms (37.25% slower) Groovy 2.5.18 Average 683.147ms ± 12.6ms (44.37% slower) Groovy 3.0.12 Average 796.938ms ± 225.73ms (68.42% slower) Here, "current" is the version of the branch you are running on (GROOVY_4_0_X for me). I also published the current branch (on Windows) to the local Maven repo in case that made any difference. The usual disclaimer about microbenchmarks applies. Also, remember, this is testing oldish code - so not code rich in AST transforms or using CompileStatic. Thoughts about whether the Pleac examples are good to add or should we move all of this to another repo? Cheers, Paul.
