I should've included a link to the PR: https://github.com/apache/groovy/pull/1761
On Wed, Aug 10, 2022 at 10:30 PM Paul King <pa...@asert.com.au> wrote: > > Hi folks, > > The performance subproject in our build was designed to check on > (among other things) compiler performance. The 'performanceTests' task > compiles a set of source files across numerous Groovy versions. Over > time, the number of source files it checks against has been diluted. > The exact history isn't too important, but the summary is something > like this: > > * We have tried to keep the tests to just relying on the core subproject. > * We have tried to not rely on too many external dependencies. > * We have tried not to include any classes which have been impacted by > package changes, e.g. using XmlParser would require > groovy.util.XmlParser in early versions and groovy.xml.XmlParser in > later versions. > * We haven't included functionality included in later versions. > * Our approach when dropping further impacted files has been to just > remove them from the list of files compiled. > > The end result is that the current performance test reveals very > little information. I think we need to add in numerous additional > files. I have also recently pruned down the versions we try to check > against. I don't think there is a lot of value in knowing performance > compared to Groovy 1.8.x. > > I have just created a PR which adds a bunch of additional files - I > used a whole bunch of examples I created some years ago for PLEAC. I > believe the licensing is all legit. There are around 10000 lines of > Groovy source in those files. The alternative would be to pick our way > through the current source base and add selected files which meet all > of the criteria we have above. > > Incidentally, results on Windows were: > > Task :performance:performanceTests > Groovy 2.4.21 Average 437.965ms ± 16.07ms > Groovy 4.0.5-SNAPSHOT Average 649.8199999999999ms ± 89.79ms (48.37% slower) > Groovy 4.0.4 Average 654.635ms ± 85.86ms (49.47% slower) > Groovy 2.5.18 Average 676.224ms ± 8.38ms (54.4% slower) > Groovy 3.0.12 Average 792.625ms ± 192.19ms (80.98% slower) > > On Linux, the results were: > > Task :performance:performanceTests > Groovy 2.4.21 Average 473.17699999999996ms ± 24.47ms > Groovy 4.0.4 Average 648.592ms ± 129.15ms (37.07% slower) > Groovy current Average 649.4449999999999ms ± 126.79ms (37.25% slower) > Groovy 2.5.18 Average 683.147ms ± 12.6ms (44.37% slower) > Groovy 3.0.12 Average 796.938ms ± 225.73ms (68.42% slower) > > Here, "current" is the version of the branch you are running on > (GROOVY_4_0_X for me). I also published the current branch (on > Windows) to the local Maven repo in case that made any difference. > > The usual disclaimer about microbenchmarks applies. Also, remember, > this is testing oldish code - so not code rich in AST transforms or > using CompileStatic. > > Thoughts about whether the Pleac examples are good to add or should we > move all of this to another repo? > > Cheers, Paul.
