Hannes, Thank you for working on this and figuring this one out. It'd be nice if we'd have a JVM mode that would force full opt at a 4-5 iterations instead of 15.
Michel On Jul 18, 2013, at 6:43 AM, Hannes Wallnoefer <[email protected]> wrote: Hi Ivo, I've looked at the less.js performance issues. What I could reproduce is that Nashorn is initially slower than Rhino. This is due to the heavy use of MethodHandles which in current JDK8 generate a lot of intermediate code that needs to be JIT-compiled. In fact, of the first run of less.js more than 90% of time is spent compiling bytecode. This is something we're aware of and we're trying to improve. However, I do see Nashorn speeding up and passing by Rhino after a few runs. Roughly speaking I see Nashorn 3 times slower than Rhino on the first run, but 3 times faster once the code is fully compiled and optimized. I've pasted the benchmark results below. You'll notice that this is not from your benchmarking code. For some reason I couldn't get your code to locate the script files, probably due to some setup problem or my inexperience with gradle, so I decided to start with the original less-rhino-1.3.3.js from the main less.js repository, adapting it to Nashorn and adding a loop around the main code. I ran this with the benchmark/benchmark.less file from the main less.js repository. I've uploaded my benchmark scripts to gist.github.com: https://gist.github.com/hns/6029223 I'm not exactly sure why you don't see Nashorn catching up and getting faster to Rhino. It may be something that has improved in Nashorn or the JDK since then. At least I haven't found anything suspicious in your source code. Maybe you could rerun your tests with a current JDK8 from https://jdk8.java.net/, or help me get your test running? cheers, Hannes Below are my results with current Nashorn tip and Rhino master and JDK8-b98 (this is with default Rhino optlevel but results for optlevel 9 is basically the same): $ /home/hannes/local/jdk1.8.0/bin/java -jar ../nashorn/dist/nashorn.jar -scripting less-nashorn-1.3.3.js -- benchmark.less benchmarking nashorn... run 1: 7973 millis run 2: 3971 millis run 3: 1514 millis run 4: 1533 millis run 5: 782 millis run 6: 651 millis run 7: 434 millis run 8: 437 millis run 9: 378 millis run 10: 309 millis run 11: 338 millis run 12: 294 millis run 13: 305 millis run 14: 313 millis run 15: 419 millis run 16: 261 millis run 17: 247 millis run 18: 251 millis run 19: 234 millis run 20: 235 millis run 21: 250 millis run 22: 258 millis run 23: 236 millis run 24: 232 millis run 25: 227 millis run 26: 237 millis run 27: 238 millis run 28: 229 millis run 29: 225 millis run 30: 276 millis done $ /home/hannes/local/jdk1.8.0/bin/java -jar ../rhino/build/rhino1_7R5pre/js.jar less-rhino-1.3.3.js benchmark.less benchmarking rhino... run 1: 2585 millis run 2: 1437 millis run 3: 1173 millis run 4: 1058 millis run 5: 856 millis run 6: 824 millis run 7: 872 millis run 8: 779 millis run 9: 808 millis run 10: 756 millis run 11: 748 millis run 12: 751 millis run 13: 785 millis run 14: 773 millis run 15: 758 millis run 16: 739 millis run 17: 750 millis run 18: 736 millis run 19: 743 millis run 20: 739 millis run 21: 739 millis run 22: 742 millis run 23: 742 millis run 24: 751 millis run 25: 737 millis run 26: 748 millis run 27: 743 millis run 28: 746 millis run 29: 739 millis run 30: 739 millis done Am 2013-06-25 22:11, schrieb Ivo Houbrechts: >> Thanks for the heads up. JDK8 is only API frozen, so we have some time to >> address performance issues as they come up. There are some fixes in the >> pipe that will address some of the things I see here, but we should examine >> in more detail. Therefore, I have a few questions and comments. >> >> How did you run less-1.3.3.min.js independently of DOM/CSS? Running straight >> up I get several reference errors. Would you post the exact code you used >> to produce these results? > The source code is on github:https://github.com/houbie/lesscss/tree/nashorn, > in the nashorn branch > The main class is LessCompiler.java, which can compile an input fie to an > output file. > The project is build with gradle 1.6, but since it has only one test > dependency, it can be compiled/executed straight from an IDE. > The src/main/resources/js directory contains the 3 javascript files that get > merged and compiled: environment.js (minimal stubs for browser/dom), > less-1.3.3(.min).js and compile.js (java callback functions) > > PerformanceComparison.groovy in src/test/groovy was used as a basis for the > test runs to compile the bundled bootstrap less > > >> The chart you provide doesn't show Nashorn JDK7 completing. How did it >> fail? It is possible that Nashorn might converge differently than Rhino >> (focus on server side.) I'll see if I can get an in house version of JDK7 >> to run (we don't use the github back port - it has issues.) > It failed with a javascript error (don't remember the exact error) during the > 3th run within the same context, but I wouldn't bother too much because the > back port is based on an older Nashorn version (although I was intrigued by > the fact that it was faster then jdk8 for the first 2 runs) > >> You should also note that there is a bug in JDK7 javax.script that causes >> javascript to be chosen randomly (rhino/nashorn.) This has been addressed >> in later releases (fix is in the pipe.) > I explicitly load the engine by name and I don't think I ran into this. > >> Comparing JDK 7 and JDK 8 is apples and oranges at this point. JDK8 suffers >> a high start up cost of JSR-292 Lambda Forms. There is also a fix in the >> pipe for this. > As you can see in PerformanceComparison.groovy, I started to measure after > creating the compiler object, so jdk startup should not matter. Furthermore, > when I run the code from the master branch, that uses Rhino 1.7R4, I get the > same results in both jdk7 and jdk8 > > Grtz, > Ivo >
