Hi Bernard,
Sorry for sending this message a second time, first one was rejected by
nashorn-dev because of the attachment.
I looked at performance of dust.js with various versions of Nashorn. I
didn't see a regression between 8u05 and 8u25, quite the contrary, it
seems that u25 is about 30% faster after warmup. However, it does look
like startup and warmup is a bit more inconsistent with u25. It
sometimes takes a bit longer to reach final speed.
With 8u40 I noticed a pretty dramatic performance regression right away,
it's about 3-4x slower than 8u05 and 8u25. The culprit turns out to be
our dual fields representation which is turned on by default in u40.
When that feature is disabled, u40 is even a bit faster than u25, and
also warmup seems to be smoother again.
You can test running with dual fields disabled by setting the
"nashorn.fields.objects" system property to "true", for example by
adding -Dnashorn.fields.objects=true to the command line.
I'm pasting in the script I used for benchmarking below. You can run it
along with the templates you sent with the following command line:
> jjs -Dnashorn.fields.objects=true -scripting dust-bench.js
Let me know if my benchmark does not measure what is relevant to your
application. It's possible for example that you're repeatedly calling
into nashorn script engines and there's some potential issues with that
(you should reuse the same script engine if possible).
Regards,
Hannes
===== begin dust-bench.js =====
if (!this.readFully) readFully = read;
load("dist/dust-full.js");
var template = readFully("basic.tl");
var compiled = dust.compile(template, "basic");
print("compiled");
dust.loadSource(compiled);
print("loaded");
var json = JSON.parse(readFully("basic.json"));
print("read json");
print();
function bench(n) {
var start = Date.now();
for (var i = 0; i < 2000; i++) {
dust.render("basic", json, function(err, out) {
if (err) throw err;
});
}
print(n, "done", Date.now() - start);
}
for (var i = 0; i < 200; i++) {
bench(i);
}
===== end dust-bench.js =====
Am 2014-11-25 um 20:28 schrieb Bernard Liang:
Hannes,
Sure, I¹ve reproduced the data you requested below, as they are small
enough to fit comfortably in a standard message. However, in addition I
would still like to reiterate the questions from my previous reply,
regarding across-build Nashorn-related changes and performance benchmarks.
Your reply also suggests that you have some familiarity with Dust, but if
that is not the case, I can certainly provide some more details.
Regards,
Bernard Liang
===== begin basic.tl =====
<html>{~n}
<head>{~n}
<title>{page_title}</title>{~n}
</head>{~n}
<body>{~n}
<ul>{~n}
{#names}
<li>{title} {name}</li>{~n}
{/names}
</ul>{~n}
</body>{~n}
</html>
===== end basic.tl =====
===== begin basic.json =====
{
"page_title": "Benchmark",
"title": "Sir",
"names": [{
"name": "Moe"
},
{
"name": "Larry"
},
{
"name": "Curly"
}]
}
===== end basic.json =====
On 11/25/14, 2:30 AM, "Hannes Wallnoefer" <[email protected]>
wrote:
Hi Bernard,
I'm trying to reproduce your problems. In your first mail to the list
you wrote you attached a Dust template and JSON data. I think you either
forgot to attach it, or it was stripped by the list software. Can you
please try to send it again, or put it somewhere we can download it?
Thanks,
Hannes
Am 2014-11-18 um 03:21 schrieb Bernard Liang:
Michel et al,
I¹ve run our local test battery using the link you provided, and while
in some cases there is improvement, overall the performance still seems
to be closer to u25 levels than u5 levels. For what it¹s worth, I did
notice that the performance improvements from u25 to u40 were generally
better in pooled environments than ones where a single instance of the
execution environment was running per thread. This leads itself to a few
questions, some of which are reiterated from the original inquiry:
* Is anyone familiar with (significant) specific changes in the
Nashorn libraries from u5 => u25 => u40 that might be related to this
regression and could explain the u25 and/or u40 changes in more detail
(that might have led to the recommendation to use u40)?
* Do you have any performance suites (internal or other) that test
various Nashorn benchmarks across different releases (of JDK8, for
instance)? Do the results of those correlate with our findings?
Regards,
Bernard Liang
PS. The output of `java -version` most recently tested was as follows:
java version "1.8.0_40-ea"
Java(TM) SE Runtime Environment (build 1.8.0_40-ea-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b16, mixed mode)
Previous versions tested:
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)
java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)
From: Michel Trudeau
<[email protected]<mailto:[email protected]>>
Date: Monday, November 17, 2014 at 1:12 PM
To: Bernard Liang <[email protected]<mailto:[email protected]>>
Cc: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: Nashorn performance regression from JDK8u5 to JDK8u25?
Bernard,
It'd be great if you could try the latest 8u40 stable build. We are
planning to release 8u40 early in the new year.
https://jdk8.java.net/download.html
We also have an optional optimizer in 8u40, enable it with the command
line argument '-ot'.
Thanks,
Michel
Bernard Liang wrote:
Hello,
After running some performance tests on the Cartesian product of
([JDK8u5, JDK8u25] x [simple template, complex templates] x
[all-or-nothing, streaming chunks] x [single dust instance per thread,
pooled dust instances] x [blank Dust instances, Dust instances with
templates preloaded]), we find that JDK8u25 performance is very
consistently considerably worse than JDK8u5 (by roughly 10-100%, with
the average falling somewhere between there). The relevant code has been
executed enough times (on the order of 10,000 times) to reach reasonably
warmed-up states. If some of the items on the axes of the Cartesian
product don¹t make much sense, you can ignore the fuzzy parts of the
detailed breakdown for now, with the general understanding that various
different environments have been tested and shown to yield the same
results.
Some additional high-level context:
Dust is basically a templating language used to render JSON data into
HTML with compilable ³templates": https://github.com/linkedin/dustjs (we
are at the v2.4.2 tag)
³simple template² = ~150 bytes each of one (precompiled) template +
context JSON (attached)
³complex templates² = ~350 compiled templates spanning ~245KB in
compiled JS + ~75KB of JSON context (proprietary data)
>From
https://blogs.oracle.com/nashorn/entry/latest_news_from_the_nashorn, it
sounded like there were some recent updates made to Nashorn performance
around the u20 mark, but that seems to have caused a regression rather
than an improvement. Is this something that nashorn-dev is aware of? Is
there any way we can help diagnose the issue further using publicly safe
data? (If you¹re looking for a way to reproduce this, the attached basic
Dust template + JSON context should be adequate under almost any
environment.)
Regards,
Bernard Liang
