Title: [168732] trunk/PerformanceTests
- Revision
- 168732
- Author
- [email protected]
- Date
- 2014-05-13 14:08:10 -0700 (Tue, 13 May 2014)
Log Message
DYEBench should run 20 iterations in browser
https://bugs.webkit.org/show_bug.cgi?id=132795
Reviewed by Gavin Barraclough.
Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
a more stable time and account for differences in the runtime environment, particularly,
ASLR (Address Space Layout Randomization).
While we can't account for the latter effect when the benchmark is ran inside a browser,
we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.
While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
it's MUCH better than the current sample size of 5.
* DoYouEvenBench/Full.html:
(benchmarkClient.iterationCount): Set the default iteration count to 20.
(startTest): Use benchmarkClient.iterationCount as the iteration count.
* DoYouEvenBench/resources/benchmark-report.js:
(benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
inside run-perf-tests.
Modified Paths
Diff
Modified: trunk/PerformanceTests/ChangeLog (168731 => 168732)
--- trunk/PerformanceTests/ChangeLog 2014-05-13 21:07:22 UTC (rev 168731)
+++ trunk/PerformanceTests/ChangeLog 2014-05-13 21:08:10 UTC (rev 168732)
@@ -1,3 +1,27 @@
+2014-05-13 Ryosuke Niwa <[email protected]>
+
+ DYEBench should run 20 iterations in browser
+ https://bugs.webkit.org/show_bug.cgi?id=132795
+
+ Reviewed by Gavin Barraclough.
+
+ Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
+ a more stable time and account for differences in the runtime environment, particularly,
+ ASLR (Address Space Layout Randomization).
+
+ While we can't account for the latter effect when the benchmark is ran inside a browser,
+ we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.
+
+ While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
+ it's MUCH better than the current sample size of 5.
+
+ * DoYouEvenBench/Full.html:
+ (benchmarkClient.iterationCount): Set the default iteration count to 20.
+ (startTest): Use benchmarkClient.iterationCount as the iteration count.
+ * DoYouEvenBench/resources/benchmark-report.js:
+ (benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
+ inside run-perf-tests.
+
2014-05-09 Ryosuke Niwa <[email protected]>
DYEBench should show 95th percentile right next to the mean with ±
Modified: trunk/PerformanceTests/DoYouEvenBench/Full.html (168731 => 168732)
--- trunk/PerformanceTests/DoYouEvenBench/Full.html 2014-05-13 21:07:22 UTC (rev 168731)
+++ trunk/PerformanceTests/DoYouEvenBench/Full.html 2014-05-13 21:08:10 UTC (rev 168732)
@@ -1,7 +1,7 @@
<!DOCTYPE html>
<html>
<head>
-<title>DoYouEvenBench v0.10</title>
+<title>DoYouEvenBench v0.11</title>
<style type="text/css">
caption { margin: 0; padding: 0; font-family: sans-serif; font-size: 1em; font-weight: bold; white-space: nowrap; }
#progressContainer { padding: 605px 0 10px 0; width: 800px; }
@@ -43,6 +43,7 @@
}
window.benchmarkClient = {
+ iterationCount: 20,
willRunTest: function () {
if (!progress) {
// We don't use the real progress element as some implementations animate it.
@@ -79,10 +80,10 @@
})();
function startTest() {
- var iterationCount = 5;
- benchmarkClient.testsCount = iterationCount * Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
+ var totalSubtestCount = Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
+ benchmarkClient.testsCount = benchmarkClient.iterationCount * totalSubtestCount;
var runner = new BenchmarkRunner(Suites, benchmarkClient);
- runner.runMultipleIterations(iterationCount);
+ runner.runMultipleIterations(benchmarkClient.iterationCount);
}
</script>
Modified: trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js (168731 => 168732)
--- trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js 2014-05-13 21:07:22 UTC (rev 168731)
+++ trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js 2014-05-13 21:08:10 UTC (rev 168732)
@@ -19,6 +19,7 @@
var valuesByIteration = new Array;
window.benchmarkClient = {
+ iterationCount: 5, // Use 4 different instances of DRT/WTR to run 5 iterations.
willStartFirstIteration: function (iterationCount) {
createTest = function (name, aggregator, isLastTest) {
return {
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
