Diff
Modified: trunk/PerformanceTests/ChangeLog (217088 => 217089)
--- trunk/PerformanceTests/ChangeLog 2017-05-19 03:03:55 UTC (rev 217088)
+++ trunk/PerformanceTests/ChangeLog 2017-05-19 03:05:07 UTC (rev 217089)
@@ -1,3 +1,14 @@
+2017-05-18 Ryosuke Niwa <[email protected]>
+
+ REGRESSION (r216694 - 216712): Performance test Speedometer/Full.html is failing
+ https://bugs.webkit.org/show_bug.cgi?id=172077
+
+ Reviewed by Chris Dumez.
+
+ Rename Full.html to index.html to match the convention of other benchmarks.
+
+ * Speedometer/index.html: Renamed from PerformanceTests/Speedometer/Full.html.
+
2017-05-18 Addy Osmani <[email protected]>
Speedometer: check-in missing React TodoMVC dependency
Deleted: trunk/PerformanceTests/Speedometer/Full.html (217088 => 217089)
--- trunk/PerformanceTests/Speedometer/Full.html 2017-05-19 03:03:55 UTC (rev 217088)
+++ trunk/PerformanceTests/Speedometer/Full.html 2017-05-19 03:05:07 UTC (rev 217089)
@@ -1,100 +0,0 @@
-<!DOCTYPE html>
-<html>
-<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
- <title>Speedometer 1.0</title>
- <link rel="stylesheet" href=""
- <script src="" defer></script>
- <script src="" defer></script>
- <script src="" defer></script>
- <script src="" defer></script>
- <script src="" defer></script>
-</head>
-<body>
-<main>
- <a id="logo-link" href="" id="logo" src=""
-
- <section id="home" class="selected">
- <p>
- Speedometer is a browser benchmark that measures the responsiveness of Web applications.
- It uses demo web applications to simulate user actions such as adding to-do items.
- </p>
- <p id="screen-size-warning"><strong>
- Your browser window is too small. For most accurate results, please make the view port size at least 850px by 650px.<br>
- It's currently <span id="screen-size"></span>.
- </strong></p>
- <div class="buttons">
- <button _onclick_="startTest()">Start Test</button>
- </div>
- <p class="show-about"><a href="" Speedometer</a></p>
- </section>
-
- <section id="running">
- <div id="testContainer"></div>
- <div id="progress"><div id="progress-completed"></div></div>
- <div id="info"></div>
- </section>
-
- <section id="summarized-results">
- <h1>Runs / Minute</h1>
- <div class="gauge"><div class="window"><div class="needle"></div></div></div>
- <hr>
- <div id="result-number"></div>
- <div id="confidence-number"></div>
- <div class="buttons">
- <button _onclick_="startTest()">Test Again</button>
- <button class="show-details" _onclick_="showResultDetails()">Details</button>
- </div>
- </section>
-
- <section id="detailed-results">
- <h1>Detailed Results</h1>
- <table class="results-table"></table>
- <table class="results-table"></table>
- <div class="arithmetic-mean"><label>Arithmetic Mean:</label><span id="results-with-statistics"></span></div>
- <div class="buttons">
- <button _onclick_="startTest()">Test Again</button>
- <button id="show-summary" _onclick_="showResultsSummary()">Summary</button>
- </div>
- <p class="show-about"><a href="" Speedometer</a></p>
- </section>
-
- <section id="about">
- <h1>About Speedometer</h1>
-
- <p>Speedometer measures simulated user interactions in web applications.</p>
-
- <p>
- The current benchmark uses TodoMVC to simulate user actions for adding, completing, and removing to-do items.
- Speedometer repeats the same actions using DOM APIs —
- a core set of web platform APIs used extensively in web applications —
- as well as six popular _javascript_ frameworks: Ember.js, Backbone.js, jQuery, AngularJS, React, and Flight.
- Many of these frameworks are used on the most popular websites in the world, such as Facebook and Twitter.
- The performance of these types of operations depends on the speed of the DOM APIs, the _javascript_ engine,
- CSS style resolution, layout, and other technologies.
- </p>
-
- <p>
- Although user-driven actions like mouse movements and keyboard input cannot be accurately emulated in _javascript_,
- Speedometer does its best to faithfully replay a typical workload within the demo applications.
- To make the run time long enough to measure with the limited precision,
- we synchronously execute a large number of the operations, such as adding one hundred to-do items.
- </p>
-
- <p>
- Some browser engines use an optimization strategy of doing some work asynchronously to reduce the run time of synchronous operations.
- While returning control back to _javascript_ execution as soon as possible is worth pursuing,
- a holistic, accurate measurement of web application performance involves measuring
- when these related, asynchronous computations actually complete.
- Thus, Speedometer measures the time browser spends executing those asynchronous tasks in Speedometer,
- estimated as the time between when a zero-second delay timer is scheduled and when it is fired.</p>
-
- <p class="note">
- <strong>Note:</strong> Speedometer is not meant to compare the performance of different _javascript_ frameworks.
- The mechanism we use to simulate user actions is different for each framework,
- and we’re forcing frameworks to do more work synchronously than needed in some cases to ensure run time can be measured.
- </p>
- </section>
-</main>
-</body>
-</html>
Copied: trunk/PerformanceTests/Speedometer/index.html (from rev 217088, trunk/PerformanceTests/Speedometer/Full.html) (0 => 217089)
--- trunk/PerformanceTests/Speedometer/index.html (rev 0)
+++ trunk/PerformanceTests/Speedometer/index.html 2017-05-19 03:05:07 UTC (rev 217089)
@@ -0,0 +1,100 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <title>Speedometer 1.0</title>
+ <link rel="stylesheet" href=""
+ <script src="" defer></script>
+ <script src="" defer></script>
+ <script src="" defer></script>
+ <script src="" defer></script>
+ <script src="" defer></script>
+</head>
+<body>
+<main>
+ <a id="logo-link" href="" id="logo" src=""
+
+ <section id="home" class="selected">
+ <p>
+ Speedometer is a browser benchmark that measures the responsiveness of Web applications.
+ It uses demo web applications to simulate user actions such as adding to-do items.
+ </p>
+ <p id="screen-size-warning"><strong>
+ Your browser window is too small. For most accurate results, please make the view port size at least 850px by 650px.<br>
+ It's currently <span id="screen-size"></span>.
+ </strong></p>
+ <div class="buttons">
+ <button _onclick_="startTest()">Start Test</button>
+ </div>
+ <p class="show-about"><a href="" Speedometer</a></p>
+ </section>
+
+ <section id="running">
+ <div id="testContainer"></div>
+ <div id="progress"><div id="progress-completed"></div></div>
+ <div id="info"></div>
+ </section>
+
+ <section id="summarized-results">
+ <h1>Runs / Minute</h1>
+ <div class="gauge"><div class="window"><div class="needle"></div></div></div>
+ <hr>
+ <div id="result-number"></div>
+ <div id="confidence-number"></div>
+ <div class="buttons">
+ <button _onclick_="startTest()">Test Again</button>
+ <button class="show-details" _onclick_="showResultDetails()">Details</button>
+ </div>
+ </section>
+
+ <section id="detailed-results">
+ <h1>Detailed Results</h1>
+ <table class="results-table"></table>
+ <table class="results-table"></table>
+ <div class="arithmetic-mean"><label>Arithmetic Mean:</label><span id="results-with-statistics"></span></div>
+ <div class="buttons">
+ <button _onclick_="startTest()">Test Again</button>
+ <button id="show-summary" _onclick_="showResultsSummary()">Summary</button>
+ </div>
+ <p class="show-about"><a href="" Speedometer</a></p>
+ </section>
+
+ <section id="about">
+ <h1>About Speedometer</h1>
+
+ <p>Speedometer measures simulated user interactions in web applications.</p>
+
+ <p>
+ The current benchmark uses TodoMVC to simulate user actions for adding, completing, and removing to-do items.
+ Speedometer repeats the same actions using DOM APIs —
+ a core set of web platform APIs used extensively in web applications —
+ as well as six popular _javascript_ frameworks: Ember.js, Backbone.js, jQuery, AngularJS, React, and Flight.
+ Many of these frameworks are used on the most popular websites in the world, such as Facebook and Twitter.
+ The performance of these types of operations depends on the speed of the DOM APIs, the _javascript_ engine,
+ CSS style resolution, layout, and other technologies.
+ </p>
+
+ <p>
+ Although user-driven actions like mouse movements and keyboard input cannot be accurately emulated in _javascript_,
+ Speedometer does its best to faithfully replay a typical workload within the demo applications.
+ To make the run time long enough to measure with the limited precision,
+ we synchronously execute a large number of the operations, such as adding one hundred to-do items.
+ </p>
+
+ <p>
+ Some browser engines use an optimization strategy of doing some work asynchronously to reduce the run time of synchronous operations.
+ While returning control back to _javascript_ execution as soon as possible is worth pursuing,
+ a holistic, accurate measurement of web application performance involves measuring
+ when these related, asynchronous computations actually complete.
+ Thus, Speedometer measures the time browser spends executing those asynchronous tasks in Speedometer,
+ estimated as the time between when a zero-second delay timer is scheduled and when it is fired.</p>
+
+ <p class="note">
+ <strong>Note:</strong> Speedometer is not meant to compare the performance of different _javascript_ frameworks.
+ The mechanism we use to simulate user actions is different for each framework,
+ and we’re forcing frameworks to do more work synchronously than needed in some cases to ensure run time can be measured.
+ </p>
+ </section>
+</main>
+</body>
+</html>
Modified: trunk/Tools/ChangeLog (217088 => 217089)
--- trunk/Tools/ChangeLog 2017-05-19 03:03:55 UTC (rev 217088)
+++ trunk/Tools/ChangeLog 2017-05-19 03:05:07 UTC (rev 217089)
@@ -1,3 +1,22 @@
+2017-05-18 Ryosuke Niwa <[email protected]>
+
+ REGRESSION (r216694 - 216712): Performance test Speedometer/Full.html is failing
+ https://bugs.webkit.org/show_bug.cgi?id=172077
+
+ Reviewed by Chris Dumez.
+
+ Fixed perftest.py by updating the lines to ignore in Speedometer.
+
+ Also start report the results as "Speedometer" instead of "DoYouEvenBench/Full.html" to perf.webkit.org.
+
+ * Scripts/webkitpy/performance_tests/perftest.py:
+ (PerfTest._lines_to_ignore_in_parser_result): Updated for Speedometer 2.0.
+ * Scripts/webkitpy/performance_tests/perftestsrunner.py:
+ (PerfTestsRunner._collect_tests): Use "Speedometer" instead of "Speedometer/index.html" as the test name.
+ (_generate_results_dict): Removed the workaround to keep reporting Speedometer as DoYouEvenBench.
+ * Scripts/webkitpy/performance_tests/perftestsrunner_unittest.py:
+ (MainTest.test_collect_tests_with_index_html_and_resources): Added.
+
2017-05-18 Andy Estes <[email protected]>
ENABLE(APPLE_PAY_DELEGATE) should be NO on macOS Sierra and earlier
Modified: trunk/Tools/Scripts/webkitpy/performance_tests/perftest.py (217088 => 217089)
--- trunk/Tools/Scripts/webkitpy/performance_tests/perftest.py 2017-05-19 03:03:55 UTC (rev 217088)
+++ trunk/Tools/Scripts/webkitpy/performance_tests/perftest.py 2017-05-19 03:05:07 UTC (rev 217089)
@@ -251,22 +251,10 @@
re.compile(re.escape("""Blocked access to external URL http://www.whatwg.org/specs/web-apps/current-work/""")),
re.compile(r"CONSOLE MESSAGE: (line \d+: )?Blocked script execution in '[A-Za-z0-9\-\.:]+' because the document's frame is sandboxed and the 'allow-scripts' permission is not set."),
re.compile(r"CONSOLE MESSAGE: (line \d+: )?Not allowed to load local resource"),
- # DoYouEvenBench
- re.compile(re.escape("CONSOLE MESSAGE: line 140: Miss the info bar? Run TodoMVC from a server to avoid a cross-origin error.")),
- re.compile(re.escape("CONSOLE MESSAGE: line 315: TypeError: Attempted to assign to readonly property.")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: DEBUG: -------------------------------")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: DEBUG: Ember : 1.3.1")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: DEBUG: Ember Data : 1.0.0-beta.6")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: DEBUG: Handlebars : 1.3.0")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: DEBUG: jQuery : 2.1.0")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: DEPRECATION: Namespaces should not begin with lowercase")),
- re.compile(re.escape("[email protected]:2:40")),
- re.compile(re.escape("[email protected]:3380:17")),
- re.compile(re.escape("CONSOLE MESSAGE: line 124: Booting in DEBUG mode")),
- re.compile(re.escape("CONSOLE MESSAGE: line 125: You can configure event logging with DEBUG.events.logAll()/logNone()/logByName()/logByAction()")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: Ember Views require jQuery 1.7, 1.8, 1.9, 1.10, or 2.0")),
- re.compile(re.escape("CONSOLE MESSAGE: line 3285: DEPRECATION: Namespaces should not begin with lowercase.")),
- re.compile(re.escape("CONSOLE MESSAGE: line 208: Miss the info bar? Run TodoMVC from a server to avoid a cross-origin error."))
+ # Speedometer 2.0
+ re.compile(r'CONSOLE MESSAGE: line \d+: DEBUG: -------------------------------'),
+ re.compile(r'CONSOLE MESSAGE: line \d+: DEBUG: Ember\s+: (\d\.)+'),
+ re.compile(r'CONSOLE MESSAGE: line \d+: DEBUG: jQuery\s+: (\d\.)+'),
]
def _filter_output(self, output):
Modified: trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py (217088 => 217089)
--- trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py 2017-05-19 03:03:55 UTC (rev 217088)
+++ trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py 2017-05-19 03:05:07 UTC (rev 217089)
@@ -179,6 +179,8 @@
relative_path = filesystem.relpath(path, self._base_path).replace('\\', '/')
if self._options.use_skipped_list and self._port.skips_perf_test(relative_path) and filesystem.normpath(relative_path) not in paths:
continue
+ if relative_path.endswith('/index.html'):
+ relative_path = relative_path[0:-len('/index.html')]
test = PerfTestFactory.create_perf_test(self._port, relative_path, path, test_runner_count=test_runner_count)
tests.append(test)
@@ -294,10 +296,6 @@
url = "" + '/'.join(path[0:i + 1]))
test_name = path[i]
- # FIXME: This is a temporary workaround for the fact perf dashboard doesn't support renaming tests.
- if test_name == 'Speedometer':
- test_name = 'DoYouEvenBench'
-
tests.setdefault(test_name, {'url': url})
current_test = tests[test_name]
if is_last_token:
Modified: trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner_unittest.py (217088 => 217089)
--- trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner_unittest.py 2017-05-19 03:03:55 UTC (rev 217088)
+++ trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner_unittest.py 2017-05-19 03:05:07 UTC (rev 217089)
@@ -80,6 +80,14 @@
port.host.filesystem.chdir(runner._port.perf_tests_dir()[:runner._port.perf_tests_dir().rfind(runner._host.filesystem.sep)])
self.assertItemsEqual(self._collect_tests_and_sort_test_name(runner), ['test1.html', 'test2.html'])
+ def test_collect_tests_with_index_html_and_resources(self):
+ runner, port = self.create_runner()
+ self._add_file(runner, 'Speedometer', 'index.html', 'test content')
+ self._add_file(runner, 'Speedometer/resources', 'resource.html', 'resource content')
+ tests = runner._collect_tests()
+ self.assertEqual(len(tests), 1)
+ self.assertEqual(tests[0].test_name(), 'Speedometer')
+
def test_collect_tests_with_skipped_list(self):
runner, port = self.create_runner()
