Title: [215599] trunk/Tools
- Revision
- 215599
- Author
- [email protected]
- Date
- 2017-04-20 18:49:59 -0700 (Thu, 20 Apr 2017)
Log Message
Test262 bot does not go red with failures
https://bugs.webkit.org/show_bug.cgi?id=171044
Reviewed by Aakash Jain.
The piped output of run-jsc-stress-tests is different from
its non-piped output. Switch to parsing the correct output.
* BuildSlaveSupport/build.webkit.org-config/master.cfg:
(RunTest262Tests.countFailures):
Just count the number of lines that start with "FAIL:".
* BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:
(RunTest262TestsTest.test_no_regressions_output):
(test_failure_output):
(test_failures_output):
Updated tests with new sample output produced partially from bots and
locally. In cases of failure the return code of run-jsc-stress-tests
is still 0 so update the expected return code in the tests.
Modified Paths
Diff
Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg (215598 => 215599)
--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg 2017-04-21 01:02:47 UTC (rev 215598)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg 2017-04-21 01:49:59 UTC (rev 215599)
@@ -335,9 +335,9 @@
def countFailures(self, cmd):
logText = cmd.logs['stdio'].getText()
- matches = re.findall(r'\(failed (\d+)\)', logText)
+ matches = re.findall(r'^FAIL:', logText, flags=re.MULTILINE)
if matches:
- return int(matches[-1])
+ return len(matches)
return 0
Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py (215598 => 215599)
--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py 2017-04-21 01:02:47 UTC (rev 215598)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py 2017-04-21 01:49:59 UTC (rev 215599)
@@ -153,21 +153,32 @@
def test_no_regressions_output(self):
self.assertResults(SUCCESS, ["test262-test"], 0, """Using the following jsc path: /WebKitBuild/Release/jsc
-168/168 """)
+Running test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default
+Running test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default-strict
+""")
+
def test_failure_output(self):
- self.assertResults(FAILURE, ["1 Test262 test failed"], 1, """Using the following jsc path: /WebKitBuild/Release/jsc
-test262.yaml/test262/test/built-ins/Array/from/iter-set-elem-prop-err.js.default: ERROR: Unexpected exit code: 0
-test262.yaml/test262/test/built-ins/Array/from/iter-set-elem-prop-err.js.default-strict: ERROR: Unexpected exit code: 0
-...
-43768/43768 (failed 1) """)
+ self.assertResults(FAILURE, ["1 Test262 test failed"], 0, """Using the following jsc path: /WebKitBuild/Release/jsc
+Running test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default
+Running test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default-strict
+test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default-strict: ERROR: Unexpected exit code: 0
+FAIL: test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default-strict
+""")
+
def test_failures_output(self):
- self.assertResults(FAILURE, ["75 Test262 tests failed"], 75, """Using the following jsc path: /WebKitBuild/Release/jsc
-...
-43768/43768 (failed 75) """)
+ self.assertResults(FAILURE, ["2 Test262 tests failed"], 0, """Using the following jsc path: /WebKitBuild/Release/jsc
+Running test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default
+test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default: ERROR: Unexpected exit code: 0
+FAIL: test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default
+Running test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default-strict
+test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default-strict: ERROR: Unexpected exit code: 0
+FAIL: test262.yaml/test262/test/annexB/built-ins/Date/prototype/getYear/length.js.default-strict
+""")
+
class RunLLINTCLoopTestsTest(unittest.TestCase):
def assertResults(self, expected_result, expected_text, rc, stdio):
cmd = StubRemoteCommand(rc, stdio)
Modified: trunk/Tools/ChangeLog (215598 => 215599)
--- trunk/Tools/ChangeLog 2017-04-21 01:02:47 UTC (rev 215598)
+++ trunk/Tools/ChangeLog 2017-04-21 01:49:59 UTC (rev 215599)
@@ -1,3 +1,25 @@
+2017-04-20 Joseph Pecoraro <[email protected]>
+
+ Test262 bot does not go red with failures
+ https://bugs.webkit.org/show_bug.cgi?id=171044
+
+ Reviewed by Aakash Jain.
+
+ The piped output of run-jsc-stress-tests is different from
+ its non-piped output. Switch to parsing the correct output.
+
+ * BuildSlaveSupport/build.webkit.org-config/master.cfg:
+ (RunTest262Tests.countFailures):
+ Just count the number of lines that start with "FAIL:".
+
+ * BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:
+ (RunTest262TestsTest.test_no_regressions_output):
+ (test_failure_output):
+ (test_failures_output):
+ Updated tests with new sample output produced partially from bots and
+ locally. In cases of failure the return code of run-jsc-stress-tests
+ is still 0 so update the expected return code in the tests.
+
2017-04-20 Brady Eidson <[email protected]>
WebContent process becomes unresponsive after returning nil from async version of -webView:createWebViewWithConfiguration:...
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
