Title: [129600] trunk/Tools
- Revision
- 129600
- Author
- zandober...@gmail.com
- Date
- 2012-09-26 00:18:16 -0700 (Wed, 26 Sep 2012)
Log Message
REGRESSION (r126189): No more mismatch-did-not-occur failures in reftests
https://bugs.webkit.org/show_bug.cgi?id=97112
Reviewed by Dirk Pranke.
Don't use the reftest's image hash when gathering output of its reference.
This indirectly forces the driver to return the reference's image output
as well. This specifically solves the problem when the reftest and its
mismatch reference are equal but the unexpected match failure is not
reported due to missing image output of the reference file.
* Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
(SingleTestRunner._run_reftest):
* Scripts/webkitpy/layout_tests/port/test.py:
(TestPort.diff_image):
(TestDriver.run_test):
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (129599 => 129600)
--- trunk/Tools/ChangeLog 2012-09-26 07:03:44 UTC (rev 129599)
+++ trunk/Tools/ChangeLog 2012-09-26 07:18:16 UTC (rev 129600)
@@ -1,3 +1,22 @@
+2012-09-26 Zan Dobersek <zandober...@gmail.com>
+
+ REGRESSION (r126189): No more mismatch-did-not-occur failures in reftests
+ https://bugs.webkit.org/show_bug.cgi?id=97112
+
+ Reviewed by Dirk Pranke.
+
+ Don't use the reftest's image hash when gathering output of its reference.
+ This indirectly forces the driver to return the reference's image output
+ as well. This specifically solves the problem when the reftest and its
+ mismatch reference are equal but the unexpected match failure is not
+ reported due to missing image output of the reference file.
+
+ * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
+ (SingleTestRunner._run_reftest):
+ * Scripts/webkitpy/layout_tests/port/test.py:
+ (TestPort.diff_image):
+ (TestDriver.run_test):
+
2012-09-25 Ryosuke Niwa <rn...@webkit.org>
Update master.cfg to invoke run-perf-tests with --no-show-results and --reset-results
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py (129599 => 129600)
--- trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py 2012-09-26 07:03:44 UTC (rev 129599)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py 2012-09-26 07:18:16 UTC (rev 129600)
@@ -295,7 +295,7 @@
putAllMismatchBeforeMatch = sorted
for expectation, reference_filename in putAllMismatchBeforeMatch(self._reference_files):
reference_test_name = self._port.relative_test_filename(reference_filename)
- reference_output = self._driver.run_test(DriverInput(reference_test_name, self._timeout, test_output.image_hash, should_run_pixel_test=True), self._stop_when_done)
+ reference_output = self._driver.run_test(DriverInput(reference_test_name, self._timeout, None, should_run_pixel_test=True), self._stop_when_done)
test_result = self._compare_output_with_reference(reference_output, test_output, reference_filename, expectation == '!=')
if (expectation == '!=' and test_result.failures) or (expectation == '==' and not test_result.failures):
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/test.py (129599 => 129600)
--- trunk/Tools/Scripts/webkitpy/layout_tests/port/test.py 2012-09-26 07:03:44 UTC (rev 129599)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/test.py 2012-09-26 07:18:16 UTC (rev 129600)
@@ -405,6 +405,10 @@
def diff_image(self, expected_contents, actual_contents, tolerance=None):
diffed = actual_contents != expected_contents
+ if not actual_contents and not expected_contents:
+ return (None, 0, None)
+ if not actual_contents or not expected_contents:
+ return (True, 0, None)
if 'ref' in expected_contents:
assert tolerance == 0
if diffed:
@@ -568,7 +572,11 @@
if stop_when_done:
self.stop()
- return DriverOutput(actual_text, test.actual_image, test.actual_checksum, audio,
+ if test.actual_checksum == test_input.image_hash:
+ image = None
+ else:
+ image = test.actual_image
+ return DriverOutput(actual_text, image, test.actual_checksum, audio,
crash=test.crash or test.web_process_crash, crashed_process_name=crashed_process_name,
crashed_pid=crashed_pid, crash_log=crash_log,
test_time=time.time() - start_time, timeout=test.timeout, error=test.error)
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-changes
