Title: [123893] trunk/Tools
Revision
123893
Author
[email protected]
Date
2012-07-27 11:21:08 -0700 (Fri, 27 Jul 2012)

Log Message

nrwt: move image diffing code to a separate module
https://bugs.webkit.org/show_bug.cgi?id=92447

Reviewed by Ryosuke Niwa.

This patch moves the code to talk to ImageDiff into its own
module, and adds more tests for it. In addition, the patch
modifies diff_image() so that we don't automatically stop
ImageDiff after a single invocation, and thus subsequent
diffs may be slightly faster. (Note that the chromium ports
don't use any of this code; that is not changed by this patch).

The main motivation for this change is to move more "generic"
code out of the port/* classes, and in particular to move more
code out of webkit.py so that we can eventually eliminate it by
merging it into base.py.

This patch also splits MockServerProcess out from driver_unittest.py
so that it can be re-used.

* Scripts/webkitpy/layout_tests/port/base.py:
(Port.__init__):
(Port.diff_image):
(Port.clean_up_test_run):
* Scripts/webkitpy/layout_tests/port/driver.py:
(Driver.__init__):
(Driver._start):
* Scripts/webkitpy/layout_tests/port/driver_unittest.py:
(DriverTest.test_stop_cleans_up_properly):
(DriverTest.test_two_starts_cleans_up_properly):
(DriverTest.test_start_actually_starts):
* Scripts/webkitpy/layout_tests/port/efl.py:
(EflPort.clean_up_test_run):
* Scripts/webkitpy/layout_tests/port/gtk.py:
(GtkPort.clean_up_test_run):
* Scripts/webkitpy/layout_tests/port/image_diff.py: Added.
(ImageDiffer):
(ImageDiffer.__init__):
(ImageDiffer.diff_image):
(ImageDiffer._start):
(ImageDiffer._read):
(ImageDiffer.stop):
* Scripts/webkitpy/layout_tests/port/image_diff_unittest.py: Added.
(for):
(FakePort):
(FakePort.__init__):
(FakePort._path_to_image_diff):
(FakePort.setup_environ_for_server):
(TestImageDiffer):
(TestImageDiffer.test_diff_image):
* Scripts/webkitpy/layout_tests/port/mock_drt_unittest.py:
(MockDRTPortTest.test_diff_image):
* Scripts/webkitpy/layout_tests/port/port_testcase.py:
(PortTestCase.test_diff_image__missing_both):
(PortTestCase.test_diff_image):
(PortTestCase.test_diff_image.make_proc):
* Scripts/webkitpy/layout_tests/port/server_process.py:
(ServerProcess._start):
* Scripts/webkitpy/layout_tests/port/server_process_mock.py: Added.
(MockServerProcess):
(MockServerProcess.__init__):
(MockServerProcess.write):
(MockServerProcess.has_crashed):
(MockServerProcess.read_stdout_line):
(MockServerProcess.read_stdout):
(MockServerProcess.pop_all_buffered_stderr):
(MockServerProcess.read_either_stdout_or_stderr_line):
(MockServerProcess.start):
(MockServerProcess.stop):
(MockServerProcess.kill):
* Scripts/webkitpy/layout_tests/port/webkit.py:
(WebKitPort._build_driver_flags):
(WebKitPort._symbols_string):

Modified Paths

Added Paths

Diff

Modified: trunk/Tools/ChangeLog (123892 => 123893)


--- trunk/Tools/ChangeLog	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/ChangeLog	2012-07-27 18:21:08 UTC (rev 123893)
@@ -1,3 +1,79 @@
+2012-07-27  Dirk Pranke  <[email protected]>
+
+        nrwt: move image diffing code to a separate module
+        https://bugs.webkit.org/show_bug.cgi?id=92447
+
+        Reviewed by Ryosuke Niwa.
+
+        This patch moves the code to talk to ImageDiff into its own
+        module, and adds more tests for it. In addition, the patch
+        modifies diff_image() so that we don't automatically stop
+        ImageDiff after a single invocation, and thus subsequent
+        diffs may be slightly faster. (Note that the chromium ports
+        don't use any of this code; that is not changed by this patch).
+
+        The main motivation for this change is to move more "generic"
+        code out of the port/* classes, and in particular to move more
+        code out of webkit.py so that we can eventually eliminate it by
+        merging it into base.py.
+
+        This patch also splits MockServerProcess out from driver_unittest.py
+        so that it can be re-used.
+
+        * Scripts/webkitpy/layout_tests/port/base.py:
+        (Port.__init__):
+        (Port.diff_image):
+        (Port.clean_up_test_run):
+        * Scripts/webkitpy/layout_tests/port/driver.py:
+        (Driver.__init__):
+        (Driver._start):
+        * Scripts/webkitpy/layout_tests/port/driver_unittest.py:
+        (DriverTest.test_stop_cleans_up_properly):
+        (DriverTest.test_two_starts_cleans_up_properly):
+        (DriverTest.test_start_actually_starts):
+        * Scripts/webkitpy/layout_tests/port/efl.py:
+        (EflPort.clean_up_test_run):
+        * Scripts/webkitpy/layout_tests/port/gtk.py:
+        (GtkPort.clean_up_test_run):
+        * Scripts/webkitpy/layout_tests/port/image_diff.py: Added.
+        (ImageDiffer):
+        (ImageDiffer.__init__):
+        (ImageDiffer.diff_image):
+        (ImageDiffer._start):
+        (ImageDiffer._read):
+        (ImageDiffer.stop):
+        * Scripts/webkitpy/layout_tests/port/image_diff_unittest.py: Added.
+        (for):
+        (FakePort):
+        (FakePort.__init__):
+        (FakePort._path_to_image_diff):
+        (FakePort.setup_environ_for_server):
+        (TestImageDiffer):
+        (TestImageDiffer.test_diff_image):
+        * Scripts/webkitpy/layout_tests/port/mock_drt_unittest.py:
+        (MockDRTPortTest.test_diff_image):
+        * Scripts/webkitpy/layout_tests/port/port_testcase.py:
+        (PortTestCase.test_diff_image__missing_both):
+        (PortTestCase.test_diff_image):
+        (PortTestCase.test_diff_image.make_proc):
+        * Scripts/webkitpy/layout_tests/port/server_process.py:
+        (ServerProcess._start):
+        * Scripts/webkitpy/layout_tests/port/server_process_mock.py: Added.
+        (MockServerProcess):
+        (MockServerProcess.__init__):
+        (MockServerProcess.write):
+        (MockServerProcess.has_crashed):
+        (MockServerProcess.read_stdout_line):
+        (MockServerProcess.read_stdout):
+        (MockServerProcess.pop_all_buffered_stderr):
+        (MockServerProcess.read_either_stdout_or_stderr_line):
+        (MockServerProcess.start):
+        (MockServerProcess.stop):
+        (MockServerProcess.kill):
+        * Scripts/webkitpy/layout_tests/port/webkit.py:
+        (WebKitPort._build_driver_flags):
+        (WebKitPort._symbols_string):
+
 2012-07-27  Tom Hudson  <[email protected]>
 
         Activate committer bit granted April 6th so I can garden.

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -56,6 +56,8 @@
 from webkitpy.layout_tests.port import config as port_config
 from webkitpy.layout_tests.port import driver
 from webkitpy.layout_tests.port import http_lock
+from webkitpy.layout_tests.port import image_diff
+from webkitpy.layout_tests.port import server_process
 from webkitpy.layout_tests.servers import apache_http_server
 from webkitpy.layout_tests.servers import http_server
 from webkitpy.layout_tests.servers import websocket_server
@@ -110,6 +112,8 @@
         self._helper = None
         self._http_server = None
         self._websocket_server = None
+        self._image_differ = None
+        self._server_process_constructor = server_process.ServerProcess  # overridable for testing
         self._http_lock = None  # FIXME: Why does this live on the port object?
 
         # Python's Popen has a bug that causes any pipes opened to a
@@ -309,8 +313,18 @@
 
         |tolerance| should be a percentage value (0.0 - 100.0).
         If it is omitted, the port default tolerance value is used.
+
+        If an error occurs (like ImageDiff isn't found, or crashes, we log an error and return True (for a diff).
         """
-        raise NotImplementedError('Port.diff_image')
+        if not actual_contents and not expected_contents:
+            return (None, 0)
+        if not actual_contents or not expected_contents:
+            return (True, 0)
+        if not self._image_differ:
+            self._image_differ = image_diff.ImageDiffer(self)
+        self.set_option_default('tolerance', 0.1)
+        tolerance = tolerance or self.get_option('tolerance')
+        return self._image_differ.diff_image(expected_contents, actual_contents, tolerance)
 
     def diff_text(self, expected_text, actual_text, expected_filename, actual_filename):
         """Returns a string containing the diff of the two text strings
@@ -780,7 +794,9 @@
 
     def clean_up_test_run(self):
         """Perform port-specific work at the end of a test run."""
-        pass
+        if self._image_differ:
+            self._image_differ.stop()
+            self._image_differ = None
 
     # FIXME: os.environ access should be moved to onto a common/system class to be more easily mockable.
     def _value_or_default_from_environ(self, name, default=None):

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/driver.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/driver.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/driver.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -36,9 +36,7 @@
 
 from webkitpy.common.system import path
 
-from webkitpy.layout_tests.port import server_process
 
-
 _log = logging.getLogger(__name__)
 
 
@@ -115,9 +113,6 @@
         self._worker_number = worker_number
         self._no_timeout = no_timeout
 
-        # overridable for testing.
-        self._server_process_constructor = server_process.ServerProcess
-
         self._driver_tempdir = None
         # WebKitTestRunner can report back subprocess crashes by printing
         # "#CRASHED - PROCESSNAME".  Since those can happen at any time
@@ -271,7 +266,7 @@
         environment['LOCAL_RESOURCE_ROOT'] = self._port.layout_tests_dir()
         self._crashed_process_name = None
         self._crashed_pid = None
-        self._server_process = self._server_process_constructor(self._port, server_name, self.cmd_line(pixel_tests, per_test_args), environment)
+        self._server_process = self._port._server_process_constructor(self._port, server_name, self.cmd_line(pixel_tests, per_test_args), environment)
         self._server_process.start()
 
     def stop(self):

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/driver_unittest.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/driver_unittest.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/driver_unittest.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -31,6 +31,7 @@
 from webkitpy.common.system.systemhost_mock import MockSystemHost
 
 from webkitpy.layout_tests.port import Port, Driver, DriverOutput
+from webkitpy.layout_tests.port.server_process_mock import MockServerProcess
 
 # FIXME: remove the dependency on TestWebKitPort
 from webkitpy.layout_tests.port.webkit_unittest import TestWebKitPort
@@ -232,8 +233,8 @@
 
     def test_stop_cleans_up_properly(self):
         port = TestWebKitPort()
+        port._server_process_constructor = MockServerProcess
         driver = Driver(port, 0, pixel_tests=True)
-        driver._server_process_constructor = MockServerProcess
         driver.start(True, [])
         last_tmpdir = port._filesystem.last_tmpdir
         self.assertNotEquals(last_tmpdir, None)
@@ -242,8 +243,8 @@
 
     def test_two_starts_cleans_up_properly(self):
         port = TestWebKitPort()
+        port._server_process_constructor = MockServerProcess
         driver = Driver(port, 0, pixel_tests=True)
-        driver._server_process_constructor = MockServerProcess
         driver.start(True, [])
         last_tmpdir = port._filesystem.last_tmpdir
         driver._start(True, [])
@@ -251,50 +252,11 @@
 
     def test_start_actually_starts(self):
         port = TestWebKitPort()
+        port._server_process_constructor = MockServerProcess
         driver = Driver(port, 0, pixel_tests=True)
-        driver._server_process_constructor = MockServerProcess
         driver.start(True, [])
         self.assertTrue(driver._server_process.started)
 
 
-class MockServerProcess(object):
-    def __init__(self, port_obj=None, name=None, cmd=None, env=None, universal_newlines=False, lines=None):
-        self.timed_out = False
-        self.lines = lines or []
-        self.crashed = False
-        self.started = False
-
-    def has_crashed(self):
-        return self.crashed
-
-    def read_stdout_line(self, deadline):
-        return self.lines.pop(0) + "\n"
-
-    def read_stdout(self, deadline, size):
-        first_line = self.lines[0]
-        if size > len(first_line):
-            self.lines.pop(0)
-            remaining_size = size - len(first_line) - 1
-            if not remaining_size:
-                return first_line + "\n"
-            return first_line + "\n" + self.read_stdout(deadline, remaining_size)
-        result = self.lines[0][:size]
-        self.lines[0] = self.lines[0][size:]
-        return result
-
-    def read_either_stdout_or_stderr_line(self, deadline):
-        # FIXME: We should have tests which intermix stderr and stdout lines.
-        return self.read_stdout_line(deadline), None
-
-    def start(self):
-        self.started = True
-
-    def stop(self, kill_directly=False):
-        return
-
-    def kill(self):
-        return
-
-
 if __name__ == '__main__':
     unittest.main()

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/efl.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/efl.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/efl.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -72,6 +72,7 @@
         return super(EflPort, self).default_timeout_ms()
 
     def clean_up_test_run(self):
+        super(EflPort, self).clean_up_test_run()
         self._restore_pulseaudio_module()
 
     def _generate_all_test_configurations(self):

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/gtk.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/gtk.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/gtk.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -51,6 +51,7 @@
         self._unload_pulseaudio_module()
 
     def clean_up_test_run(self):
+        super(GtkPort, self).clean_up_test_run()
         self._restore_pulseaudio_module()
 
     def setup_environ_for_server(self, server_name=None):

Added: trunk/Tools/Scripts/webkitpy/layout_tests/port/image_diff.py (0 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/image_diff.py	                        (rev 0)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/image_diff.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -0,0 +1,117 @@
+#!/usr/bin/env python
+# Copyright (C) 2010 Google Inc. All rights reserved.
+# Copyright (C) 2010 Gabor Rapcsanyi <[email protected]>, University of Szeged
+# Copyright (C) 2011, 2012 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+#     * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above
+# copyright notice, this list of conditions and the following disclaimer
+# in the documentation and/or other materials provided with the
+# distribution.
+#     * Neither the Google name nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""WebKit implementations of the Port interface."""
+
+import logging
+import re
+import time
+
+from webkitpy.layout_tests.port import server_process
+
+
+_log = logging.getLogger(__name__)
+
+
+class ImageDiffer(object):
+    def __init__(self, port):
+        self._port = port
+        self._tolerance = None
+        self._process = None
+
+    def diff_image(self, expected_contents, actual_contents, tolerance):
+        if tolerance != self._tolerance:
+            self.stop()
+        try:
+            assert(expected_contents)
+            assert(actual_contents)
+            assert(tolerance is not None)
+
+            if not self._process:
+                self._start(tolerance)
+            self._process.write('Content-Length: %d\n%sContent-Length: %d\n%s' % (
+                len(actual_contents), actual_contents,
+                len(expected_contents), expected_contents))
+            return self._read()
+        except IOError as exception:
+            _log.error("Failed to compute an image diff: %s" % str(exception))
+            return (True, 0)
+
+    def _start(self, tolerance):
+        command = [self._port._path_to_image_diff(), '--tolerance', str(tolerance)]
+        environment = self._port.setup_environ_for_server('ImageDiff')
+        self._process = self._port._server_process_constructor(self._port, 'ImageDiff', command, environment)
+        self._process.start()
+        self._tolerance = tolerance
+
+    def _read(self):
+        deadline = time.time() + 2.0
+        output = None
+        output_image = ""
+
+        while True:
+            output = self._process.read_stdout_line(deadline)
+            if self._process.timed_out or self._process.has_crashed() or not output:
+                break
+
+            if output.startswith('diff'):  # This is the last line ImageDiff prints.
+                break
+
+            if output.startswith('Content-Length'):
+                m = re.match('Content-Length: (\d+)', output)
+                content_length = int(m.group(1))
+                output_image = self._process.read_stdout(deadline, content_length)
+                output = self._process.read_stdout_line(deadline)
+                break
+
+        stderr = self._process.pop_all_buffered_stderr()
+        if stderr:
+            _log.warn("ImageDiff produced stderr output:\n" + stderr)
+        if self._process.timed_out:
+            _log.error("ImageDiff timed out")
+        if self._process.has_crashed():
+            _log.error("ImageDiff crashed")
+        # FIXME: There is no need to shut down the ImageDiff server after every diff.
+        self._process.stop()
+
+        diff_percent = 0
+        if output and output.startswith('diff'):
+            m = re.match('diff: (.+)% (passed|failed)', output)
+            if m.group(2) == 'passed':
+                return [None, 0]
+            diff_percent = float(m.group(1))
+
+        return (output_image, diff_percent)
+
+    def stop(self):
+        if self._process:
+            self._process.stop()
+            self._process = None

Added: trunk/Tools/Scripts/webkitpy/layout_tests/port/image_diff_unittest.py (0 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/image_diff_unittest.py	                        (rev 0)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/image_diff_unittest.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -0,0 +1,52 @@
+# Copyright (C) 2012 Google Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+#    * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#    * Redistributions in binary form must reproduce the above
+# copyright notice, this list of conditions and the following disclaimer
+# in the documentation and/or other materials provided with the
+# distribution.
+#    * Neither the name of Google Inc. nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""Unit testing base class for Port implementations."""
+
+import unittest
+
+from webkitpy.layout_tests.port.server_process_mock import MockServerProcess
+from webkitpy.layout_tests.port.image_diff import ImageDiffer
+
+
+class FakePort(object):
+    def __init__(self, server_process_output):
+        self._server_process_constructor = lambda port, nm, cmd, env: MockServerProcess(lines=server_process_output)
+
+    def _path_to_image_diff(self):
+        return ''
+
+    def setup_environ_for_server(self, nm):
+        return None
+
+
+class TestImageDiffer(unittest.TestCase):
+    def test_diff_image(self):
+        port = FakePort(['diff: 100% failed\n'])
+        image_differ = ImageDiffer(port)
+        self.assertEquals(image_differ.diff_image('foo', 'bar', 0.1), ('', 100.0))
Property changes on: trunk/Tools/Scripts/webkitpy/layout_tests/port/image_diff_unittest.py
___________________________________________________________________

Added: svn:executable

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/mock_drt_unittest.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/mock_drt_unittest.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/mock_drt_unittest.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -57,6 +57,9 @@
     def test_check_sys_deps(self):
         pass
 
+    def test_diff_image(self):
+        pass
+
     def test_uses_apache(self):
         pass
 

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/port_testcase.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/port_testcase.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/port_testcase.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -41,6 +41,7 @@
 from webkitpy.layout_tests.servers import http_server_base
 from webkitpy.layout_tests.port import factory
 from webkitpy.layout_tests.port.config_mock import MockConfig
+from webkitpy.layout_tests.port.server_process_mock import MockServerProcess
 from webkitpy.tool.mocktool import MockOptions
 
 
@@ -209,6 +210,7 @@
         self.assertFalse(port.diff_image(None, None)[0])
         self.assertFalse(port.diff_image(None, '')[0])
         self.assertFalse(port.diff_image('', None)[0])
+
         self.assertFalse(port.diff_image('', '')[0])
 
     def test_diff_image__missing_actual(self):
@@ -221,6 +223,22 @@
         self.assertTrue(port.diff_image('foo', None)[0])
         self.assertTrue(port.diff_image('foo', '')[0])
 
+    def test_diff_image(self):
+        port = self.make_port()
+        self.proc = None
+
+        def make_proc(port, nm, cmd, env):
+            self.proc = MockServerProcess(port, nm, cmd, env, lines=['diff: 100% failed\n'])
+            return self.proc
+
+        port._server_process_constructor = make_proc
+        port.setup_test_run()
+        self.assertEquals(port.diff_image('foo', 'bar'), ('', 100.0))
+        self.assertEquals(self.proc.cmd[1:3], ["--tolerance", "0.1"])
+        port.clean_up_test_run()
+        self.assertTrue(self.proc.stopped)
+        self.assertEquals(port._image_differ, None)
+
     def test_check_build(self):
         port = self.make_port()
         port.check_build(needs_http=True)

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/server_process.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/server_process.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/server_process.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -32,7 +32,6 @@
 import errno
 import logging
 import signal
-import subprocess
 import sys
 import time
 
@@ -99,12 +98,12 @@
         self._reset()
         # close_fds is a workaround for http://bugs.python.org/issue2320
         close_fds = not self._host.platform.is_win()
-        self._proc = subprocess.Popen(self._cmd, stdin=subprocess.PIPE,
-                                      stdout=subprocess.PIPE,
-                                      stderr=subprocess.PIPE,
-                                      close_fds=close_fds,
-                                      env=self._env,
-                                      universal_newlines=self._universal_newlines)
+        self._proc = self._host.executive.popen(self._cmd, stdin=self._host.executive.PIPE,
+            stdout=self._host.executive.PIPE,
+            stderr=self._host.executive.PIPE,
+            close_fds=close_fds,
+            env=self._env,
+            universal_newlines=self._universal_newlines)
         self._pid = self._proc.pid
         fd = self._proc.stdout.fileno()
         if not self._use_win32_apis:

Added: trunk/Tools/Scripts/webkitpy/layout_tests/port/server_process_mock.py (0 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/server_process_mock.py	                        (rev 0)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/server_process_mock.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -0,0 +1,77 @@
+# Copyright (C) 2012 Google Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+#    * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#    * Redistributions in binary form must reproduce the above
+# copyright notice, this list of conditions and the following disclaimer
+# in the documentation and/or other materials provided with the
+# distribution.
+#    * Neither the name of Google Inc. nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+class MockServerProcess(object):
+    def __init__(self, port_obj=None, name=None, cmd=None, env=None, universal_newlines=False, lines=None):
+        self.timed_out = False
+        self.lines = lines or []
+        self.crashed = False
+        self.writes = []
+        self.cmd = cmd
+        self.env = env
+        self.started = False
+        self.stopped = False
+
+    def write(self, bytes):
+        self.writes.append(bytes)
+
+    def has_crashed(self):
+        return self.crashed
+
+    def read_stdout_line(self, deadline):
+        return self.lines.pop(0) + "\n"
+
+    def read_stdout(self, deadline, size):
+        first_line = self.lines[0]
+        if size > len(first_line):
+            self.lines.pop(0)
+            remaining_size = size - len(first_line) - 1
+            if not remaining_size:
+                return first_line + "\n"
+            return first_line + "\n" + self.read_stdout(deadline, remaining_size)
+        result = self.lines[0][:size]
+        self.lines[0] = self.lines[0][size:]
+        return result
+
+    def pop_all_buffered_stderr(self):
+        return ''
+
+    def read_either_stdout_or_stderr_line(self, deadline):
+        # FIXME: We should have tests which intermix stderr and stdout lines.
+        return self.read_stdout_line(deadline), None
+
+    def start(self):
+        self.started = True
+
+    def stop(self, kill_directly=False):
+        self.stopped = True
+        return
+
+    def kill(self):
+        return

Modified: trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py (123892 => 123893)


--- trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py	2012-07-27 18:17:27 UTC (rev 123892)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py	2012-07-27 18:21:08 UTC (rev 123893)
@@ -34,11 +34,9 @@
 import itertools
 import logging
 import operator
-import re
-import time
 
-from webkitpy.common.system.executive import Executive, ScriptError
-from webkitpy.layout_tests.port import server_process, Port
+from webkitpy.common.system.executive import ScriptError
+from webkitpy.layout_tests.port import Port
 
 
 _log = logging.getLogger(__name__)
@@ -96,81 +94,6 @@
     def _build_driver_flags(self):
         return []
 
-    def diff_image(self, expected_contents, actual_contents, tolerance=None):
-        # Handle the case where the test didn't actually generate an image.
-        # FIXME: need unit tests for this.
-        if not actual_contents and not expected_contents:
-            return (None, 0)
-        if not actual_contents or not expected_contents:
-            # FIXME: It's not clear what we should return in this case.
-            # Maybe we should throw an exception?
-            return (True, 0)
-
-        process = self._start_image_diff_process(expected_contents, actual_contents, tolerance=tolerance)
-        return self._read_image_diff(process)
-
-    def _image_diff_command(self, tolerance=None):
-        # FIXME: There needs to be a more sane way of handling default
-        # values for options so that you can distinguish between a default
-        # value of None and a default value that wasn't set.
-        if tolerance is None:
-            if self.get_option('tolerance') is not None:
-                tolerance = self.get_option('tolerance')
-            else:
-                tolerance = 0.1
-
-        command = [self._path_to_image_diff(), '--tolerance', str(tolerance)]
-        return command
-
-    def _start_image_diff_process(self, expected_contents, actual_contents, tolerance=None):
-        command = self._image_diff_command(tolerance)
-        environment = self.setup_environ_for_server('ImageDiff')
-        process = server_process.ServerProcess(self, 'ImageDiff', command, environment)
-
-        process.write('Content-Length: %d\n%sContent-Length: %d\n%s' % (
-            len(actual_contents), actual_contents,
-            len(expected_contents), expected_contents))
-        return process
-
-    def _read_image_diff(self, sp):
-        deadline = time.time() + 2.0
-        output = None
-        output_image = ""
-
-        while True:
-            output = sp.read_stdout_line(deadline)
-            if sp.timed_out or sp.has_crashed() or not output:
-                break
-
-            if output.startswith('diff'):  # This is the last line ImageDiff prints.
-                break
-
-            if output.startswith('Content-Length'):
-                m = re.match('Content-Length: (\d+)', output)
-                content_length = int(m.group(1))
-                output_image = sp.read_stdout(deadline, content_length)
-                output = sp.read_stdout_line(deadline)
-                break
-
-        stderr = sp.pop_all_buffered_stderr()
-        if stderr:
-            _log.warn("ImageDiff produced stderr output:\n" + stderr)
-        if sp.timed_out:
-            _log.error("ImageDiff timed out")
-        if sp.has_crashed():
-            _log.error("ImageDiff crashed")
-        # FIXME: There is no need to shut down the ImageDiff server after every diff.
-        sp.stop()
-
-        diff_percent = 0
-        if output and output.startswith('diff'):
-            m = re.match('diff: (.+)% (passed|failed)', output)
-            if m.group(2) == 'passed':
-                return [None, 0]
-            diff_percent = float(m.group(1))
-
-        return (output_image, diff_percent)
-
     def _tests_for_other_platforms(self):
         # By default we will skip any directory under LayoutTests/platform
         # that isn't in our baseline search path (this mirrors what
@@ -201,7 +124,7 @@
         symbols = ''
         for path_to_module in self._modules_to_search_for_symbols():
             try:
-                symbols += self._executive.run_command([self.nm_command(), path_to_module], error_handler=Executive.ignore_error)
+                symbols += self._executive.run_command([self.nm_command(), path_to_module], error_handler=self._executive.ignore_error)
             except OSError, e:
                 _log.warn("Failed to run nm: %s.  Can't determine supported features correctly." % e)
         return symbols
_______________________________________________
webkit-changes mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to