Title: [117049] trunk/Tools
- Revision
- 117049
- Author
- [email protected]
- Date
- 2012-05-15 03:16:35 -0700 (Tue, 15 May 2012)
Log Message
Teach webkit-patch to gracefully handle writes of raw utf-8 encoded strings to sys.stdout
https://bugs.webkit.org/show_bug.cgi?id=85809
webkit-patch overrides sys.stdout with a StreamWriter that ensures we write utf8
output to stdout, but StreamWriter will by default always encode the object that's
passed to write(). This blows up when the object is a raw string, already encoded
as utf8, and does not match the default behavior of sys.stdout.
To solve this in one place, instead of every single call-site that might write
raw strings, we extend the stdout-wrapper to assume raw strings are utf8 before
passing them on to StreamWriter.write(). This has the benefit of also making the
stacktrace easy to read in the case where this assumption does not hold.
The source of this particular bug was that Executive.run_and_throw_if_fail() will
tee the output of the subprocess, and in the case of running prepare-ChangeLog this
might produce non-ascii output, which gets written directly to sys.stdout as raw
utf8-encoded strings.
Reviewed by Dirk Pranke.
* Scripts/webkit-patch:
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (117048 => 117049)
--- trunk/Tools/ChangeLog 2012-05-15 10:09:31 UTC (rev 117048)
+++ trunk/Tools/ChangeLog 2012-05-15 10:16:35 UTC (rev 117049)
@@ -1,3 +1,28 @@
+2012-05-15 Tor Arne Vestbø <[email protected]>
+
+ Teach webkit-patch to gracefully handle writes of raw utf-8 encoded strings to sys.stdout
+
+ https://bugs.webkit.org/show_bug.cgi?id=85809
+
+ webkit-patch overrides sys.stdout with a StreamWriter that ensures we write utf8
+ output to stdout, but StreamWriter will by default always encode the object that's
+ passed to write(). This blows up when the object is a raw string, already encoded
+ as utf8, and does not match the default behavior of sys.stdout.
+
+ To solve this in one place, instead of every single call-site that might write
+ raw strings, we extend the stdout-wrapper to assume raw strings are utf8 before
+ passing them on to StreamWriter.write(). This has the benefit of also making the
+ stacktrace easy to read in the case where this assumption does not hold.
+
+ The source of this particular bug was that Executive.run_and_throw_if_fail() will
+ tee the output of the subprocess, and in the case of running prepare-ChangeLog this
+ might produce non-ascii output, which gets written directly to sys.stdout as raw
+ utf8-encoded strings.
+
+ Reviewed by Dirk Pranke.
+
+ * Scripts/webkit-patch:
+
2012-05-15 Tomasz Morawski <[email protected]>
[EFL][DRT] Split ewk_private.h file to multiple private files.
Modified: trunk/Tools/Scripts/webkit-patch (117048 => 117049)
--- trunk/Tools/Scripts/webkit-patch 2012-05-15 10:09:31 UTC (rev 117048)
+++ trunk/Tools/Scripts/webkit-patch 2012-05-15 10:16:35 UTC (rev 117049)
@@ -43,13 +43,28 @@
from webkitpy.common.system.logutils import configure_logging
from webkitpy.tool.main import WebKitPatch
+# A StreamWriter will by default try to encode all objects passed
+# to write(), so when passed a raw string already encoded as utf8,
+# it will blow up with an UnicodeDecodeError. This does not match
+# the default behaviour of writing to sys.stdout, so we intercept
+# the case of writing raw strings and make sure StreamWriter gets
+# input that it can handle.
+class ForgivingUTF8Writer(codecs.lookup('utf-8')[-1]):
+ def write(self, object):
+ if isinstance(object, str):
+ # Assume raw strings are utf-8 encoded. If this line
+ # fails with an UnicodeDecodeError, our assumption was
+ # wrong, and the stacktrace should show you where we
+ # write non-Unicode/UTF-8 data (which we shouldn't).
+ object = object.decode('utf-8')
+ return codecs.StreamWriter.write(self, object)
+
# By default, sys.stdout assumes ascii encoding. Since our messages can
# contain unicode strings (as with some peoples' names) we need to apply
# the utf-8 codec to prevent throwing and exception.
# Not having this was the cause of https://bugs.webkit.org/show_bug.cgi?id=63452.
-sys.stdout = codecs.lookup('utf-8')[-1](sys.stdout)
+sys.stdout = ForgivingUTF8Writer(sys.stdout)
-
_log = logging.getLogger("webkit-patch")
def main():
_______________________________________________
webkit-changes mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo.cgi/webkit-changes
