Your message dated Wed, 02 Dec 2015 21:58:21 +0000 with message-id <[email protected]> and subject line Bug#806826: fixed in pyyaml 3.11-3 has caused the Debian Bug report #806826, regarding pyyaml does not support literals in unicode over codepoint 0xffff to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact [email protected] immediately.) -- 806826: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=806826 Debian Bug Tracking System Contact [email protected] with problems
--- Begin Message ---Source: pyyaml Severity: normal Tags: upstream patch Dear Maintainer, the yaml spec says that “The allowed character range explicitly excludes the surrogate block #xD800-#xDFFF, DEL #x7F, the C0 control block #x0-#x1F (except for #x9, #xA, and #xD), the C1 control block #x80-#x9F, #xFFFE, and #xFFFF.” however pyyaml has chosen to negate that check and apply it to only plane 0. This means that any yaml document that contains unicode literals in higher planes will fail to parse (and, on output, use the rather unfriendly \Uxxxxxxxx format). The attached patch fixes this in a minimally intrusive way, by extending the checks to cover the additional codepoints where appropriate. A better fix would be to use the check as the spec specifies it, but that would be a bigger change. I tried to file this upstream first, but was unable to find a way to report bugs there; sorry. -- System Information: Debian Release: stretch/sid APT prefers xenial-updates APT policy: (500, 'xenial-updates'), (500, 'xenial-security'), (500, 'xenial'), (100, 'xenial-backports') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.2.0-19-generic (SMP w/4 CPU cores) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system)Index: pyyaml-3.11/lib/yaml/emitter.py =================================================================== --- pyyaml-3.11.orig/lib/yaml/emitter.py +++ pyyaml-3.11/lib/yaml/emitter.py @@ -8,9 +8,13 @@ __all__ = ['Emitter', 'EmitterError'] +import sys + from error import YAMLError from events import * +has_ucs4 = sys.maxunicode > 0xffff + class EmitterError(YAMLError): pass @@ -701,7 +705,8 @@ class Emitter(object): line_breaks = True if not (ch == u'\n' or u'\x20' <= ch <= u'\x7E'): if (ch == u'\x85' or u'\xA0' <= ch <= u'\uD7FF' - or u'\uE000' <= ch <= u'\uFFFD') and ch != u'\uFEFF': + or u'\uE000' <= ch <= u'\uFFFD' + or ((not has_ucs4) or (u'\U00010000' <= ch < u'\U0010ffff'))) and ch != u'\uFEFF': unicode_characters = True if not self.allow_unicode: special_characters = True Index: pyyaml-3.11/lib/yaml/reader.py =================================================================== --- pyyaml-3.11.orig/lib/yaml/reader.py +++ pyyaml-3.11/lib/yaml/reader.py @@ -19,7 +19,9 @@ __all__ = ['Reader', 'ReaderError'] from error import YAMLError, Mark -import codecs, re +import codecs, re, sys + +has_ucs4 = sys.maxunicode > 0xffff class ReaderError(YAMLError): @@ -134,7 +136,10 @@ class Reader(object): self.encoding = 'utf-8' self.update(1) - NON_PRINTABLE = re.compile(u'[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD]') + if has_ucs4: + NON_PRINTABLE = re.compile(u'[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD\U00010000-\U0010ffff]') + else: + NON_PRINTABLE = re.compile(u'[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD]') def check_printable(self, data): match = self.NON_PRINTABLE.search(data) if match: Index: pyyaml-3.11/lib3/yaml/emitter.py =================================================================== --- pyyaml-3.11.orig/lib3/yaml/emitter.py +++ pyyaml-3.11/lib3/yaml/emitter.py @@ -698,7 +698,8 @@ class Emitter: line_breaks = True if not (ch == '\n' or '\x20' <= ch <= '\x7E'): if (ch == '\x85' or '\xA0' <= ch <= '\uD7FF' - or '\uE000' <= ch <= '\uFFFD') and ch != '\uFEFF': + or '\uE000' <= ch <= '\uFFFD' + or '\U00010000' <= ch < '\U0010ffff') and ch != '\uFEFF': unicode_characters = True if not self.allow_unicode: special_characters = True Index: pyyaml-3.11/lib3/yaml/reader.py =================================================================== --- pyyaml-3.11.orig/lib3/yaml/reader.py +++ pyyaml-3.11/lib3/yaml/reader.py @@ -134,7 +134,7 @@ class Reader(object): self.encoding = 'utf-8' self.update(1) - NON_PRINTABLE = re.compile('[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD]') + NON_PRINTABLE = re.compile('[^\x09\x0A\x0D\x20-\x7E\x85\xA0-\uD7FF\uE000-\uFFFD\U00010000-\U0010ffff]') def check_printable(self, data): match = self.NON_PRINTABLE.search(data) if match:
--- End Message ---
--- Begin Message ---Source: pyyaml Source-Version: 3.11-3 We believe that the bug you reported is fixed in the latest version of pyyaml, which is due to be installed in the Debian FTP archive. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to [email protected], and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Barry Warsaw <[email protected]> (supplier of updated pyyaml package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing [email protected]) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Format: 1.8 Date: Wed, 02 Dec 2015 16:29:22 -0500 Source: pyyaml Binary: python-yaml python-yaml-dbg python3-yaml python3-yaml-dbg Architecture: source amd64 Version: 3.11-3 Distribution: unstable Urgency: medium Maintainer: Barry Warsaw <[email protected]> Changed-By: Barry Warsaw <[email protected]> Description: python-yaml - YAML parser and emitter for Python python-yaml-dbg - YAML parser and emitter for Python (debug build) python3-yaml - YAML parser and emitter for Python3 python3-yaml-dbg - YAML parser and emitter for Python3 (debug build) Closes: 806826 Changes: pyyaml (3.11-3) unstable; urgency=medium . * Team upload. * d/control: Bump Standards-Version with no other changes necessary. * d/patches/support-high-codepoints.patch: Added to fix support for codepoints above 0xffff. Given by John R. Lenton. (Closes: #806826) * d/tests/{control,python2.sh,python3.sh}: Added to test the above patch since a unittest is problematic. Checksums-Sha1: 0c8527b560332c8e323aff2608af0cd435984af9 2248 pyyaml_3.11-3.dsc b5b0a0b54b7b7e2cc787bcf40ec08787a2239fb5 8232 pyyaml_3.11-3.debian.tar.xz 8efe84fe0260b6d9fc926deaf3eadd491d801f52 80208 python-yaml-dbg_3.11-3_amd64.deb 1c551fb7487f81a020067851cc2c22d96284798e 107936 python-yaml_3.11-3_amd64.deb f62e8fdd2097bf148becd4577f58ce82b0ac9d79 94532 python3-yaml-dbg_3.11-3_amd64.deb 2c13ac583e3239c519dfbbea6bb435a869577ab6 129014 python3-yaml_3.11-3_amd64.deb Checksums-Sha256: 068592696e2690ee8a24e83b4779c167b14ef67a8595bc545904c6eafe2b80b9 2248 pyyaml_3.11-3.dsc d04f11dc2a5018f79c4540e523b4f65788bc3f7b85e74fca8c51ccfc8ef9f82c 8232 pyyaml_3.11-3.debian.tar.xz 9bf638617bafad8bd85bb06ba387d3eea87ecad603a63ff3432bd89bfb6967b4 80208 python-yaml-dbg_3.11-3_amd64.deb ad7c5ba7152bd2a891fa0732c6c3e3f4c9fc406737890b37398f0be73b16a78b 107936 python-yaml_3.11-3_amd64.deb 7a076d1ec29da78686664d0d78b8a5e565f6b32a2536745d9e82181369444cd8 94532 python3-yaml-dbg_3.11-3_amd64.deb 964c6aaca99aa59171b685721c6d6d4c9d9c3cfcead95cb2ba6acbba06f99820 129014 python3-yaml_3.11-3_amd64.deb Files: e8eee475362fa45e1f3c6665fbacef4b 2248 python optional pyyaml_3.11-3.dsc 5ff5902d242faaa8708f9b6e7baadb90 8232 python optional pyyaml_3.11-3.debian.tar.xz 48b90a1e648e7d1ab9a4089d57309bc7 80208 debug extra python-yaml-dbg_3.11-3_amd64.deb 5a62c810395261996104ce15100ea4d3 107936 python optional python-yaml_3.11-3_amd64.deb 49e46249c1ae453417522e9079ab89e4 94532 debug extra python3-yaml-dbg_3.11-3_amd64.deb 0e769d86349834707e9ccdf2f3cc098a 129014 python optional python3-yaml_3.11-3_amd64.deb -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJWX2ZSAAoJEBJutWOnSwa/Z2EP/1hUPM/3OkVxI+pj/raE0XD5 1ZGpuj4iNZWwPc+dVmXLgQdER8OMRBFMkS/Ebn8ryq2w+ZHAXV2cMhmEC7G/1D6Q xiy0KvRacRfZZCzV2Ptf+Q5aKmSpnc95ix1cHqzT/hpPbRj/8Dw7PN9YdI0HbUrw LUnf2bBhe+74clswV9H0k55aHGivZ6lo7wiNjIjmKWL0HB3h9V1tlb9SQ9m+xk+Q 4Tzzwq+AsHLGA/89svgACd54LuzXXPgaDFqzhJ+vIXEDIdL1chYivMkN4yutrT+f DhQBugGRdY/ycHyfQ3qQlkq6x8biQBpHla1dlrckdjKJF5XRYPmq/zOpfO8ZtaP3 B/FoDyLAG+NfXuSZ3Q1Eo3mjYDpEIgq/iJwQupbHYkjah7tFLrzHFcQqo6H38f6Q 3uTbMWHV1E1ozl9EUyQPx+EObvCKdQvW2xJdQiaLTdpQhoXhOtOiiY9SQncxj6b9 D3rWrI8KQB0Inm6YUHKH3vPgsXy2N26BLO8DrddHeP+9QTj57/BFq0jsfLzGnyCA XURqK1vJGDwMyMKBOaXORWQjDXBVLU8tcTySvUkHQUNbopwIGFnqn+eqgURwZAZC vGKlzbw/GZXo922tNqKi1NAaC1F9ViE2urFXfdIoGOlzoSA+mDcZhm44ChMm+OUi I17f32l/LWk6qC85zyG1 =jaSE -----END PGP SIGNATURE-----
--- End Message ---
_______________________________________________ Python-modules-team mailing list [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/python-modules-team
