On 29.08.2014 13:22, Isaac Morland wrote: > On Fri, 29 Aug 2014, M.-A. Lemburg wrote: > >> On 29.08.2014 02:41, Stephen J. Turnbull wrote: >> Since Python allows working with lone surrogates in Unicode (they >> are valid code points) and we're using UTF-8 for marshal, we needed >> a way to make sure that Python 3 also optionally supports working >> with lone surrogates in such UTF-8 streams (nowadays called CESU-8: >> http://en.wikipedia.org/wiki/CESU-8). > > If I want that wouldn't I specify "cesu-8" as the encoding? > > i.e., instead of .decode ('utf-8') I would use .decode ('cesu-8'). Right > now, trying this I get > that cesu-8 is an unknown encoding but that could be changed without > affecting the behaviour of the > utf-8 codec.
Why write a new codec that's almost identical to the utf-8 codec, if you can get the same functionality by explicitly using a special error handler ? >From a maintenance POV that does not sound like a good approach. > It seems to me that .decode ('utf-8') should decode exactly and only valid > utf-8, including the > non-use of surrogate pairs as an intermediate encoding step. It does in Python 3. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-08-27: Released eGenix PyRun 2.0.1 ... http://egenix.com/go62 2014-09-19: PyCon UK 2014, Coventry, UK ... 21 days to go 2014-09-27: PyDDF Sprint 2014 ... 29 days to go eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com