#12849: django's development server raises an encoding exception when trying to
colorize non-ascii text
------------------------------------------------+---------------------------
Reporter: jype | Owner: nobody
Status: closed | Milestone: 1.2
Component: django-admin.py runserver | Version: SVN
Resolution: fixed | Keywords:
Stage: Accepted | Has_patch: 1
Needs_docs: 0 | Needs_tests: 0
Needs_better_patch: 0 |
------------------------------------------------+---------------------------
Comment (by kmtracey):
Huh. The description of file.encoding here:
http://docs.python.org/library/stdtypes.html?highlight=write#file.write
gave me the impression that Python would automatically use the file's
encoding to convert unicode passed to write(), but testing shows it's not
doing that. Even if it did, now that I think about it for the colorize
case, we're adding control bytes so I expect we need to do the encoding to
a bytestring prior to adding those bytes.
I'm not sure the committed fix is the best alternative, though. When
printing out management command errors I think it would be better to use
sys.stderr.encoding, if it exists, since that is where we are sending the
output. Windows for example won't be using utf-8 as the terminal encoding
(by default), so the fix as committed will result in unreadable output for
non-ASCII exception data on Windows. Better than an exception I suppose
but I think it would be better to attempt to use the right encoding, and
also specify replace rather than strict for error handling so that if the
data can't be encoded in the target charset then we still don't raise an
exception.
Also simply removing the `text = str(text)` call from the top of colorize
is probably just going to move the exception to the end of that routine,
when `return ('\x1b[%sm' % ';'.join(code_list)) + text` will implicitly
convert a unicode text value to str. Which brings me to: I'd still like to
understand when this problem crops up, for the colorize case. Unicode
query string parms are percent-encoded in output -- under what
circumstances is the server being asked to colorize unicode data
containing non-ASCII characters?
(Your test box sounds like the build bots that were all broken by a
[http://code.djangoproject.com/wiki/ExpectedTestFailures recent test
change]. The one I looked at -- it was an Ubuntu server also -- had no
language packs installed, so even though the locale it was supposedly
configured to use had an encoding of utf-8, python was falling back to
that highfalutin-sounding ASCII encoding as "preferred". I believe
installing the language pack for the specified locale language fixed it. )
--
Ticket URL: <http://code.djangoproject.com/ticket/12849#comment:13>
Django <http://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
