On Sat, Sep 27, 2008 at 2:10 PM, Arnar Birgisson <[EMAIL PROTECTED]> wrote: > On Sat, Sep 27, 2008 at 22:13, Bob Ippolito <[EMAIL PROTECTED]> wrote: >> If you give it unicode input, it will decode to unicode. Basically it >> scans through the str until it finds non-ASCII, escape, or end quote. >> If it finds the end quote first it will just allocate a new string >> with exactly that day, which is super fast since it's just an alloc >> and copy. >> >> It will of course always decode everything containing non-ASCII >> characters or any escape sequences to unicode. It is not currently >> configurable. It was done for performance, but also does produce nicer >> looking repr output because you don't have so many 'u' characters to >> look at :) Given the way str works in Python 2.x it should not be an >> incompatible change except for doctests... and I guess code that >> explicitly checks for unicode and doesn't know what to do with str, >> but that would be weird. > > The reason I asked was because I've had problems even with pure-ASCII > strs when mixed with unicode objects in some DB-API drivers, working > with filesystems on the OS-X and others. A "solution" was to have > everything in unicode.
I've never seen a pure ASCII str cause problems. I've seen pure ASCII unicode cause problems in stupid ways though, because not all Python C code that handles text can handle unicode. Dumb stuff like this bites me all the time in Genshi templates (where all string literals are unicode): >>> datetime.datetime.now().strftime(u'%Y') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: strftime() argument 1 must be str, not unicode Any operation involving a str and unicode should up-convert to unicode, and regardless of the defaultencoding a pure ASCII str will properly get handled (at least in Python 2.5, I don't remember what 2.4 did)... e.g. ''.join(['', u'foo']) returns u'foo' > Since that means the string given to simplejson to decode will be a > unicode string anyways, so in that case there's no problem :) If you can prove that there is an actual problem I'm sure I can come up with a flag that would ensure unicode, but the implementation would probably be just translation of the input document to unicode before decoding ;) -bob _______________________________________________ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
