STINNER Victor added the comment:
msg95988> Hi, the following works in 2.7 but not in 3.x: ...
Sure it works because Python 2 pass the raw byte string, it does not try to
decode it. But did you try to display the result in a terminal for example?
Example with Python 2 in an UTF-8 terminal:
$ python
Python 2.7.5 (default, Oct 8 2013, 12:19:40)
[GCC 4.8.1 20130603 (Red Hat 4.8.1-1)] on linux2
>>> import locale
>>> # set the locale encoding to UTF-8
... locale.setlocale(locale.LC_CTYPE, 'fr_FR.utf8')
'fr_FR.utf8'
>>> # set the thousand separator to U+00A0
... locale.setlocale(locale.LC_NUMERIC, 'fi_FI')
'fi_FI'
>>> locale.getlocale(locale.LC_CTYPE)
('fr_FR', 'UTF-8')
>>> locale.getlocale(locale.LC_NUMERIC)
('fi_FI', 'ISO8859-15')
>>> locale.format('%d', 123456, True)
'123\xa0456'
>>> print(locale.format('%d', 123456, True))
123�456
Mojibake! � means that b'\xA0' cannot be decoded from the locale encoding
(UTF-8).
There is probably the same issue with a LC_MONETARY using a different encoding
than LC_CTYPE.
> I suspect that this is related: #5905
It is unrelated: time.strftime() uses the LC_CTYPE, but the Python was using
the wrong encoding. Python used the locale encoding read at startup, whereas
the *current* locale encoding must be used.
This issue is specific to LC_NUMERIC with a LC_CTYPE using different encoding.
> If I set LC_CTYPE and LC_NUMERIC together, things work.
Sure, because in this case, LC_NUMERIC produces data in the same encoding than
LC_CTYPE.
> call setlocale(LC_CTYPE, setlocale(LC_NUMERIC, NULL)) before
> mbstowcs. This is not really an option.
Setting a locale is process-wide and should be avoided. FYI
locale.getpreferredencoding() changes temporarly the LC_CTYPE by default, it
only uses the current LC_CTYPE if you pass False. open() changed temporarly
LC_CTYPE because of that in Python 3.0-3.2 (see issue #11022).
The following PostgreSQL issue looks to be the same than this Python issue:
http://www.postgresql.org/message-id/[email protected]
The fix changes temporarly the LC_CTYPE encoding:
#ifdef WIN32
setlocale(LC_CTYPE, locale_monetary);
#endif
(I don't know why the code is specific to Windows.)
----------
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue7442>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
