[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor  added the comment:

> I prefer to close the issue and wait until more users ask for it before 
> considering again the patch, or find a different way to implement the feature 
> (support LC_NUMERIC and LC_CTYPE locales using a different encoding).

Here we are: https://bugs.python.org/issue31900

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor added the comment:

The issue has a workaround: use LC_NUMERIC and LC_CTYPE locales which use the 
same encoding. To avoid issues, it's probably safer to only use UTF-8 locales, 
which are now available on modern Linux distro.

I don't like the idea of calling setlocale() just for this corner case, because 
it changes the locale for all threads. Even if Python is protected by the GIL, 
Python can be embedded or modules written in C may spawn threads which don't 
care of the GIL. Usually, if it can fail, it will fail :-)

I see that various people contributed to the issue, but it looks like the only 
user asking for the request is Stefan Krah. I prefer to close the issue and 
wait until more users ask for it before considering again the patch, or find a 
different way to implement the feature (support LC_NUMERIC and LC_CTYPE locales 
using a different encoding).

To be clear, I'm closing the issue right now.

--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Well, I originally opened this issue but personally I'm not that
bothered by it any more.

Victor, do you want to keep it open?

--
versions: +Python 3.2 -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

STINNER Victor  wrote:
> What is this locale_t type used for the locale parameter of mbstowcs_l()?
> Are you sure that it is a string? According to this patch, it looks like a 
> structure:
> http://www.winehq.org/pipermail/wine-cvs/2010-May/067264.html

Yes, the string was mainly for benchmarking. FreeBSD seems to have thread safe
versions (xlocale.h), that take a locale_t as the extra parameter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor added the comment:

What is this locale_t type used for the locale parameter of mbstowcs_l()? Are 
you sure that it is a string? According to this patch, it looks like a 
structure:
http://www.winehq.org/pipermail/wine-cvs/2010-May/067264.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Yes, I saw the comments. I'm still wondering if we should just write an
mbstowcs_l() function instead.

Even then, there would still be a small chance that a C extension that
creates its own thread picks up the wrong LC_CTYPE.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor added the comment:

@Stefan: Did you my comments on Rietveld?
http://bugs.python.org/review/7442/#ps1473

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Changes by Stefan Krah :


Added file: http://bugs.python.org/file32306/mbstowcs_l.c

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Here's a quick-and-dirty version of mbstowcs_l(). The difference in the
benchmark to the plain mbstowcs() is 0.82s vs 0.55s. In the context of
a Python function this is unlikely to be measurable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

STINNER Victor  wrote:
> "So it should happen somewhere in PyUnicode_DecodeLocaleAndSize(). Perhaps we 
> can create _PyUnicode_DecodeLocaleAndSize() which would take an LC_CTYPE 
> parameter?"
> 
> For this issue, it means that Python localeconv() will have to change the 
> LC_CTYPE locale many time, for each monetary and each number value. I prefer 
> your patch :-)

Windows and OS X have mbstowcs_l(), which takes a locale arg. Linux doesn't
(as far as I can see).  I agree this solution is ugly, but it probably won't
have an impact on benchmarks (naively assuming that setlocale() is fast).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Matej Cepl]

Matej Cepl added the comment:

On 22/10/13 17:32, Stefan Krah wrote:
> It looks like some other test in the test suite did not restore a changed
> locale.  If you're motivated enough, you could try if it still happens in
> 3.4 and open a new issue (it's unrelated to this one).

Very plain checkout of git and ./configure && make && make test leads to
another failed test, but not this one (issue #19353). So either we do
something wrong in all those Fedora patches, or this has been fixed
since 3.3.2.

Curiouser and curiouser.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor added the comment:

"So it should happen somewhere in PyUnicode_DecodeLocaleAndSize(). Perhaps we 
can create _PyUnicode_DecodeLocaleAndSize() which would take an LC_CTYPE 
parameter?"

For this issue, it means that Python localeconv() will have to change the 
LC_CTYPE locale many time, for each monetary and each number value. I prefer 
your patch :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Matej Cepl  wrote:
> santiago:optimized (el6) $ ./python -m test test_format
> [1/1] test_format
> 1 test OK.

It looks like some other test in the test suite did not restore a changed
locale.  If you're motivated enough, you could try if it still happens in
3.4 and open a new issue (it's unrelated to this one).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Victor, thanks for the comments.  I also think we should set LC_CTYPE closer
to the actual call to mbstowcs(), otherwise there are many API calls in
between.

So it should happen somewhere in PyUnicode_DecodeLocaleAndSize(). Perhaps
we can create _PyUnicode_DecodeLocaleAndSize() which would take an LC_CTYPE
parameter?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

STINNER Victor  wrote:
> The following PostgreSQL issue looks to be the same than this Python issue:
> http://www.postgresql.org/message-id/20100422015552.4b7e0754...@cvs.postgresql.org
> 
> The fix changes temporarly the LC_CTYPE encoding:

So does my patch. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Matej Cepl]

Matej Cepl added the comment:

What I did:

1) run the build (it is a building of Fedora Rawhide python3 package on RHEL), 
and see it failed in this test.
2) see below

santiago:python3 (el6) $ cd Python-3.3.2/build/optimized/
santiago:optimized (el6) $ ./python -m test test_format
[1/1] test_format
1 test OK.
santiago:optimized (el6) $ 

Oh well

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor added the comment:

msg95988> Hi, the following works in 2.7 but not in 3.x: ...

Sure it works because Python 2 pass the raw byte string, it does not try to 
decode it. But did you try to display the result in a terminal for example?

Example with Python 2 in an UTF-8 terminal:

$ python
Python 2.7.5 (default, Oct  8 2013, 12:19:40) 
[GCC 4.8.1 20130603 (Red Hat 4.8.1-1)] on linux2
>>> import locale
>>> # set the locale encoding to UTF-8
... locale.setlocale(locale.LC_CTYPE, 'fr_FR.utf8')
'fr_FR.utf8'
>>> # set the thousand separator to U+00A0
... locale.setlocale(locale.LC_NUMERIC, 'fi_FI')
'fi_FI'
>>> locale.getlocale(locale.LC_CTYPE)
('fr_FR', 'UTF-8')
>>> locale.getlocale(locale.LC_NUMERIC)
('fi_FI', 'ISO8859-15')
>>> locale.format('%d', 123456, True)
'123\xa0456'
>>> print(locale.format('%d', 123456, True))
123�456

Mojibake! � means that b'\xA0' cannot be decoded from the locale encoding 
(UTF-8).


There is probably the same issue with a LC_MONETARY using a different encoding 
than LC_CTYPE.


> I suspect that this is related: #5905

It is unrelated: time.strftime() uses the LC_CTYPE, but the Python was using 
the wrong encoding. Python used the locale encoding read at startup, whereas 
the *current* locale encoding must be used.

This issue is specific to LC_NUMERIC with a LC_CTYPE using different encoding.


> If I set LC_CTYPE and LC_NUMERIC together, things work.

Sure, because in this case, LC_NUMERIC produces data in the same encoding than 
LC_CTYPE.


> call setlocale(LC_CTYPE, setlocale(LC_NUMERIC, NULL)) before
> mbstowcs. This is not really an option.

Setting a locale is process-wide and should be avoided. FYI 
locale.getpreferredencoding() changes temporarly the LC_CTYPE by default, it 
only uses the current LC_CTYPE if you pass False. open() changed temporarly 
LC_CTYPE because of that in Python 3.0-3.2 (see issue #11022).

The following PostgreSQL issue looks to be the same than this Python issue:
http://www.postgresql.org/message-id/20100422015552.4b7e0754...@cvs.postgresql.org

The fix changes temporarly the LC_CTYPE encoding:

#ifdef WIN32
setlocale(LC_CTYPE, locale_monetary);
#endif

(I don't know why the code is specific to Windows.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor added the comment:

Title: _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

Oh, I just realized that the issue is a LC_NUMERIC using an encoding A with a 
LC_CTYPE using an encoding B. It looks like the glibc does not support this 
setup, at least for the fi_FI locale which has a non-ASCII thousand separator 
(non-breaking space: U+00A0).

Try attached inconsistent_locale_encodings.py script (it uses locale names for 
Fedora 19, you may have to adapt it to your OS).


Output on Fedora 19:

fi_FI numeric (ISO-8859-1) with fr_FR.utf8 ctype (UTF-8)
UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: 
Virheellinen tai epätäydellinen monitavumerkki tai leveä merkki

fi_FI@euro numeric (ISO-8859-15) with fr_FR.utf8 ctype (UTF-8)
UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: 
Virheellinen tai epätäydellinen monitavumerkki tai leveä merkki

fi_FI.iso88591 numeric (ISO-8859-1) with fr_FR.utf8 ctype (UTF-8)
UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: 
Virheellinen tai epätäydellinen monitavumerkki tai leveä merkki

fi_FI.iso885915@euro numeric (ISO-8859-15) with fr_FR.utf8 ctype (UTF-8)
UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: 
Virheellinen tai epätäydellinen monitavumerkki tai leveä merkki

fi_FI.utf8 numeric (UTF-8) with fr_FR.utf8 ctype (UTF-8)
{'grouping': [3, 3, 0], 'p_cs_precedes': 0, 'mon_thousands_sep': '\xa0', 
'decimal_point': ',', 'n_sep_by_space': 1, 'n_sign_posn': 1, 
'mon_decimal_point': ',', 'frac_digits': 2, 'positive_sign': '', 
'mon_grouping': [3, 3, 0], 'n_cs_precedes': 0, 'thousands_sep': '\xa0', 
'p_sep_by_space': 1, 'p_sign_posn': 1, 'int_frac_digits': 2, 'currency_symbol': 
'€', 'negative_sign': '-', 'int_curr_symbol': 'EUR '}

--
Added file: http://bugs.python.org/file32303/inconsistent_locale_encodings.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Matej Cepl  wrote:
> >>> import locale
> >>> locale.localeconv()
> {'p_cs_precedes': 127, 'n_sep_by_space': 127, 'n_sign_posn': 127, 
> 'n_cs_precedes': 127, 'grouping': [], 'positive_sign': '', 'mon_grouping': 
> [], 'p_sep_by_space': 127, 'mon_thousands_sep': '', 'currency_symbol': '', 
> 'mon_decimal_point': '', 'int_curr_symbol': '', 'thousands_sep': '', 
> 'frac_digits': 127, 'int_frac_digits': 127, 'negative_sign': '', 
> 'decimal_point': '.', 'p_sign_posn': 127}
> >>> locale.setlocale(locale.LC_ALL, '')
> 'LC_CTYPE=en_US.utf8;LC_NUMERIC=en_IE.utf8;LC_TIME=en_IE.utf8;LC_COLLATE=en_US.utf8;LC_MONETARY=en_IE.utf8;LC_MESSAGES=en_US.utf8;LC_PAPER=en_IE.utf8;LC_NAME=en_US.utf8;LC_ADDRESS=en_US.utf8;LC_TELEPHONE=en_US.utf8;LC_MEASUREMENT=en_IE.utf8;LC_IDENTIFICATION=en_US.utf8'
> >>> locale.localeconv()
> {'p_cs_precedes': 1, 'n_sep_by_space': 0, 'n_sign_posn': 1, 'n_cs_precedes': 
> 1, 'grouping': [3, 3, 0], 'positive_sign': '', 'mon_grouping': [3, 3, 0], 
> 'p_sep_by_space': 0, 'mon_thousands_sep': ',', 'currency_symbol': '€', 
> 'mon_decimal_point': '.', 'int_curr_symbol': 'EUR ', 'thousands_sep': ',', 
> 'frac_digits': 2, 'int_frac_digits': 2, 'negative_sign': '-', 
> 'decimal_point': '.', 'p_sign_posn': 1}

These look normal.  I'm puzzled, because that's what is going on in the test
as well.  Do you get the failure when running the test in isolation:

./python -m test test_format

If this passes, there might be some interaction with other tests.

If it doesn't pass, I would step through the test in gdb (break 
PyLocale_localeconv)
and see which member of struct lconv is the troublemaker.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Matej Cepl]

Matej Cepl added the comment:

Perhaps version of glibc might be interesting as well?

glibc-2.12-1.107.el6_4.5.i686

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Matej Cepl]

Matej Cepl added the comment:

>>> import locale
>>> locale.localeconv()
{'p_cs_precedes': 127, 'n_sep_by_space': 127, 'n_sign_posn': 127, 
'n_cs_precedes': 127, 'grouping': [], 'positive_sign': '', 'mon_grouping': [], 
'p_sep_by_space': 127, 'mon_thousands_sep': '', 'currency_symbol': '', 
'mon_decimal_point': '', 'int_curr_symbol': '', 'thousands_sep': '', 
'frac_digits': 127, 'int_frac_digits': 127, 'negative_sign': '', 
'decimal_point': '.', 'p_sign_posn': 127}
>>> locale.setlocale(locale.LC_ALL, '')
'LC_CTYPE=en_US.utf8;LC_NUMERIC=en_IE.utf8;LC_TIME=en_IE.utf8;LC_COLLATE=en_US.utf8;LC_MONETARY=en_IE.utf8;LC_MESSAGES=en_US.utf8;LC_PAPER=en_IE.utf8;LC_NAME=en_US.utf8;LC_ADDRESS=en_US.utf8;LC_TELEPHONE=en_US.utf8;LC_MEASUREMENT=en_IE.utf8;LC_IDENTIFICATION=en_US.utf8'
>>> locale.localeconv()
{'p_cs_precedes': 1, 'n_sep_by_space': 0, 'n_sign_posn': 1, 'n_cs_precedes': 1, 
'grouping': [3, 3, 0], 'positive_sign': '', 'mon_grouping': [3, 3, 0], 
'p_sep_by_space': 0, 'mon_thousands_sep': ',', 'currency_symbol': '€', 
'mon_decimal_point': '.', 'int_curr_symbol': 'EUR ', 'thousands_sep': ',', 
'frac_digits': 2, 'int_frac_digits': 2, 'negative_sign': '-', 'decimal_point': 
'.', 'p_sign_posn': 1}
>>>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[STINNER Victor]

STINNER Victor added the comment:

This issue is very close to the issue #13706 which I solved with the new 
function PyUnicode_DecodeLocale(): see get_locale_info() in 
Python/formatter_unicode.c.

We might copy/paste the code, or we should maybe add a private API to get 
locale information: get_locale_info() => _Py_get_locale_info() and expose the 
LocaleInfo structure. It may be added to unicodeobject.h for example, there is 
already a function related to locales: _PyUnicode_InsertThousandsGrouping().

--
nosy: +haypo
versions: +Python 3.3, Python 3.4 -Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Matej Cepl  wrote:
> To #200912: now, system locale is UTF-8 all the way:
> santiago:python3 (el6) $ locale
> LANG=en_US.utf8
> LC_CTYPE="en_US.utf8"
> LC_NUMERIC=en_IE.utf8
> LC_TIME=en_IE.utf8
> LC_COLLATE="en_US.utf8"
> LC_MONETARY=en_IE.utf8
> LC_MESSAGES="en_US.utf8"
> LC_PAPER=en_IE.utf8
> LC_NAME="en_US.utf8"
> LC_ADDRESS="en_US.utf8"
> LC_TELEPHONE="en_US.utf8"
> LC_MEASUREMENT=en_IE.utf8
> LC_IDENTIFICATION="en_US.utf8"
> LC_ALL=
> santiago:python3 (el6) $

The test passes here with these values (Debian).

What is the output of:

  a) locale.localeconv()

  b) locale.setlocale(locale.LC_ALL, '')
 locale.localeconv()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Matej Cepl]

Matej Cepl added the comment:

To #200912: now, system locale is UTF-8 all the way:
santiago:python3 (el6) $ locale
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC=en_IE.utf8
LC_TIME=en_IE.utf8
LC_COLLATE="en_US.utf8"
LC_MONETARY=en_IE.utf8
LC_MESSAGES="en_US.utf8"
LC_PAPER=en_IE.utf8
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT=en_IE.utf8
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=
santiago:python3 (el6) $

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Matej Cepl]

Matej Cepl added the comment:

Hmm, so with this patch

diff -up Python-3.3.2/Lib/test/test_format.py.fixFormatTest 
Python-3.3.2/Lib/test/test_format.py
--- Python-3.3.2/Lib/test/test_format.py.fixFormatTest  2013-10-22 
10:05:12.253426746 +0200
+++ Python-3.3.2/Lib/test/test_format.py2013-10-22 10:16:58.510530570 
+0200
@@ -288,7 +288,7 @@ class FormatTest(unittest.TestCase):
 def test_locale(self):
 try:
 oldloc = locale.setlocale(locale.LC_ALL)
-locale.setlocale(locale.LC_ALL, '')
+locale.setlocale(locale.LC_ALL, 'ps_AF')
 except locale.Error as err:
 self.skipTest("Cannot set locale: {}".format(err))
 try:

(or any other explicit locale, I have tried also en_IE) test doesn't fail.

Using Python-3.3.2 on RHEL-6 (kernel 2.6.32-358.23.2.el6.i686).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah added the comment:

Matej Cepl  wrote:
> Is this the same as when tests with python-3.3.2 fails on me with RHEL-6?

If LC_CTYPE is UTF-8 and LC_NUMERIC something like ISO-8859-2 then it's
the same issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Matej Cepl]

Matej Cepl added the comment:

Is this the same as when tests with python-3.3.2 fails on me with RHEL-6?

test_locale (test.test_format.FormatTest) ... ERROR
test_non_ascii (test.test_format.FormatTest) ... test test_format failed
'\u20ac=%f' % (1.0,) =? '\u20ac=1.00' ... yes
ok

==
ERROR: test_locale (test.test_format.FormatTest)
--
Traceback (most recent call last):
  File "/home/matej/build/Extras/python3/Python-3.3.2/Lib/test/test_format.py", 
line 295, in test_locale
localeconv = locale.localeconv()
  File "/home/matej/build/Extras/python3/Python-3.3.2/Lib/locale.py", line 111, 
in localeconv
d = _localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0xe2 in position 0: 
Invalid or incomplete multibyte or wide character

--

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Mark Lawrence]

Mark Lawrence added the comment:

Could we have a patch review please.  Also note that #5905 has been closed.

--
nosy: +BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah  added the comment:

I have a patch that fixes this specific issue. Probably there are similar
issues in other places, e.g. when LC_TIME and LC_CTYPE differ.

I suspect that this is related:

http://bugs.python.org/issue5905

--
keywords: +patch
Added file: http://bugs.python.org/file16221/set_ctype_before_mbstowcs.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +loewis
priority:  -> normal
stage:  -> needs patch
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7442] _localemodule.c: str2uni() with different LC_NUMERIC and LC_CTYPE

[Stefan Krah]

Stefan Krah  added the comment:

Changed title (was: decimal.py: format failure with locale specifier)

--
title: decimal.py: format failure with locale specifier -> _localemodule.c: 
str2uni() with different LC_NUMERIC and LC_CTYPE

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com