Roundup Robot added the comment:
New changeset 3d805bee06e2 by Serhiy Storchaka in branch '2.7':
Issue #5815: Fixed support for locales with modifiers. Fixed support for
http://hg.python.org/cpython/rev/3d805bee06e2
New changeset 28883e89f335 by Serhiy Storchaka in branch '3.3':
Issue #5815:
Serhiy Storchaka added the comment:
Committed without devanagari special case and tests.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
Serhiy Storchaka added the comment:
For devanagari modifier opened new issue20027.
--
resolution: - fixed
stage: patch review - committed/rejected
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
STINNER Victor added the comment:
Buildbot failure:
http://buildbot.python.org/all/builders/x86%20Gentoo%20Non-Debug%203.3/builds/1314/steps/test/logs/stdio
==
ERROR: test_locale_alias (test.test_locale.NormalizeTest)
Roundup Robot added the comment:
New changeset e0675408f4af by Serhiy Storchaka in branch '2.7':
Don't use sebTest() in tests for issue #5815.
http://hg.python.org/cpython/rev/e0675408f4af
New changeset ed16f6695638 by Serhiy Storchaka in branch '3.3':
Don't use sebTest() in tests for issue
Serhiy Storchaka added the comment:
Oh, thanks Victor.
--
resolution: - fixed
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
Serhiy Storchaka added the comment:
Marc-Andre, do you have comments or objections?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
___
Marc-Andre Lemburg added the comment:
On 18.12.2013 22:57, Serhiy Storchaka wrote:
Marc-Andre, do you have comments or objections?
Your last patch looks fine, but I don't have time to test it.
Regarding the broken *devanagari* entries in the alias table:
I think we should remove or correct
Mike FABIAN added the comment:
Serhiy While normalize can return sd...@devanagari.utf-8, _parse_localename()
Serhiy should be able correctly parse it.
But if normalize returns sd...@devanagari.utf-8, isn’t that quite
useless because it is a locale name which does not actually work
in glibc?
Marc-Andre Lemburg added the comment:
Then I don't understand changes such as:
-'chinese-s':'zh_CN.eucCN',
+'chinese-s':'zh_CN.gb2312',
or
-'sp': 'sr_CS.ISO8859-5',
-'sp_yu':
Serhiy Storchaka added the comment:
That's not intended. The normalize() function is supposed to
prepare the locale for the lookup. It's not supposed to be applied
to the looked up value.
Last patch doesn't contain this part of tests.
--
___
Python
Serhiy Storchaka added the comment:
There are no such systems really, in X.org this is just a mistake.
glibc doesn’t write it like this and it is agains the specification
here:
While normalize can return sd...@devanagari.utf-8, _parse_localename() should
be able correctly parse it. Removing
Marc-Andre Lemburg added the comment:
On 11.11.2013 20:21, Serhiy Storchaka wrote:
That's not intended. The normalize() function is supposed to
prepare the locale for the lookup. It's not supposed to be applied
to the looked up value.
Last patch doesn't contain this part of tests.
Mike FABIAN added the comment:
Serhiy, in your patch you seem to have special treatment for
the devanagari modifier:
+# Devanagari modifier placed before encoding.
+return code, modifier.split('.')[1]
Probably because of
'ks_in@devanagari':
Serhiy Storchaka added the comment:
The /usr/share/X11/locale/locale.alias file in Ubuntu 12.04 LTS contains
ks...@devanagari.utf-8 and sd...@devanagari.utf-8 entities. While the encoding
is expected to be before the modifier, if there are systems with
ks...@devanagari.utf-8 or
Mike FABIAN added the comment:
Serhiy The /usr/share/X11/locale/locale.alias file in Ubuntu 12.04 LTS
Serhiy contains ks...@devanagari.utf-8 and sd...@devanagari.utf-8
Serhiy entities.
Yes, I know, that’s why I wrote that the Python code inherited this mistake
from X.org.
Serhiy While the
Mike FABIAN added the comment:
In glibc, sd...@devanagari.utf-8 is an invalid locale name,
only sd_IN.UTF-8@devanagari is valid:
mfabian@ari:~
$ LC_ALL=sd_IN.UTF-8@devanagari locale charmap
UTF-8
mfabian@ari:~
$ LC_ALL=sd...@devanagari.utf-8 locale charmap
locale: Cannot set LC_CTYPE to default
Serhiy Storchaka added the comment:
Ping. There are two duplicate issues opened last month.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
Changes by STINNER Victor victor.stin...@gmail.com:
--
nosy: +haypo
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
___
Python-bugs-list
Serhiy Storchaka added the comment:
Patch updated. Added tests. The locale_alias mapping updated to be
self-consistency (i.e. for every name in locale_alias.values() normalize(name)
== name).
--
assignee: docs@python - serhiy.storchaka
keywords: -easy
nosy: +lemburg
stage: needs
R. David Murray added the comment:
It would be great if this could get a review by MAL, since it looks like a
non-trivial change.
Also, you have some (commented out) debug prints in there.
--
___
Python tracker rep...@bugs.python.org
Marc-Andre Lemburg added the comment:
On 13.09.2013 15:30, Serhiy Storchaka wrote:
Serhiy Storchaka added the comment:
Patch updated. Added tests. The locale_alias mapping updated to be
self-consistency (i.e. for every name in locale_alias.values()
normalize(name) == name).
Could you
Serhiy Storchaka added the comment:
Also, you have some (commented out) debug prints in there.
These debug prints were in old code.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
R. David Murray added the comment:
Ah, I see. I only scanned the patch quickly, obviously.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
Serhiy Storchaka added the comment:
Could you elaborate on the alias changes ?
Were those coming from an updated X11 local.alias file ?
No, they are not from X11 local.alias file. They are a result of the
test_locale_alias self-test, I have fixed all failures.
This test can't be backported
Serhiy Storchaka added the comment:
Here is a patch without changes to locale_alias.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
___
Changes by Serhiy Storchaka storch...@gmail.com:
Added file: http://bugs.python.org/file31742/locale_parse_2a.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
Marc-Andre Lemburg added the comment:
On 13.09.2013 16:34, Serhiy Storchaka wrote:
Serhiy Storchaka added the comment:
Could you elaborate on the alias changes ?
Were those coming from an updated X11 local.alias file ?
No, they are not from X11 local.alias file. They are a result of
Serhiy Storchaka added the comment:
Then I don't understand changes such as:
-'chinese-s':'zh_CN.eucCN',
+'chinese-s':'zh_CN.gb2312',
or
-'sp': 'sr_CS.ISO8859-5',
-'sp_yu':
Dmitry Jemerov added the comment:
A related issue (a case which isn't taken into account by Serhiy's patch) is
http://bugs.python.org/issue18378
--
nosy: +Dmitry.Jemerov
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
Changes by Serhiy Storchaka storch...@gmail.com:
--
versions: +Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
___
___
Serhiy Storchaka storch...@gmail.com added the comment:
Here is yet some inconsistency:
$ LANG=uk_ua.microsoftcp1251 ./python -c import locale;
print(locale.getdefaultlocale())
('uk_UA', 'CP1251')
$ LANG=uk_ua.microsoft-cp1251 ./python -c import locale;
print(locale.getdefaultlocale())
Serhiy Storchaka storch...@gmail.com added the comment:
Here is a complex patch for more careful locale parsing.
--
Added file: http://bugs.python.org/file26380/locale_parse.patch
___
Python tracker rep...@bugs.python.org
rg3 sarbalap+freshm...@gmail.com added the comment:
I don't know if the behavior is considered a bug or just undocumented, but
under Python 2.7.3 it's still the same. locale.getpreferredencoding() does
return UTF-8, but the second element in the tuple locale.getdefaultlocale() is
Serhiy Storchaka storch...@gmail.com added the comment:
The patch is not work for ca_ES@valencia locale.
And there are issues for such locales: ks_in@devanagari,
ks...@devanagari.utf-8, sd, sd...@devanagari.utf-8 (ks_in@devanagari in
locale_alias maps to ks...@devanagari.utf-8 and sd to
Greg Roodt gro...@gmail.com added the comment:
Bumping this as part of a bug scrub at EuroPython. Is this still an issue?
Should we fix in docs or in code?
--
nosy: +groodt
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
Changes by Ezio Melotti ezio.melo...@gmail.com:
--
keywords: +easy
versions: +Python 3.2, Python 3.3 -Python 2.6, Python 3.0, Python 3.1
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5815
New submission from rg3 sarbalap+freshm...@gmail.com:
A recent issue with one of my programs has shown that
locale.getdefaultlocale() does not handle correctly a corner case. The
issue URL is this one:
http://bitbucket.org/rg3/youtube-dl/issue/7/
Essentially, some users have LANG set to
rg3 sarbalap+freshm...@gmail.com added the comment:
I just realized that the if I introduced is not really needed.
encoding = encoding.split('@')[0] works whether the '@' symbol is
present or not.
--
___
Python tracker rep...@bugs.python.org
R. David Murray rdmur...@bitdance.com added the comment:
I wasn't able to reproduce this by just setting my LC_ALL environment
variable to es_ca.ut...@valencia and calling getdefaultlocale. Can you
provide more complete steps to reproduce?
--
nosy: +r.david.murray
priority: - normal
rg3 sarbalap+freshm...@gmail.com added the comment:
You are right. The issue is not reproduced with es_ca.ut...@valencia but
with ca_es.ut...@valencia. The fact that the first case works makes me
think maybe there's another way to solve the problem. Can you check that?
--
rg3 sarbalap+freshm...@gmail.com added the comment:
Further investigation:
The guy who had this issue may be from Valencia, Spain. According to the
manpage for setlocale(3) in my system, the form is usually
language[_territory][.codese...@modifier]. So, in this case, it would
make sense for the
R. David Murray rdmur...@bitdance.com added the comment:
OK, it turns out that this is one of a class of known bugs of long
standing (see issue554676 and issue1080864, for example). The
recommended solution is to not use locale.getdefaultlocale, but to use
locale.getperferredencoding. I have
rg3 sarbalap+freshm...@gmail.com added the comment:
Excellent. Thanks for the tip. I'll now proceed to modify my code to use
getpreferredencoding. Still, I think getdefaultlocale should work
because it could be used in other situations, I suppose.
--
44 matches
Mail list logo
