Re: [Python-Dev] locale and LC_NUMERIC
On 1/8/06, Martin v. Löwis [EMAIL PROTECTED] wrote: Neal Norwitz wrote: I know very little about locale's. /f assigned me a bug http://python.org/sf/1391872 which suggests I run all the tests in a different locale than C. I think this is a good idea, but when I set LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without me calling locale.setlocale(). Shouldn't we maintain the value from the env't? I feel I'm lacking some link here: why do you think we should do that? [EMAIL PROTECTED] ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python import locale locale.setlocale(locale.LC_ALL) 'C' locale.setlocale(locale.LC_ALL, 'de_DE') 'de_DE' I would have expected the first call to setlocale() to return de_DE. /f solution of adding a flag to the test runner should work for the bug report I mentioned. Part of the reason why I asked was because I'm confused by the current behaviour. Is the first call to setlocale() supposed to return C? If so, always? I was looking for some way for the initial call to setlocale() to return de_DE or whatever other locale I set. What does start with the correct locale mean? What is the correct locale? Python should work in any locale, Hopefully my explanation above answered all those questions. and locale.format should always give the locale-specific formatting, whereas %f % value should always give the result from the C locale (regardless of whether the application has called setlocale or not). Right. I'm working on a patch for this problem which is independent of what I was asking. I really should have started a new thread, but I wasn't completely sure when I asked if there was a relationship. Also, in Georg's snippet, he didn't show that he was setting the locale, only what it was set to. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
Neal Norwitz wrote: I feel I'm lacking some link here: why do you think we should do that? [EMAIL PROTECTED] ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python import locale locale.setlocale(locale.LC_ALL) 'C' locale.setlocale(locale.LC_ALL, 'de_DE') 'de_DE' I would have expected the first call to setlocale() to return de_DE. the locale is a program-specific setting (handled by the C runtime library), and is set to C by default. if you want to use a specific locale, you have to call setlocale to indicate that you really want a non-default locale. the argument is just a con- venience; it tells the C runtime library to set the locale based on OS-level language settings (e.g. LANG on Unix, control panel settings on Windows, etc). LANG in itself isn't the locale; it's just a way to tell a locale-aware program what the preferred locale is. /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
On 1/8/06, Martin v. Löwis [EMAIL PROTECTED] wrote: Georg Brandl wrote: import locale locale.setlocale(locale.LC_NUMERIC, ) '[EMAIL PROTECTED]' u%f % 1.0 u'1,00' Is this intended? This breaks test_format on my box when test_builtin (method test_float_with_comma) is executed first. No. %-formatting should always use the C locale. One should use locale.format to get locale-aware formatting. http://python.org/sf/1400181 fixes this problem I hope. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
Fredrik Lundh wrote: my rationale for running the tests with a non-US locale was to increase the chance of catching bugs where the external locale setting affects Python's behaviour. maybe it would be a good idea to add a use setlocale flag to the test- runner ? Sounds like a good use for buildbot. After running the tests normally they could be re-run with a different locale. -- Benji York ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
Martin v. Löwis wrote: Georg Brandl wrote: import locale locale.setlocale(locale.LC_NUMERIC, ) '[EMAIL PROTECTED]' %f % 1.0 '1.00' u%f % 1.0 u'1,00' Is this intended? This breaks test_format on my box when test_builtin (method test_float_with_comma) is executed first. No. %-formatting should always use the C locale. One should use locale.format to get locale-aware formatting. While we're at it: test_builtin does import locale orig_locale = locale.setlocale(locale.LC_NUMERIC, '') locale.setlocale(locale.LC_NUMERIC, 'fr_FR') and later finally: locale.setlocale(locale.LC_NUMERIC, orig_locale) Shouldn't the first setlocale have no second argument? regards, Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
Neal Norwitz wrote: [EMAIL PROTECTED] ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python import locale locale.setlocale(locale.LC_ALL) 'C' locale.setlocale(locale.LC_ALL, 'de_DE') 'de_DE' I would have expected the first call to setlocale() to return de_DE. No, it shouldn't. We are providing C semantics here, which is that no locale functionality is activated unless the application explicitly asks for it. Is the first call to setlocale() supposed to return C? If so, always? The return value is implementation-defined, but yes, it is supposed to return the C locale (which is synonymous to the POSIX locale on a POSIX system). I was looking for some way for the initial call to setlocale() to return de_DE or whatever other locale I set. That shouldn't happen. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
Georg Brandl wrote: import locale orig_locale = locale.setlocale(locale.LC_NUMERIC, '') locale.setlocale(locale.LC_NUMERIC, 'fr_FR') and later finally: locale.setlocale(locale.LC_NUMERIC, orig_locale) Shouldn't the first setlocale have no second argument? Oops, right - it certainly should. Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
On 1/9/06, Martin v. Löwis [EMAIL PROTECTED] wrote: I would have expected the first call to setlocale() to return de_DE. No, it shouldn't. We are providing C semantics here, which is that no locale functionality is activated unless the application explicitly asks for it. Thanks (to /f too). That was the part that I was missing. It makes sense now. I made the patch to do this. It's called before each class' test run in regrtest.py. I would prefer to set the locale before each test method run, but I need to muck with unittest for that IIRC. There were 2 failures. One was in test_email which needs the C locale for generating the proper timestamp. The other was (IIRC) in test_re, 2 methods failed. So we are hopefully in decent shape. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
Georg Brandl wrote: import locale locale.setlocale(locale.LC_NUMERIC, ) '[EMAIL PROTECTED]' %f % 1.0 '1.00' u%f % 1.0 u'1,00' Is this intended? This breaks test_format on my box when test_builtin (method test_float_with_comma) is executed first. No. %-formatting should always use the C locale. One should use locale.format to get locale-aware formatting. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
On 1/8/06, Martin v. Löwis [EMAIL PROTECTED] wrote: Georg Brandl wrote: %f % 1.0 '1.00' u%f % 1.0 u'1,00' Is this intended? This breaks test_format on my box when test_builtin (method test_float_with_comma) is executed first. No. %-formatting should always use the C locale. One should use locale.format to get locale-aware formatting. I know very little about locale's. /f assigned me a bug http://python.org/sf/1391872 which suggests I run all the tests in a different locale than C. I think this is a good idea, but when I set LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without me calling locale.setlocale(). Shouldn't we maintain the value from the env't? The problem I found is in Python/pythonrun.c. At least I think that's the problem. Around line 230-240 we call setlocale(NULL), then call setlocale(). Then later on we restore the locale by calling setlocale() with the return result from setlocale(NULL). The man page on my box (gentoo) says that passing NULL returns the current locale() while passing sets the locale from the env't variable. Should we just capture the results of setlocale()? If my understanding is wrong, how can I start python with the correct locale? It seems like that I'm confused since this problem hasn't seemed to come up before. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locale and LC_NUMERIC
Neal Norwitz wrote: No. %-formatting should always use the C locale. One should use locale.format to get locale-aware formatting. I know very little about locale's. /f assigned me a bug http://python.org/sf/1391872 which suggests I run all the tests in a different locale than C. I think this is a good idea, but when I set LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without me calling locale.setlocale(). Shouldn't we maintain the value from the env't? the locale is a global program setting, and should only be activated when it's actually needed. my rationale for running the tests with a non-US locale was to increase the chance of catching bugs where the external locale setting affects Python's behaviour. maybe it would be a good idea to add a use setlocale flag to the test- runner ? /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
