Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 9 May 2017 at 13:44, Nick Coghlan wrote: > On 8 May 2017 at 15:34, Nick Coghlan wrote: >> On 7 May 2017 at 15:22, INADA Naoki wrote: >>> ## Background >>> >>> Locale coercion in current PEP 538 has some downsides: >>> >>> * If

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 8 May 2017 at 15:34, Nick Coghlan wrote: > On 7 May 2017 at 15:22, INADA Naoki wrote: >> ## Background >> >> Locale coercion in current PEP 538 has some downsides: >> >> * If user set `LANG=C LC_DATE=ja_JP.UTF-8`, locale coercion may >> overrides

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

>>> On platforms where they would have no effect (e.g. Mac OS X, iOS, Android, >>> Windows) these preprocessor variables would always be undefined. >> >> Why ``--with[out]-c-locale-coercion`` have no effect on macOS, iOS and >> Android? > > On these three, we know the system encoding is UTF-8, so

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 7 May 2017 at 15:22, INADA Naoki wrote: > Hi, Nick. > > After thinking about relationship between PEP 538 and 540 in two days, > I came up with idea which removes locale coercion by default from PEP 538, > it does just enables UTF-8 mode and show warning about C locale.

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

Hi, Nick. After thinking about relationship between PEP 538 and 540 in two days, I came up with idea which removes locale coercion by default from PEP 538, it does just enables UTF-8 mode and show warning about C locale. Of course, this idea is based on PEP 540. There are no "If PEP 540 is

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 6 May 2017 at 18:33, Nick Coghlan wrote: > On 6 May 2017 at 18:00, Nick Coghlan wrote: >> On 5 March 2017 at 17:50, Nick Coghlan wrote: >>> Hi folks, >>> >>> Late last year I started working on a change to the CPython CLI (*not* the

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 6 May 2017 at 18:00, Nick Coghlan wrote: > On 5 March 2017 at 17:50, Nick Coghlan wrote: >> Hi folks, >> >> Late last year I started working on a change to the CPython CLI (*not* the >> shared library) to get it to coerce the legacy C locale to

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 5 March 2017 at 17:50, Nick Coghlan wrote: > Hi folks, > > Late last year I started working on a change to the CPython CLI (*not* the > shared library) to get it to coerce the legacy C locale to something based > on UTF-8 when a suitable locale is available. > > After a

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 5 May 2017 at 23:21, INADA Naoki wrote: > On Fri, May 5, 2017 at 1:25 AM, Antoine Pitrou wrote: >> If we want to adopt PEP 538 before pronouncing on PEP 540, then PEP 538 >> should remove all points conditional on PEP 540 adoption, and PEP 540 >>

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Fri, May 5, 2017 at 1:25 AM, Antoine Pitrou wrote: > On Thu, 4 May 2017 11:24:27 +0900 > INADA Naoki wrote: >> Hi, Nick and all core devs who are interested in this PEP. >> >> I'm reviewing PEP 538 and I want to accept it in this month. >> It will

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 5 May 2017 at 19:45, Erik Bray wrote: > On Thu, May 4, 2017 at 6:25 PM, Antoine Pitrou wrote: >> If we want to adopt PEP 538 before pronouncing on PEP 540, then PEP 538 >> should remove all points conditional on PEP 540 adoption, and PEP 540 >>

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Thu, May 4, 2017 at 6:25 PM, Antoine Pitrou wrote: > On Thu, 4 May 2017 11:24:27 +0900 > INADA Naoki wrote: >> Hi, Nick and all core devs who are interested in this PEP. >> >> I'm reviewing PEP 538 and I want to accept it in this month. >> It will

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 5 May 2017 at 02:25, Antoine Pitrou wrote: > On Thu, 4 May 2017 11:24:27 +0900 > INADA Naoki wrote: >> Hi, Nick and all core devs who are interested in this PEP. >> >> I'm reviewing PEP 538 and I want to accept it in this month. >> It will reduces

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Sat, Mar 4, 2017 at 11:50 PM, Nick Coghlan wrote: > > Providing implicit locale coercion only when running standalone > --- > > Over the course of Python 3.x development, multiple attempts have been made > to

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Thu, 4 May 2017 11:24:27 +0900 INADA Naoki wrote: > Hi, Nick and all core devs who are interested in this PEP. > > I'm reviewing PEP 538 and I want to accept it in this month. > It will reduces much UnicodeError pains which server-side OPs facing. > Thank you Nick for

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 4 May 2017 at 12:24, INADA Naoki wrote: > [PEP 538] >> * PEP 540 proposes to entirely decouple CPython's default text encoding from >> the C locale system in that case, allowing text handling inconsistencies to >> arise between CPython and other locale-aware

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

Hi, Nick and all core devs who are interested in this PEP. I'm reviewing PEP 538 and I want to accept it in this month. It will reduces much UnicodeError pains which server-side OPs facing. Thank you Nick for working on this PEP. If you have something worrying about this PEP, please post a

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 16 March 2017 at 00:30, Barry Warsaw wrote: > On Mar 15, 2017, at 12:29 PM, Nick Coghlan wrote: > > >From a mainstream Linux point of view, it's not common - on > systemd-managed > >systems, for example, the only way to get the C locale these days is to > >either specify it

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Mar 15, 2017, at 12:29 PM, Nick Coghlan wrote: >From a mainstream Linux point of view, it's not common - on systemd-managed >systems, for example, the only way to get the C locale these days is to >either specify it in /etc/locale.conf, or to set it specifically in the >environment. I think

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 15 March 2017 at 06:22, Chris Barker wrote: > So the question nis -- is anyone counting on errors in this case? i.e., is > a sysadmin thinking: > > "I want an ASCII-only system, so I'll set the locale, and now I can expect > any program running on this system that is

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 15 March 2017 at 00:17, Nick Coghlan wrote: > On 13 March 2017 at 23:31, Random832 wrote: > >> On Mon, Mar 13, 2017, at 04:37, INADA Naoki wrote: >> > But locale coercing works nice on platforms like android. >> > So how about simplified version of

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

There was a bunch of discussion about all this a while back, in which I think these points were addressed: However, in some cases the C locale is a normal environment for system > services, cron scripts, distro package builds and whatnot. > Indeed it is. But: if you run a Python (or any)

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Tue, Mar 14, 2017, at 10:17, Nick Coghlan wrote: > It's not that you *can't* run Python 3 in that kind of environment, and > it's not that there are never any valid reasons to do so. It's that lots > of > things that you'd typically expect to work are going to misbehave (one I > discovered

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 13 March 2017 at 23:31, Random832 wrote: > On Mon, Mar 13, 2017, at 04:37, INADA Naoki wrote: > > But locale coercing works nice on platforms like android. > > So how about simplified version of PEP 538? Just adding configure > > option for locale coercing > > which

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Mon, Mar 13, 2017 at 10:31 PM, Random832 wrote: > On Mon, Mar 13, 2017, at 04:37, INADA Naoki wrote: >> But locale coercing works nice on platforms like android. >> So how about simplified version of PEP 538? Just adding configure >> option for locale coercing >> which

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Mon, Mar 13, 2017 at 8:01 PM, Nick Coghlan wrote: > On 13 March 2017 at 18:37, INADA Naoki wrote: >> >> But locale coercing works nice on platforms like android. >> So how about simplified version of PEP 538? Just adding configure >> option for

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 13 March 2017 at 18:37, INADA Naoki wrote: > But locale coercing works nice on platforms like android. > So how about simplified version of PEP 538? Just adding configure > option for locale coercing > which is disabled by default. No envvar options and no warnings.

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

> It seems to based on an assumption that the C locale is always some kind of > pathology. Admittedly, it sometimes is a result of misconfiguration or a > mistake. (But I don't see why it's the interpreter's job to correct such > mistakes.) However, in some cases the C locale is a normal

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 12 March 2017 at 22:57, Nick Coghlan wrote: > However, I'm also open to having [PYTHONCOERCECLOCALE=0] also disable the > runtime warning from the shared library. > Considering this a little further, I think this is going to be necessary in order to sensibly handle the

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 12 March 2017 at 08:36, Jakub Wilk wrote: > This is a very bad idea. > > It seems to based on an assumption that the C locale is always some kind > of pathology. Admittedly, it sometimes is a result of misconfiguration or a > mistake. (But I don't see why it's the

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

This is a very bad idea. It seems to based on an assumption that the C locale is always some kind of pathology. Admittedly, it sometimes is a result of misconfiguration or a mistake. (But I don't see why it's the interpreter's job to correct such mistakes.) However, in some cases the C locale

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 9 March 2017 at 07:58, Guido van Rossum wrote: > On Wed, Mar 8, 2017 at 4:35 AM, Nick Coghlan wrote: > >> >> On 5 March 2017 at 17:50, Nick Coghlan wrote: >> >>> Late last year I started working on a change to the CPython CLI (*not*

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Wed, Mar 8, 2017 at 4:35 AM, Nick Coghlan wrote: > > On 5 March 2017 at 17:50, Nick Coghlan wrote: > >> Late last year I started working on a change to the CPython CLI (*not* >> the shared library) to get it to coerce the legacy C locale to something

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 5 March 2017 at 17:50, Nick Coghlan wrote: > Hi folks, > > Late last year I started working on a change to the CPython CLI (*not* the > shared library) to get it to coerce the legacy C locale to something based > on UTF-8 when a suitable locale is available. > > After a

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On 6 March 2017 at 00:39, INADA Naoki wrote: > I prefer just "locale-aware" / "locale-independent" (application | > library | function) > to "locale-aware C/C++ application" / "C/C++ independent" here. Good point, I'll fix that in the next update. > Backporting to Python

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

LGTM and I love this PEP and PEP 540. Some comments: ... > * PEP 540 proposes to entirely decouple CPython's default text encoding from > the C locale system in that case, allowing text handling inconsistencies > to > arise between CPython and other C/C++ components running in the same >

[Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

Hi folks, Late last year I started working on a change to the CPython CLI (*not* the shared library) to get it to coerce the legacy C locale to something based on UTF-8 when a suitable locale is available. After a couple of rounds of iteration on linux-sig and python-ideas, I'm now bringing it