[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-14 Thread Stephen J. Turnbull
Andrew Barnert writes: > On Jan 13, 2020, at 19:32, Stephen J. Turnbull > wrote: > > > > There is still tons of data in legacy > > applications, both as text files and in various application data > > formats, that use legacy encodings (in Japanese, that means MBCS). > > Sadly, it's not

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-14 Thread Eryk Sun
On 1/14/20, Inada Naoki wrote: > > UTF-8 mode shouldn't take precedence over legacy FS encoding. > > Mercurial uses legacy encoding for file paths. They use > sys._enablelegacywindowsfsencoding() on Windows. > https://www.mercurial-scm.org/repo/hg/rev/8d5489b048b7 This runtime call can override

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-14 Thread Inada Naoki
On Sun, Jan 12, 2020 at 9:32 PM Eryk Sun wrote: > > In both of the above cases, what I'd prefer is for UTF-8 mode to take > precedence over legacy modes, i.e. to disable > config->legacy_windows_fs_encoding and config->legacy_windows_stdio in > the startup configuration. > UTF-8 mode shouldn't

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-13 Thread Andrew Barnert via Python-ideas
On Jan 13, 2020, at 19:32, Stephen J. Turnbull wrote: > > There is still tons of data in legacy > applications, both as text files and in various application data > formats, that use legacy encodings (in Japanese, that means MBCS). > Sadly, it's not as simple as running "iconv -f shift_jis -t

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-13 Thread Stephen J. Turnbull
Kyle Stanley writes: > I'll admit though that I have no experience working with any application > that implicitly assumes MBCS is the default encoding format, is this a > common occurrence in some older applications? At least in East Asia, it is. Eg, my university's student information

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-12 Thread Kyle Stanley
Inada Naoki wrote: > Current header is: > Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC > v.1916 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > I'm proposing adding one more line: > UTF-8 mode is disabled. (See

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-12 Thread Eryk Sun
On 1/10/20, Andrew Barnert via Python-ideas wrote: > On Jan 10, 2020, at 03:45, Inada Naoki wrote: > > Also, PYTHONUTF8 is only supported on Unix, so presumably it’s ignored if > you set it on Windows, right? The implementation of UTF-8 mode (i.e. -Xutf8) is cross-platform, though I think it

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-11 Thread Inada Naoki
On Sat, Jan 11, 2020 at 11:03 AM Kyle Stanley wrote: > > > 1. Recommend it in the official document "Using Python on Windows" [2]. > > 2. Show the UTF-8 mode status in the command line mode header [3] on > > Windows. > > 3. Show the link to the UTF-8 mode document in the command line mode header

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-10 Thread Kyle Stanley
> 1. Recommend it in the official document "Using Python on Windows" [2]. > 2. Show the UTF-8 mode status in the command line mode header [3] on Windows. > 3. Show the link to the UTF-8 mode document in the command line mode header too. > 4. Add checkbox to set "PYTHONUTF8=1" environment variable

[Python-ideas] Re: Recommend UTF-8 mode on Windows

2020-01-10 Thread Andrew Barnert via Python-ideas
On Jan 10, 2020, at 03:45, Inada Naoki wrote: > > Hi, all. > > I believe UTF-8 should be chosen by default for text encoding. Correct me if I’m wrong, but I think in Python 3.7 on Windows 10, the filesystem encoding is already UTF-8, and the stdio console files are UTF-8 (but under the