Re: [Python-Dev] File system path encoding on Windows

On 31 August 2016 at 10:27, Steve Dower wrote: > On 30Aug2016 1702, Victor Stinner wrote: >> I can try to run more tests if you know some other major Python >> applications (modules?) working on Windows/Python 3. > > The major ones aren't really the concern. I'd be

Re: [Python-Dev] File system path encoding on Windows

On 30Aug2016 1702, Victor Stinner wrote: I made another quick test on Django 1.10 (I ran Django test suite on my modified Python raising exception on bytes path): I didn't notice any exception related to bytes path. Django seems to only use Unicode for paths. I can try to run more tests if you

Re: [Python-Dev] File system path encoding on Windows

I made another quick test on Django 1.10 (I ran Django test suite on my modified Python raising exception on bytes path): I didn't notice any exception related to bytes path. Django seems to only use Unicode for paths. I can try to run more tests if you know some other major Python applications

Re: [Python-Dev] File system path encoding on Windows

On 30Aug2016 1611, Victor Stinner wrote: 2016-08-30 23:51 GMT+02:00 Victor Stinner : As I already wrote once, my problem is also tjat I simply have no idea how much Python 3 code uses bytes filename. For example, does it concern more than 25% of py3 modules on PyPi, or

Re: [Python-Dev] File system path encoding on Windows

2016-08-30 23:51 GMT+02:00 Victor Stinner : > As I already wrote once, my problem is also tjat I simply have no idea how > much Python 3 code uses bytes filename. For example, does it concern more > than 25% of py3 modules on PyPi, or less than 5%? I made a very quick

Re: [Python-Dev] File system path encoding on Windows

Le 30 août 2016 8:05 PM, "Nick Coghlan" a écrit : > This seems to be the crux of the disagreement: our perceptions of the > relative risks to native Windows Python applications that currently > work properly on Python 3.5 vs the potential compatibility benefits to > primarily

Re: [Python-Dev] File system path encoding on Windows

On 30Aug2016 1108, Guido van Rossum wrote: Is this thread something I need to follow closely? I have PEPs coming, and I'll distil the technical parts of the discussion into those. We may need you to impose an opinion on whether 3.6 is an appropriate time for the change or it should wait

Re: [Python-Dev] File system path encoding on Windows

On 31 August 2016 at 01:06, Victor Stinner wrote: > 2016-08-30 16:31 GMT+02:00 Steve Dower : >> Any system that requires communication between two different versions of >> Python must have install instructions (if it's public) or someone who >>

Re: [Python-Dev] File system path encoding on Windows

Is this thread something I need to follow closely? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] File system path encoding on Windows

On 30Aug2016 0806, Victor Stinner wrote: 2016-08-30 16:31 GMT+02:00 Steve Dower : It's the random user on Windows who installed their library that has the problem. They don't know the fix, and may not know how to apply it (e.g. if it's their Jupyter notebook that won't

Re: [Python-Dev] File system path encoding on Windows

2016-08-30 16:31 GMT+02:00 Steve Dower : > It's the > random user on Windows who installed their library that has the problem. > They don't know the fix, and may not know how to apply it (e.g. if it's > their Jupyter notebook that won't find one of their files - no obvious

Re: [Python-Dev] File system path encoding on Windows

m: "Victor Stinner" <victor.stin...@gmail.com> Sent: ‎8/‎30/‎2016 1:21 To: "Nick Coghlan" <ncogh...@gmail.com> Cc: "Steve Dower" <steve.do...@python.org>; "Python Dev" <python-dev@python.org> Subject: Re: [Python-Dev] File system path enco

Re: [Python-Dev] File system path encoding on Windows

Le 30 août 2016 03:10, "Nick Coghlan" a écrit : > However, this view is also why I don't agree with being aggressive in > making this behaviour the default on Windows - I think we should make > it readily available as a provisional feature through a single > cross-platform

Re: [Python-Dev] File system path encoding on Windows

On 30 August 2016 at 13:29, Steve Dower wrote: > The other discussion about OpenSSL and LTS systems is also interesting. Do > we really expect users to take their fully functioning systems and blindly > upgrade to a new major version of Python expecting everything to just

Re: [Python-Dev] File system path encoding on Windows

On 29Aug2016 1810, Nick Coghlan wrote: On 30 August 2016 at 08:38, Victor Stinner wrote: Hi, tl; dr: just drop byte support and help developers to use Unicode in their application! My view (and Steve's) is that this approach is likely to result in Linux-centric

Re: [Python-Dev] File system path encoding on Windows

On 30 August 2016 at 11:10, Daniel Holth wrote: > A nice recent blog post about paths encoding in a media player. > http://beets.io/blog/paths.html . It's not merely the porting that makes it > hard. Interestingly, with Steve's proposed changes, their approach of using bytes

Re: [Python-Dev] File system path encoding on Windows

On 30 August 2016 at 08:38, Victor Stinner wrote: > Hi, > > tl; dr: just drop byte support and help developers to use Unicode in > their application! My view (and Steve's) is that this approach is likely to result in Linux-centric projects just dropping even nominal

Re: [Python-Dev] File system path encoding on Windows

A nice recent blog post about paths encoding in a media player. http://beets.io/blog/paths.html . It's not merely the porting that makes it hard. On Mon, Aug 29, 2016, 19:32 Victor Stinner wrote: > 2016-08-24 17:44 GMT+02:00 Steve Dower : > > I

Re: [Python-Dev] File system path encoding on Windows

2016-08-24 17:44 GMT+02:00 Steve Dower : > I know Nick and Victor like the idea of a -X flag (or a direct -utf8 flag), > but I prefer more specific environment variables: > > - PYTHONWINDOWSLEGACYSTDIO (for the console changes) > - PYTHONWINDOWSLEGACYPATHENCODING (assuming

Re: [Python-Dev] File system path encoding on Windows

2016-08-20 21:31 GMT+02:00 Nick Coghlan : > Reading your summary meant this finally clicked with something Victor > has been considering for a while: a "Force UTF-8" switch that told > Python to ignore the locale encoding on Linux, and instead assume > UTF-8 everywhere (command

Re: [Python-Dev] File system path encoding on Windows

Hi, tl; dr: just drop byte support and help developers to use Unicode in their application! As you may already know, I dislike your whole project. But first of all, IMHO you should open a separated thread to discuss changes related to the Windows console. I consider that they are unrelated,

Re: [Python-Dev] File system path encoding on Windows

On 28Aug2016 2043, Stephen J. Turnbull wrote: tritium-l...@sdamon.com writes: > Once you get to var lengths like that, arcane single character flags start > looking preferable. How about "PYTHONWINLEGACY" to just turn it all on or > off. If the code breaks on one thing, it obviously isn't

Re: [Python-Dev] File system path encoding on Windows

tritium-l...@sdamon.com writes: > Once you get to var lengths like that, arcane single character flags start > looking preferable. How about "PYTHONWINLEGACY" to just turn it all on or > off. If the code breaks on one thing, it obviously isn't written to use the > other two, so might as

Re: [Python-Dev] File system path encoding on Windows

tephen J. Turnbull <turnbull.stephen...@u.tsukuba.ac.jp> >> Cc: Nick Coghlan <ncogh...@gmail.com>; Python Dev > d...@python.org> >> Subject: Re: [Python-Dev] File system path encoding on Windows >> >> On 23Aug2016 2150, Stephen J. Turnbull wrote:

Re: [Python-Dev] File system path encoding on Windows

ncogh...@gmail.com>; Python Dev d...@python.org> > Subject: Re: [Python-Dev] File system path encoding on Windows > > On 23Aug2016 2150, Stephen J. Turnbull wrote: > > Steve Dower writes: > > > > > * Stephen sees "no reason not to change locale.getprefer

Re: [Python-Dev] File system path encoding on Windows

On 23Aug2016 2150, Stephen J. Turnbull wrote: Steve Dower writes: > * Stephen sees "no reason not to change locale.getpreferredencoding()" > (default encoding for open()) at the same time with the same switches, > while I'm not quite as confident. Do users generally specify an encoding >

Re: [Python-Dev] File system path encoding on Windows

Steve Dower writes: > * Stephen sees "no reason not to change locale.getpreferredencoding()" > (default encoding for open()) at the same time with the same switches, > while I'm not quite as confident. Do users generally specify an encoding > these days? I know I always put utf-8 there. I

Re: [Python-Dev] File system path encoding on Windows

I've trimmed fairly aggressively for the sake of not causing the rest of the list to mute our discussion (again :) ). Stephen - feel free to email me off list if I go too far or misrepresent you. As a summary for people who don't want to read on (and Stephen will correct me if I misquote):

Re: [Python-Dev] File system path encoding on Windows

eryk sun writes: > I just wrote a simple function to enumerate the 822 system locales on > my Windows box (using EnumSystemLocalesEx and GetLocaleInfoEx, which > are Unicode-only functions), and 36.7% of them lack an ANSI codepage. > They're Unicode-only locales. UTF-8 is the only way to

Re: [Python-Dev] File system path encoding on Windows

Steve Dower writes: > The Windows world is Unicode. Mostly represented in UTF-16, but UTF-8 is > entirely equivalent. Sort of, yes, and not for present purposes. AFAICS, the Windows world is mostly application/* media that require substantial developer effort to extract text from; character

Re: [Python-Dev] File system path encoding on Windows

On Mon, Aug 22, 2016 at 3:58 PM, Steve Dower wrote: > All MSVC users have been pushed towards Unicode for many years. The .NET > Framework has defaulted to UTF-8 its entire existence. The use of code pages > has been discouraged for decades. We're not going first :) I

Re: [Python-Dev] File system path encoding on Windows

On 22Aug2016 0247, Stephen J. Turnbull wrote: Nick Coghlan writes: > On 21 August 2016 at 06:31, Steve Dower wrote: > > My biggest concern is that it then falls onto users to know how > > to start Python with that flag. The users I'm most worried about belong to

Re: [Python-Dev] File system path encoding on Windows

Nick Coghlan writes: > On 21 August 2016 at 06:31, Steve Dower wrote: > > My biggest concern is that it then falls onto users to know how > > to start Python with that flag. The users I'm most worried about belong to organizations where concerted effort has been made

Re: [Python-Dev] File system path encoding on Windows

On 21 August 2016 at 06:31, Steve Dower wrote: > My biggest concern is that it then falls onto users to know how to start > Python with that flag. Not necessarily, as this is one of the areas where commercial redistributors can earn their revenue stream - by deciding that

Re: [Python-Dev] File system path encoding on Windows

On 20Aug2016 1231, Nick Coghlan wrote: I'd like to suggest an option that didn't come up on python-ideas: add such a flag to Python 3.6, and then actively seek feedback from folks using non-UTF-8 encodings before making a decision on what to do by default in Python 3.7. My biggest concern is

Re: [Python-Dev] File system path encoding on Windows

On 20 August 2016 at 04:59, Steve Dower wrote: > Questions: > * should we always use Window's Unicode APIs instead of switching between > bytes/Unicode based on parameter type? Yes > * should we allow users to pass bytes and interpret them as utf-8 rather > than letting

Re: [Python-Dev] File system path encoding on Windows

On 19Aug2016 1225, Daniel Holth wrote: #1 sounds like a great idea. I suppose surrogatepass solves approximately the same problem of Rust's WTF-8, which is a way to round-trip bad UCS-2? https://simonsapin.github.io/wtf-8/ Yep. #2 sounds like it would leave several problems, since mbcs is

Re: [Python-Dev] File system path encoding on Windows

#1 sounds like a great idea. I suppose surrogatepass solves approximately the same problem of Rust's WTF-8, which is a way to round-trip bad UCS-2? https://simonsapin.github.io/wtf-8/ #2 sounds like it would leave several problems, since mbcs is not the same as a normal text encoding, IIUC it

[Python-Dev] File system path encoding on Windows

Hi python-dev About a week ago I proposed on python-ideas making some changes to how Python deals with encodings on Windows, specifically in relation to how Python interacts with the operating system. Changes to the console were uncontroversial, and I have posted patches at