Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
Nick Coghlan schrieb: So this is taking something that *already works properly on POSIX systems* and making it work on Windows as well. I doubt it does without side effects. For example, an application that would go through sys.path, and encode everything with sys.getfilesystemencoding()

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Steve Holden
Martin v. Löwis wrote: Nick Coghlan schrieb: So this is taking something that *already works properly on POSIX systems* and making it work on Windows as well. I doubt it does without side effects. For example, an application that would go through sys.path, and encode everything with

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Martin v. Löwis wrote: Nick Coghlan schrieb: So this is taking something that *already works properly on POSIX systems* and making it work on Windows as well. I doubt it does without side effects. For example, an application that would go through sys.path, and encode everything with

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: I doubt it does without side effects. For example, an application that would go through sys.path, and encode everything with sys.getfilesystemencoding() currently works, but will break if the patch is applied and non-mbcs strings are put on sys.path. Huh? It won't

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Nick Coghlan
David Hopwood wrote: Martin v. Löwis wrote: Nick Coghlan schrieb: So this is taking something that *already works properly on POSIX systems* and making it work on Windows as well. I doubt it does without side effects. For example, an application that would go through sys.path, and encode

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
Nick Coghlan schrieb: I think MvL is looking at it from the point of view of consumers of the list of strings in sys.path, such as PEP 302 importer and loader objects, and tools like module_finder. Currently, the list of values in sys.path is limited to: That, and all kinds of inspection

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Nick Coghlan wrote: David Hopwood wrote: Martin v. Löwis wrote: Nick Coghlan schrieb: So this is taking something that *already works properly on POSIX systems* and making it work on Windows as well. I doubt it does without side effects. For example, an application that would go through

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: On Windows, file system pathnames can contain arbitrary Unicode characters (well, almost). Despite the existence of ANSI filesystem APIs, and regardless of what 'sys.getfilesystemencoding()' returns, the underlying file system encoding for NTFS and FAT filesystems is

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Martin v. Löwis wrote: David Hopwood schrieb: On Windows, file system pathnames can contain arbitrary Unicode characters (well, almost). Despite the existence of ANSI filesystem APIs, and regardless of what 'sys.getfilesystemencoding()' returns, the underlying file system encoding for NTFS and

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: Elements of sys.path can be Unicode strings in Python 2.5, and should be pathnames supported by the underlying OS. Where is it documented that there is any further restriction on them? And why should there be any further restriction on them? It's not documented in that

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Nick Coghlan
David Hopwood wrote: Martin v. Löwis wrote: Programs that rely on sys.getfilesystemencoding() being able to represent arbitrary file names on Windows might have a bug; programs that rely on sys.getfilesystemencoding() being able to encode all elements of sys.path do not (at least not for

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: On Friday 08 September 2006 02:56, Kristján V. Jónsson wrote: Hello All. I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) which allows unicode paths in sys.path and uses the unicode file api on windows. This is tried and tested on 2.5, and

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Anthony Baxter
On Friday 08 September 2006 18:24, Steve Holden wrote: As this can't be considered a bugfix (that I can see), I'd be against it being checked into 2.5. Are you suggesting that Python's inability to correctly handle Unicode path elements isn't a bug? Or simply that this inability isn't

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: On Friday 08 September 2006 18:24, Steve Holden wrote: As this can't be considered a bugfix (that I can see), I'd be against it being checked into 2.5. Are you suggesting that Python's inability to correctly handle Unicode path elements isn't a bug? Or simply that this

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Nick Coghlan
Steve Holden wrote: Anthony Baxter wrote: On Friday 08 September 2006 18:24, Steve Holden wrote: I agree it's a relatively large patch for a release candidate but if prudence suggests deferring it, it should be a *definite* for 2.5.1 and subsequent releases. Possibly. I remain unconvinced.

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Anthony Baxter
On Friday 08 September 2006 19:19, Steve Holden wrote: But it *is* a desirable, albeit new, feature, so I'm surprised that you don't appear to perceive it as such for a downstream release. Point releases (2.x.1 and suchlike) are absolutely not for new features. They're for bugfixes, only. It's

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: On Friday 08 September 2006 19:19, Steve Holden wrote: But it *is* a desirable, albeit new, feature, so I'm surprised that you don't appear to perceive it as such for a downstream release. Point releases (2.x.1 and suchlike) are absolutely not for new features.

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Guido van Rossum
On 9/8/06, Steve Holden [EMAIL PROTECTED] wrote: Anthony Baxter wrote: On Friday 08 September 2006 19:19, Steve Holden wrote: But it *is* a desirable, albeit new, feature, so I'm surprised that you don't appear to perceive it as such for a downstream release. Point releases (2.x.1 and

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread skip
Guido IMO it's the call of the release managers. Board members ought to Guido trust the release managers and not apply undue pressure. Indeed. Let's not go whacking people with boards. The Perl people would just laugh at us... Skip ___

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Giovanni Bajo
Guido van Rossum [EMAIL PROTECTED] wrote: IMO it's the call of the release managers. Board members ought to trust the release managers and not apply undue pressure. +1, but I would love to see a more formal definition of what a bugfix is, which would reduce the ambiguous cases, and thus

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Raymond Hettinger
Giovanni Bajo wrote: +1, but I would love to see a more formal definition of what a bugfix is, which would reduce the ambiguous cases, and thus reduce the number of times the release managers are called to pronounce. Sorry, that is just a pipe-dream. To some degree, all bug-fixes are new

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread M.-A. Lemburg
Kristján V. Jónsson wrote: Hello All. I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) which allows unicode paths in sys.path and uses the unicode file api on windows. This is tried and tested on 2.5, and backported to 2.3 and is currently running on clients in

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: As this can't be considered a bugfix (that I can see), I'd be against it being checked into 2.5. Are you suggesting that Python's inability to correctly handle Unicode path elements isn't a bug? Not sure whether Anthony suggests it, but I do. Or simply that this

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: I agree it's a relatively large patch for a release candidate but if prudence suggests deferring it, it should be a *definite* for 2.5.1 and subsequent releases. Possibly. I remain unconvinced. But it *is* a desirable, albeit new, feature, so I'm surprised that you

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Nick Coghlan schrieb: But it *is* a desirable, albeit new, feature, so I'm surprised that you don't appear to perceive it as such for a downstream release. And unlike 2.2's True/False problem, it is an *environmental* feature, rather than a programmatic one. Not sure what you mean by

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: I don't regard this as the provision of a new feature but as the removal of an unnecessary restriction (which I would prefer to call a bug). You got the definition of bug wrong. Primarily, a bug is a deviation from the specification. Extending the domain of an argument to

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Giovanni Bajo schrieb: +1, but I would love to see a more formal definition of what a bugfix is, which would reduce the ambiguous cases, and thus reduce the number of times the release managers are called to pronounce. Other projects, for instance, describe point releases as open for

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Nick Coghlan
Martin v. Löwis wrote: Steve Holden schrieb: Or simply that this inability isn't currently described in a bug report on Sourceforge? No: sys.path is specified (originally) as containing a list of byte strings; it was extended to also support path importers (or whatever that PEP calls

Re: [Python-Dev] Unicode Imports

2006-09-07 Thread Anthony Baxter
On Friday 08 September 2006 02:56, Kristján V. Jónsson wrote: Hello All. I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) which allows unicode paths in sys.path and uses the unicode file api on windows. This is tried and tested on 2.5, and backported to 2.3 and is

Re: [Python-Dev] unicode imports

2006-06-20 Thread Thomas Heller
Martin v. Löwis schrieb: Thomas Heller wrote: It should be noted that I once started to convert the import machinery to be fully unicode aware. As far as I can tell, a *lot* has to be changed to make this work. Is that code available somewhere still? Does it still work? Available as patch

Re: [Python-Dev] unicode imports

2006-06-20 Thread Martin v. Löwis
Thomas Heller wrote: Is that code available somewhere still? Does it still work? Available as patch 1093253, I have not tried if it stil works I see. It's quite a huge change, that's probably why nobody found the time to review it, yet. To be really useful, wide char versions of other

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
, but I cannot write kristján = 1. But that's for a future PEP. Kristján -Original Message- From: Nick Coghlan [mailto:[EMAIL PROTECTED] Sent: 16. júní 2006 15:30 To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson wrote: A cursory glance

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
- From: Nick Coghlan [mailto:[EMAIL PROTECTED] Sent: 17. júní 2006 04:17 To: Phillip J. Eby Cc: Kristján V. Jónsson; Python Dev Subject: Re: [Python-Dev] unicode imports Phillip J. Eby wrote: Actually, you would want to put it in sys.path_hooks, and then instances would be placed

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
. Kristján -Original Message- From: Neil Hodgson [mailto:[EMAIL PROTECTED] Sent: 17. júní 2006 04:53 To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson: Although python has had full unicode support for filenames for a long time

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson wrote: The standard install path in chinese distributions can be with a non-ANSI path, and installing an embedded python application there will break it. I very much doubt this. On a Chinese system, the Program

Re: [Python-Dev] unicode imports

2006-06-19 Thread Thomas Heller
It should be noted that I once started to convert the import machinery to be fully unicode aware. As far as I can tell, a *lot* has to be changed to make this work. I started with refactoring Python/import.c, but nobody responded to the question whether such a refactoring patch would be accepted

Re: [Python-Dev] unicode imports

2006-06-19 Thread M.-A. Lemburg
Thomas Heller wrote: It should be noted that I once started to convert the import machinery to be fully unicode aware. As far as I can tell, a *lot* has to be changed to make this work. I started with refactoring Python/import.c, but nobody responded to the question whether such a

Re: [Python-Dev] unicode imports

2006-06-19 Thread Nick Coghlan
Kristján V. Jónsson wrote: Funny that no other platforms could benefit from a unicode import path. Does that mean that windows will reign supreme? Please explain. As near as I can tell, other platforms use encoded strings with the normal (byte-based) posix file API, so the Python interpreter

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
, it is something I will have to patch in for our application. Cheers, Kristján -Original Message- From: Nick Coghlan [mailto:[EMAIL PROTECTED] Sent: 19. júní 2006 13:46 To: Kristján V. Jónsson Cc: Martin v. Löwis; Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson wrote

Re: [Python-Dev] unicode imports

2006-06-19 Thread Guido van Rossum
On 6/16/06, Kristján V. Jónsson [EMAIL PROTECTED] wrote: Although python has had full unicode support for filenames for a long time on selected platforms (e.g. Windows), there is one glaring deficiency: It cannot import from paths containing unicode. I´ve tried creating folders with chinese

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Kristján V. Jónsson wrote: I don't have specific information on the machines. We didn´t try very hard to get things to work with 2.3 since we simply assumed it would work automatically when we upgraded to a more mature 2.4. I could try to get more info, but it would be 2.3 specific. Have

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Kristján V. Jónsson wrote: Wouldn´t it be possible then to emulate the unix way? Simply encode any unicode paths to utf-8, process them as normal, and then decode them just prior to the actual windows io call? That won't work. People also put path names from the ANSI code page onto sys.path

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Thomas Heller wrote: It should be noted that I once started to convert the import machinery to be fully unicode aware. As far as I can tell, a *lot* has to be changed to make this work. Is that code available somewhere still? Does it still work? I started with refactoring Python/import.c,

Re: [Python-Dev] unicode imports

2006-06-17 Thread Martin v. Löwis
Kristján V. Jónsson wrote: The standard install path in chinese distributions can be with a non-ANSI path, and installing an embedded python application there will break it. I very much doubt this. On a Chinese system, the Program Files folder likely has a non-*ASCII* name, but it will have a

Re: [Python-Dev] unicode imports

2006-06-17 Thread Martin v. Löwis
Neil Hodgson wrote: It should be unusual for a Chinese installation to use an install path that can not be represented in MBCS. Try encoding the install directory into MBCS before adding it to sys.path. Indeed. Unfortunately, people apparently install an English version (because they can

Re: [Python-Dev] unicode imports

2006-06-17 Thread Ronald Oussoren
On 17-jun-2006, at 6:44, Nick Coghlan wrote: Bob Ippolito wrote: There's a similar issue in that if sys.prefix contains a colon, Python is also busted: http://python.org/sf/1507224 Of course, that's not a Windows issue, but it is everywhere else. The offending code in that case is

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Kristján V. Jónsson wrote: A cursory glance at import.c shows that the import mechanism is fairly complicated, and riddled with char *path thingies, and manual string arithmetic. Do you have any suggestions on a clean way to unicodify the import mechanism? Can you install a PEP 302 path

Re: [Python-Dev] unicode imports

2006-06-16 Thread Phillip J. Eby
At 01:29 AM 6/17/2006 +1000, Nick Coghlan wrote: Kristján V. Jónsson wrote: A cursory glance at import.c shows that the import mechanism is fairly complicated, and riddled with char *path thingies, and manual string arithmetic. Do you have any suggestions on a clean way to unicodify the

Re: [Python-Dev] unicode imports

2006-06-16 Thread Bob Ippolito
On Jun 16, 2006, at 9:02 AM, Phillip J. Eby wrote: At 01:29 AM 6/17/2006 +1000, Nick Coghlan wrote: Kristján V. Jónsson wrote: A cursory glance at import.c shows that the import mechanism is fairly complicated, and riddled with char *path thingies, and manual string arithmetic. Do

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Phillip J. Eby wrote: Actually, you would want to put it in sys.path_hooks, and then instances would be placed in path_importer_cache automatically. If you are adding it to the path_hooks after the fact, you should simply clear the path_importer_cache. Simply poking stuff into the

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Bob Ippolito wrote: There's a similar issue in that if sys.prefix contains a colon, Python is also busted: http://python.org/sf/1507224 Of course, that's not a Windows issue, but it is everywhere else. The offending code in that case is Modules/getpath.c, Since it has to do with the