Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Martin v. Löwis
>> The answer to all of this is the filesystem encoding, which is already >> supported. Doesn't appear particularly difficult to me. > > sys.getfilesystemencoding() is None on most Linux computers I have access to. That's strange. Is LANG not set? > How is the problem solved there? A default n

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Guido van Rossum
On 5/17/07, Greg Ewing <[EMAIL PROTECTED]> wrote: > James Y Knight wrote: > > The *really* tricky thing is that on unix systems, if you want to be > > able to access all the files on the disk, you have to use the byte- > > string API ... But on windows ... you *CANNOT* use the byte-string api > > H

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Greg Ewing
James Y Knight wrote: > The *really* tricky thing is that on unix systems, if you want to be > able to access all the files on the disk, you have to use the byte- > string API ... But on windows ... you *CANNOT* use the byte-string api How are we going to cope with this in Py3k with unicode-on

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Giovanni Bajo
On 18/05/2007 1.24, James Y Knight wrote: > unicode_filename.encode(sys.getfilesystemencoding() or 'ascii', > 'xmlcharrefreplace') would work. Thanks - using "xmlcharrefreplace" hadn't occurred to me! > The *really* tricky thing is that on unix systems, if you want to be > able to access all

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread James Y Knight
On May 17, 2007, at 7:04 PM, Giovanni Bajo wrote: > On 13/05/2007 21.31, Guido van Rossum wrote: > >> The answer to all of this is the filesystem encoding, which is >> already >> supported. Doesn't appear particularly difficult to me. > > sys.getfilesystemencoding() is None on most Linux comput

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Guido van Rossum
On 5/17/07, Giovanni Bajo <[EMAIL PROTECTED]> wrote: > On 18/05/2007 1.10, Guido van Rossum wrote: > > >> In fact, I have a question about this. Can anybody show me a valid > >> multi-platform Python code snippet that, given a filename as *unicode* > >> string, > >> create a file with that name

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Giovanni Bajo
On 18/05/2007 1.10, Guido van Rossum wrote: >> In fact, I have a question about this. Can anybody show me a valid >> multi-platform Python code snippet that, given a filename as *unicode* >> string, >> create a file with that name, possibly adjusting the name so to ignore an >> encoding probl

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Guido van Rossum
On 5/17/07, Giovanni Bajo <[EMAIL PROTECTED]> wrote: > On 13/05/2007 21.31, Guido van Rossum wrote: > > > The answer to all of this is the filesystem encoding, which is already > > supported. Doesn't appear particularly difficult to me. > > sys.getfilesystemencoding() is None on most Linux computer

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Giovanni Bajo
On 13/05/2007 21.31, Guido van Rossum wrote: > The answer to all of this is the filesystem encoding, which is already > supported. Doesn't appear particularly difficult to me. sys.getfilesystemencoding() is None on most Linux computers I have access to. How is the problem solved there? In fact,

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-17 Thread Martin v. Löwis
> Does the tokenizer do this for all string literals, too? Otherwise you > could still get surprises with things like x.foo vs. getattr(x, > "foo"), if the name foo were normalized but the string "foo" were not. No. If you use a string literal, chances are very high that you put NFC into your sour

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-14 Thread Jason Orendorff
On 5/14/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > Does the tokenizer do this for all string literals, too? Otherwise you > could still get surprises with things like x.foo vs. getattr(x, > "foo"), if the name foo were normalized but the string "foo" were not. It does not; so yes, you could

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-14 Thread Guido van Rossum
On 5/14/07, Jason Orendorff <[EMAIL PROTECTED]> wrote: > On 5/14/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > Isn't normalization also going to be an issue with using non-ASCII in > > general? Does it mean that Python will have to use a normalization > > before comparing identifiers as equal

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-14 Thread Jason Orendorff
On 5/14/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > Isn't normalization also going to be an issue with using non-ASCII in > general? Does it mean that Python will have to use a normalization > before comparing identifiers as equal? That's terrible, as it will > vastly increase the amount need

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-14 Thread Guido van Rossum
On 5/14/07, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > On Sun, 13 May 2007 22:03:26 -0500, Michael Urman <[EMAIL PROTECTED]> wrote: > >On 5/13/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > >> The answer to all of this is the filesystem encoding, which is already > >> supported. Doesn't ap

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-14 Thread Jean-Paul Calderone
On Sun, 13 May 2007 22:03:26 -0500, Michael Urman <[EMAIL PROTECTED]> wrote: >On 5/13/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: >> The answer to all of this is the filesystem encoding, which is already >> supported. Doesn't appear particularly difficult to me. > >Okay, that's fair. It seems r

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-13 Thread Michael Urman
On 5/13/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > The answer to all of this is the filesystem encoding, which is already > supported. Doesn't appear particularly difficult to me. Okay, that's fair. It seems reasonable to accept the limitations of following the filesystem encoding for modul

Re: [Python-3000] Unicode strings, identifiers, and import

2007-05-13 Thread Guido van Rossum
The answer to all of this is the filesystem encoding, which is already supported. Doesn't appear particularly difficult to me. On 5/13/07, Michael Urman <[EMAIL PROTECTED]> wrote: > This occurred to me while reading the PEP 3131 discussion, and while > it's not limited to PEP 3131 concerns, I don'

[Python-3000] Unicode strings, identifiers, and import

2007-05-13 Thread Michael Urman
This occurred to me while reading the PEP 3131 discussion, and while it's not limited to PEP 3131 concerns, I don't believe I've seen discussed yet elsewhere. What is the interaction between import or __import__ and Unicode module names (or at least Unicode strings describing them). Currently in py