Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-19 Thread gabor
Martin v. Löwis wrote: gabor schrieb: depends on the application. in the one where it happened i would just display an error message, and tell the admins to check the filesystem-encoding. (in other ones, where it's not critical to get the correct name, i would probably just convert the text

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-19 Thread Fredrik Lundh
gabor wrote: yes, sure... but then.. it's possible to implement it also on top of an raise-when-error version :) not necessarily if raise-when-error means raise-error-in-os-listdir. /F -- http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-19 Thread gabor
Fredrik Lundh wrote: gabor wrote: yes, sure... but then.. it's possible to implement it also on top of an raise-when-error version :) not necessarily if raise-when-error means raise-error-in-os-listdir. could you please clarify? currently i see 2 approaches how to do it on the

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-19 Thread Martin v. Löwis
gabor schrieb: 1. simply fix the documentation, and state that if the file-name cannot be decoded into unicode, then it's returned as byte-string. For 2.5, this should be done. Contributions are welcome. [...then] [os.path.join(path,n) for n in os.listdir(path)] will not work. 2. add

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-19 Thread gabor
Martin v. Löwis wrote: gabor schrieb: 1. simply fix the documentation, and state that if the file-name cannot be decoded into unicode, then it's returned as byte-string. For 2.5, this should be done. Contributions are welcome. [...then] [os.path.join(path,n) for n in os.listdir(path)]

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-19 Thread Martin v. Löwis
gabor schrieb: I may have missed something, but did you present a solution that would make the case above work? if we use the same decoding flags as binary-string.decode(), then we could do: [os.path.join(path,n) for n in os.listdir(path,'ignore')] That wouldn't work. The characters in

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-19 Thread gabor
Martin v. Löwis wrote: gabor schrieb: I may have missed something, but did you present a solution that would make the case above work? if we use the same decoding flags as binary-string.decode(), then we could do: [os.path.join(path,n) for n in os.listdir(path,'ignore')] That wouldn't

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-18 Thread Leo Kislov
Martin v. Löwis wrote: Leo Kislov schrieb: How about returning two lists, first list contains unicode names, the second list contains undecodable names: files, troublesome = os.listdir(separate_errors=True) and make separate_errors=True by default in python 3.0 ? That would be quite

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Laurent Pointal
gabor a écrit : hi, from the documentation (http://docs.python.org/lib/os-file-dir.html) for os.listdir: On Windows NT/2k/XP and Unix, if path is a Unicode object, the result will be a list of Unicode objects. Maybe, for each filename, you can test if it is an unicode string, and if not,

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Marc 'BlackJack' Rintsch
In [EMAIL PROTECTED], Jean-Paul Calderone wrote: How would you propose listdir should behave? Umm, just a wild guess, but how about raising an exception which includes the name of the file which could not be decoded? Suppose you have a directory with just some files having a name that can't

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread gabor
Laurent Pointal wrote: Laurent Pointal wrote: gabor a écrit : hi, from the documentation (http://docs.python.org/lib/os-file-dir.html) for os.listdir: On Windows NT/2k/XP and Unix, if path is a Unicode object, the result will be a list of Unicode objects. Maybe, for each filename, you

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Leo Kislov
Martin v. Löwis wrote: gabor schrieb: All this code will typically work just fine with the current behavior, so people typically don't see any problem. i am sorry, but it will not work. actually this is exactly what i did, and it did not work. it dies in the os.path.join call, where

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread gabor
Marc 'BlackJack' Rintsch wrote: In [EMAIL PROTECTED], Jean-Paul Calderone wrote: How would you propose listdir should behave? Umm, just a wild guess, but how about raising an exception which includes the name of the file which could not be decoded? Suppose you have a directory with just

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread gabor
Fredrik Lundh wrote: gabor wrote: get an Unicode-exception, as everywhere else. you see, exceptions are ok, i can deal with them. p.s: one additional note. if you code expects os.listdir to return unicode, that usually means that all your code uses unicode strings. which in turn

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread gabor
Martin v. Löwis wrote: gabor schrieb: i also recommend this approach. also, raising an exception goes well with the principle of the least surprise imho. Are you saying you wouldn't have been surprised if that had been the behavior? yes, i would not have been surprised. because it's

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Leo Kislov
gabor wrote: Martin v. Löwis wrote: gabor schrieb: i also recommend this approach. also, raising an exception goes well with the principle of the least surprise imho. Are you saying you wouldn't have been surprised if that had been the behavior? yes, i would not have been

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Johan von Boisman
Laurent Pointal wrote: gabor a écrit : hi, from the documentation (http://docs.python.org/lib/os-file-dir.html) for os.listdir: On Windows NT/2k/XP and Unix, if path is a Unicode object, the result will be a list of Unicode objects. Maybe, for each filename, you can test if it is an

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Marc 'BlackJack' Rintsch
In [EMAIL PROTECTED], gabor wrote: Marc 'BlackJack' Rintsch wrote: In [EMAIL PROTECTED], Jean-Paul Calderone wrote: How would you propose listdir should behave? Umm, just a wild guess, but how about raising an exception which includes the name of the file which could not be decoded?

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Martin v. Löwis
Leo Kislov schrieb: How about returning two lists, first list contains unicode names, the second list contains undecodable names: files, troublesome = os.listdir(separate_errors=True) and make separate_errors=True by default in python 3.0 ? That would be quite an incompatible change, no?

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Martin v. Löwis
gabor schrieb: depends on the application. in the one where it happened i would just display an error message, and tell the admins to check the filesystem-encoding. (in other ones, where it's not critical to get the correct name, i would probably just convert the text to unicode using the

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-17 Thread Fredrik Lundh
Martin v. Löwis wrote: How about returning two lists, first list contains unicode names, the second list contains undecodable names: files, troublesome = os.listdir(separate_errors=True) and make separate_errors=True by default in python 3.0 ? That would be quite an incompatible change,

os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread gabor
hi, from the documentation (http://docs.python.org/lib/os-file-dir.html) for os.listdir: On Windows NT/2k/XP and Unix, if path is a Unicode object, the result will be a list of Unicode objects. i'm on Unix. (linux, ubuntu edgy) so it seems that it does not always return unicode filenames.

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread Terry Reedy
gabor [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] so if the to-unicode-conversion fails, it falls back to the original byte-string. i went and have read the patch-discussion. and now i'm not sure what to do. i know that: 1. the documentation is completely wrong. it does not

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread Martin v. Löwis
gabor schrieb: so basically i'd like to ask here: am i reading something incorrectly? You are reading it correctly. This is how it behaves. or am i using os.listdir the wrong way? how do other people deal with this? You didn't say why the behavior causes a problem for you - you only

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread gabor
Martin v. Löwis wrote: gabor schrieb: or am i using os.listdir the wrong way? how do other people deal with this? You didn't say why the behavior causes a problem for you - you only explained what the behavior is. Most people use os.listdir in a way like this: for name in

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread Martin v. Löwis
gabor schrieb: All this code will typically work just fine with the current behavior, so people typically don't see any problem. i am sorry, but it will not work. actually this is exactly what i did, and it did not work. it dies in the os.path.join call, where file_name is converted into

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread Jean-Paul Calderone
On Fri, 17 Nov 2006 00:31:06 +0100, \Martin v. Löwis\ [EMAIL PROTECTED] wrote: gabor schrieb: All this code will typically work just fine with the current behavior, so people typically don't see any problem. i am sorry, but it will not work. actually this is exactly what i did, and it did

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread gabor
Jean-Paul Calderone wrote: On Fri, 17 Nov 2006 00:31:06 +0100, \Martin v. Löwis\ [EMAIL PROTECTED] wrote: gabor schrieb: All this code will typically work just fine with the current behavior, so people typically don't see any problem. i am sorry, but it will not work. actually this is

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread Martin v. Löwis
Jean-Paul Calderone schrieb: How would you propose listdir should behave? Umm, just a wild guess, but how about raising an exception which includes the name of the file which could not be decoded? There may be multiple of these, of course, but I assume that you want it to report the first

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread Martin v. Löwis
gabor schrieb: i also recommend this approach. also, raising an exception goes well with the principle of the least surprise imho. Are you saying you wouldn't have been surprised if that had been the behavior? How would you deal with that exception in your code? Regards, Martin --

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

2006-11-16 Thread Fredrik Lundh
gabor wrote: get an Unicode-exception, as everywhere else. you see, exceptions are ok, i can deal with them. p.s: one additional note. if you code expects os.listdir to return unicode, that usually means that all your code uses unicode strings. which in turn means, that those filenames