Re: [Python-3000] New io system and binary data

2007-09-25 Thread skip
Charles> There's another place in the tutorial section Charles> http://docs.python.org/dev/3.0/tutorial/inputoutput.html and Charles> search for "On Windows and the Macintosh, 'b' appended to the Charles> mode opens the file in binary mode," I fixed that up as well. I mentioned t

Re: [Python-3000] New io system and binary data

2007-09-25 Thread Greg Ewing
Bill Janssen wrote: > Now that we've moved away from the C stdio model, and the distinction > between text and binary streams is meaningful even on POSIX systems, > perhaps we should also change those names to reflect that difference > from C. I don't think anything would be gained by changing the

Re: [Python-3000] New io system and binary data

2007-09-25 Thread Guido van Rossum
On 9/25/07, Uche Ogbuji <[EMAIL PROTECTED]> wrote: > Bill Janssen wrote: > > That's certainly the prescribed behavior for the C stdio streams on > > POSIX-compliant systems. I think a lot of the original design of the > > Python I/O system was based on that C stdio system, including names > > like

Re: [Python-3000] New io system and binary data

2007-09-25 Thread Uche Ogbuji
Bill Janssen wrote: > That's certainly the prescribed behavior for the C stdio streams on > POSIX-compliant systems. I think a lot of the original design of the > Python I/O system was based on that C stdio system, including names > like stdin, stdout, and stderr. > > Now that we've moved away fro

Re: [Python-3000] New io system and binary data

2007-09-24 Thread Bill Janssen
> Also, if there are other places in the > > documentation > > where it seems to imply that the distinction between text and binary modes > > is > > meaningless on Unix systems, drop me a note and I'll have a look. That's certainly the prescribed behavior for the C stdio streams on POSIX-complian

Re: [Python-3000] New io system and binary data

2007-09-24 Thread Charles D Hixson
Skip Montanaro wrote: >> P.S.: If opening files on Linux is now to be semantically meaningful, >> then the documentation on that section also needs to change. Currently >> it appears to mean that it's a meaningless specification that will be >> ignored unless you happen to be using the MSWindows

Re: [Python-3000] New io system and binary data

2007-09-23 Thread Skip Montanaro
> P.S.: If opening files on Linux is now to be semantically meaningful, > then the documentation on that section also needs to change. Currently > it appears to mean that it's a meaningless specification that will be > ignored unless you happen to be using the MSWindows platform. I just checked

Re: [Python-3000] New io system and binary data

2007-09-23 Thread Martin v. Löwis
> The problem here seems to be that this isn't currently well documented. > I've got no objection to using the buffer attribute... Ok, then it seems you missed the obvious way: Open the file in binary mode ('rb' or 'wb') if you want to read or write bytes. It has always been that way in Python;

Re: [Python-3000] New io system and binary data

2007-09-23 Thread Charles D Hixson
Brett Cannon wrote: > On 9/19/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > >> Guido> You can repeat that until you're blue in the face but it's not >> Guido> going to change. Way more programs (especially simple ones) deal >> Guido> with txet than with binary data. >> >> For us

Re: [Python-3000] New io system and binary data

2007-09-23 Thread Martin v. Löwis
> So there needs to be an easy and obvious way to deal with files whose > records are arrays of byte valued data...that is commonly manipulated by > an editor using ascii-8. Did you follow the thread at all? There is an easy and obvious way to deal with such files. Regards, Martin _

Re: [Python-3000] New io system and binary data

2007-09-23 Thread Charles D Hixson
Guido van Rossum wrote: > On 9/19/07, Bill Janssen <[EMAIL PROTECTED]> wrote: > >> This really isn't a UTF-8 problem. It is the problem with file opens >> defaulting to "text" mode instead of "binary" mode rearing its ugly >> head again. >> > > You can repeat that until you're blue in the

Re: [Python-3000] New io system and binary data

2007-09-21 Thread Christian Heimes
Guido van Rossum wrote: > You can repeat that until you're blue in the face but it's not going > to change. Way more programs (especially simple ones) deal with txet > than with binary data. I have to agree with Guido. The new behavior is much better than the default in Python 2.x. It seems that I

Re: [Python-3000] New io system and binary data

2007-09-20 Thread Greg Ewing
Bill Janssen wrote: > Nice idea, but it would have been a tad more true to the origin of the > names if "stdin", "stderr", and "stdout" were binary (as the re-use of > those fine names automatically implies to anyone who knows what > they're doing) No, the names only imply that to Unix users who

Re: [Python-3000] New io system and binary data

2007-09-20 Thread Guido van Rossum
On 9/20/07, Bill Janssen <[EMAIL PROTECTED]> wrote: > Greg Ewing writes: > > Christian Heimes writes: > > > I recommend that the problem and fix gets documented. Maybe stdin, > > > stdout and stderr should get a method that disables the implicit > > > conversion like setMode("b") / setMode("t"). >

Re: [Python-3000] New io system and binary data

2007-09-20 Thread Bill Janssen
Greg Ewing writes: > Christian Heimes writes: > > I recommend that the problem and fix gets documented. Maybe stdin, > > stdout and stderr should get a method that disables the implicit > > conversion like setMode("b") / setMode("t"). > > Or maybe another set of objects called stdbin, stdbout, std

Re: [Python-3000] New io system and binary data

2007-09-20 Thread Eric Smith
Christian Heimes wrote: > Greg Ewing wrote: >> Christian Heimes wrote: >>> With Python 2.5 the input and output data isn't implicitly >>> converted >> Are you sure that's always true? What about systems >> where newlines aren't \n? > > Windows is a strange beast. As far as I can remember the OS co

Re: [Python-3000] New io system and binary data

2007-09-20 Thread Christian Heimes
Greg Ewing wrote: > Christian Heimes wrote: >> With Python 2.5 the input and output data isn't implicitly >> converted > > Are you sure that's always true? What about systems > where newlines aren't \n? Windows is a strange beast. As far as I can remember the OS converts the incoming and outgoing

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Greg Ewing
[EMAIL PROTECTED] wrote: > Binary has (like it or not) been the default behavior on all previous > Pythons running on Unix systems where text and binary were never different Um, no, *text* has always been the default on all systems. It's just that on systems where text and binary are the same, you

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Greg Ewing
Christian Heimes wrote: > With Python 2.5 the input and output data isn't implicitly > converted Are you sure that's always true? What about systems where newlines aren't \n? > I recommend that the problem and fix gets documented. Maybe stdin, > stdout and stderr should get a method that disables

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Guido van Rossum
Yeah, the PEP is pretty out of date (perhaps only surpassed by PEP 3135, super()). It's on my list to update it. This should definitely be added. On 9/19/07, Terry Reedy <[EMAIL PROTECTED]> wrote: > > "Guido van Rossum" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > | Changing the

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Terry Reedy
"Guido van Rossum" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] | Changing the mode between text and binary is not feasible (since it | would have to change the class). But it is perfectly acceptable to use | sys.std{in,out}.buffer if you need to write a binary transparent | filter

Re: [Python-3000] New io system and binary data

2007-09-19 Thread skip
Guido> I agree that binary pipelines are useful and should be Guido> possible. I just don't think this should be the default behavior Guido> for stdin/stdout. Binary has (like it or not) been the default behavior on all previous Pythons running on Unix systems where text and binary we

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Fred Drake
On Sep 19, 2007, at 5:00 PM, Guido van Rossum wrote: > Since the binary stream underlying stdin is readily available as > sys.stdin.buffer (and ditto for stdout and even stderr) I don't think > any action needs to be taken. note that the instance variable doesn't > start with an underscore. It's pa

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Guido van Rossum
On 9/19/07, Fred Drake <[EMAIL PROTECTED]> wrote: > On Sep 19, 2007, at 3:58 PM, Jason Orendorff wrote: > > Given the context (stdin/stdout/stderr), I'd love to know what you're > > thinking of here. I can't name a program offhand that wants to > > operate on binary data via a pipeline. There are

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Brett Cannon
On 9/19/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Guido> You can repeat that until you're blue in the face but it's not > Guido> going to change. Way more programs (especially simple ones) deal > Guido> with txet than with binary data. > > For us Unix-heads the notion that a

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Fred Drake
On Sep 19, 2007, at 3:58 PM, Jason Orendorff wrote: > Given the context (stdin/stdout/stderr), I'd love to know what you're > thinking of here. I can't name a program offhand that wants to > operate on binary data via a pipeline. There are a few that *can*, > like gzip, but my impression is that

Re: [Python-3000] New io system and binary data

2007-09-19 Thread skip
Jason> Given the context (stdin/stdout/stderr), I'd love to know what Jason> you're thinking of here. I can't name a program offhand that Jason> wants to operate on binary data via a pipeline. You've obviously never used the netpbm (nee pbmplus, nee pbm) tools. I still use this pipe

Re: [Python-3000] New io system and binary data

2007-09-19 Thread skip
Guido> You can repeat that until you're blue in the face but it's not Guido> going to change. Way more programs (especially simple ones) deal Guido> with txet than with binary data. For us Unix-heads the notion that a file is anything other than a stream of bytes is rather foreign. I

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Jason Orendorff
On 9/19/07, Bill Janssen <[EMAIL PROTECTED]> wrote: > > Way more programs (especially simple ones) deal with txet > > than with binary data. > > I'd love to see stats on that, Guido. I'm sure it's true in your > immediate vicinity, given what you work on, but I don't believe it's > true in general

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Bill Janssen
> You can repeat that until you're blue in the face but it's not going > to change. That happens to me a lot :-). > Way more programs (especially simple ones) deal with txet > than with binary data. I'd love to see stats on that, Guido. I'm sure it's true in your immediate vicinity, given what

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Guido van Rossum
On 9/19/07, Bill Janssen <[EMAIL PROTECTED]> wrote: > This really isn't a UTF-8 problem. It is the problem with file opens > defaulting to "text" mode instead of "binary" mode rearing its ugly > head again. You can repeat that until you're blue in the face but it's not going to change. Way more p

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Bill Janssen
GvR wrote: > I wouldn't do the assignments you propose > though, since that might surprise other code which expects text files. But presumably that code wouldn't be used in that same program. This really isn't a UTF-8 problem. It is the problem with file opens defaulting to "text" mode instead o

Re: [Python-3000] New io system and binary data

2007-09-19 Thread Guido van Rossum
Changing the mode between text and binary is not feasible (since it would have to change the class). But it is perfectly acceptable to use sys.std{in,out}.buffer if you need to write a binary transparent filter. Of course you'll be dealing with bytes at that point so the usual cautions apply. I wou

[Python-3000] New io system and binary data

2007-09-19 Thread Christian Heimes
Today I stumbled over another problem that is related to the unicode and OS string topic. The new io system - or to be more precisely the implicit converting of input and output data to UTF-8 makes it impossible to pipe binary data through Python 3.0. For example an user wants to write a filter fo