Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Lucas P Melo wrote: Am I understanding this correctly: * The blocking version would not do any raw reads. No, the blocking version would keep doing raw reads until the buffer contains enough bytes. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Greg Ewing wrote: That's exactly why I think the blocking version should keep reading until the requested number of bytes is available (or the buffer is full or EOF occurs). Do you mean that the blocking version should keep waiting for new bytes until they show up? This would not be acceptable, since the program would hang forever most of the time (no changes to the buffer would ever occur in this situation when there's only the main thread running). Am I understanding this correctly: * The blocking version would not do any raw reads. * The non-blocking version would do. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Greg Ewing wrote: Even if you don't mention it explicitly, its existence shows through in the fact that there is an arbitrary limit on the amount you can peek ahead, and that limit needs to be documented so that people can write correct programs. This is true of both kinds of peeking, so I concede that they both break the abstraction. However I think the non-blocking peek breaks it more than the blocking one, because it also brings non-deterministic behaviour. It depends on the point of view. For example, someone is writing a program that must read from any kind of file descriptor and generate the derivation tree of the text read based on some context-free grammar. The problem is that the chosen method to accomplish it would read 2 symbols (bytes) ahead and this guy is using peek() to grab these 2 bytes. The program will seem to work correctly most of the time, but on the 4095th byte read, he would grab 1 byte at most using peek() (despite the existence of say 10k bytes ahead). The blocking definition of peek() would create this hard-to-spot bug. Thus, for him, this behavior would seem non-deterministic. On the other hand, if someone is conscious about the number of raw reads, he would definitely be willing to look into the documentation for any parameters that match his special needs. That's why the non-blocking behavior should be the default one while the blocking behavior should be accessible by choice. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Lucas P Melo wrote: The problem is that the chosen method to accomplish it would read 2 symbols (bytes) ahead and this guy is using peek() to grab these 2 bytes. The program will seem to work correctly most of the time, but on the 4095th byte read, he would grab 1 byte at most using peek() That's exactly why I think the blocking version should keep reading until the requested number of bytes is available (or the buffer is full or EOF occurs). In other words, the blocking version should be fully deterministic given knowledge of the buffer size. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: But people not using threads, or at any rate not dedicating a thread to the reading task, don't have such luxury. But without a dedicated thread you need to use select() or poll(), and then buffering causes other headaches. Are we disputing the utility of being able to ask how much might I read/peek without blocking? I'm saying that I don't see how I would make use of such a thing, so I probably wouldn't mind if it didn't exist. Or disputing the purpose of peek, which feels to me like it should/might ask that question, but doesn't. I think what I'm saying is that there are two distinct use cases being talked about for a peek-like operation, and different people seem to have different ideas on which one should be mapped to the name peek. So perhaps they should both be given more-explicit names. It's perfectly possible, even reasonable, to avoid talking about the buffer at all. I'd be happy not to mention the buffer. Even if you don't mention it explicitly, its existence shows through in the fact that there is an arbitrary limit on the amount you can peek ahead, and that limit needs to be documented so that people can write correct programs. This is true of both kinds of peeking, so I concede that they both break the abstraction. However I think the non-blocking peek breaks it more than the blocking one, because it also brings non-deterministic behaviour. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: On 16Jun2009 02:18, MRAB pyt...@mrabarnett.plus.com wrote: My itch is that peek() _feels_ like it should be look into the buffer but actually can block and/or change the buffer. Can block, but not if you don't want it too. You might just want to see what, if anything, is currently available, up to n bytes. Am I missing something? In the face of an _empty_ buffer (which I can't tell from outside) how do I prevent peek() blocking? More generally, if I go peek(n) and if n bytes_in_buffer_right_now and the raw stream would block if a raw read is done? My concerns would go away if I could probe the buffer content size; then I could ensure peek(n) chose n = the content size. If that's not enough, my problem - I can choose to read-and-block or go away and come back later. I was thinking along the lines of: def peek(self, size=None, block=True) If 'block' is True then return 'size' bytes, unless the end of the file/stream is reached; if 'block' is False then return up to 'size' bytes, without blocking. The blocking form might impose a limit to how much can be peeked (the maximum size of the buffer), or it might enlarge the buffer as necessary. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: Indeed, though arguably read1() is a lousy name too, on the same basis. My itch is that peek() _feels_ like it should be look into the buffer but actually can block and/or change the buffer. I guess all the buffer operations should be transparent to the user if he wants it to be like that, since not so many people want to have a tight control over this kind of detail. I think of peek() as an operation that allows me to peek what's going to show up in the future without affecting further read()s. This kind of behavior is expected by users without prior knowledge of the inner workings of buffered IO. So, if an user _really_ wants to take a look at what's to come without affecting the buffer, we could allow that by doing something like this: peek(5, change_buffer=False) This is an alternative to the peek0(). But I am ok wih the peek0() too. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
MRAB wrote: I was thinking along the lines of: def peek(self, size=None, block=True) I think this is fine too. :) If 'block' is True then return 'size' bytes, unless the end of the file/stream is reached; if 'block' is False then return up to 'size' bytes, without blocking. The blocking form might impose a limit to how much can be peeked (the maximum size of the buffer), or it might enlarge the buffer as necessary. I guess the limit wouldn't be a problem to someone that chose to block further reads. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On approximately 6/16/2009 11:20 AM, came the following characters from the keyboard of Scott David Daniels: MRAB wrote: I was thinking along the lines of: def peek(self, size=None, block=True) If 'block' is True then return 'size' bytes, unless the end of the file/stream is reached; if 'block' is False then return up to 'size' bytes, without blocking I tend to prefer zero-ish defaults, how about: def peek(self, size=None, nonblocking=False) No, no, no! Double negatives are extremely easy to not code correctly. The lack of ease of not understanding of code containing double negatives quadruples, at least! Not so differently, I'm sure my sentences here are not easy to understand because I put the inverse logic in them in the places that are not the usual. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Scott David Daniels Scott.Daniels at Acm.Org writes: MRAB wrote: I was thinking along the lines of: def peek(self, size=None, block=True) If 'block' is True then return 'size' bytes, unless the end of the file/stream is reached; if 'block' is False then return up to 'size' bytes, without blocking I tend to prefer zero-ish defaults, how about: def peek(self, size=None, nonblocking=False) Since blocking and non-blocking are already used to refer to different types of raw streams, another verb should be found for this option. Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: I normally avoid non-blocking requirements by using threads, so that the thread gathering from the stream can block. If you have a thread dedicated to reading from that stream, then I don't see why you need to peek into the buffer. Just have it loop reading a packet at a time and put each completed packet in the queue. If several packets arrive at once, it'll loop around that many times before blocking. arguably read1() is a lousy name too, on the same basis. Certainly. My itch is that peek() _feels_ like it should be look into the buffer but actually can block and/or change the buffer. My problem with the idea of looking into the buffer is that it crosses levels of abstraction. A buffered stream is supposed to behave the same way as a deterministic non-buffered stream, with the buffer being an internal optimisation detail that doesn't exist as far as the outside world is concerned. Sometimes it's pragmatic to break the abstraction, but it should be made very obvious when you're doing that. So I'd prefer something like peek_buffer() to make it perfectly clear what's being done. Anything else such as peek() that doesn't explicitly mention the buffer should fit into the abstraction properly. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Greg Ewing greg.ewing at canterbury.ac.nz writes: Anything else such as peek() that doesn't explicitly mention the buffer should fit into the abstraction properly. peek() doesn't fit into the abstraction since it doesn't even exist on raw streams. While buffered and non-buffered streams have a reasonably similar API, expecting them to behave the same in all circumstances is IMO unrealistic. Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 17Jun2009 10:55, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Cameron Simpson wrote: I normally avoid non-blocking requirements by using threads, so that the thread gathering from the stream can block. If you have a thread dedicated to reading from that stream, then I don't see why you need to peek into the buffer. Just have it loop reading a packet at a time and put each completed packet in the queue. If several packets arrive at once, it'll loop around that many times before blocking. Yes, this is true. But people not using threads, or at any rate not dedicating a thread to the reading task, don't have such luxury. Are we disputing the utility of being able to ask how much might I read/peek without blocking? Or disputing the purpose of peek, which feels to me like it should/might ask that question, but doesn't. [...] My itch is that peek() _feels_ like it should be look into the buffer but actually can block and/or change the buffer. My problem with the idea of looking into the buffer is that it crosses levels of abstraction. A buffered stream is supposed to behave the same way as a deterministic non-buffered stream, with the buffer being an internal optimisation detail that doesn't exist as far as the outside world is concerned. Sometimes it's pragmatic to break the abstraction, but it should be made very obvious when you're doing that. So I'd prefer something like peek_buffer() to make it perfectly clear what's being done. Anything else such as peek() that doesn't explicitly mention the buffer should fit into the abstraction properly. It's perfectly possible, even reasonable, to avoid talking about the buffer at all. I'd be happy not to mention the buffer. For example, one can readily imagine the buffered stream being capable of querying its input raw stream if there's available now data. The raw stream can sometimes know if a read of a given size would block, or be able to ask what size read will not block. As a concrete example, the UNIX FIONREAD ioctl can generally query a file descriptor for instantly-available data (== in the OS buffer). I've also used UNIXen where your can fstat() a pipe and use the st_size field to test for available unread data in the pipe buffer. Raw streams which can't do that would return 0 (== can't guarentee any non-blocking data) unless the stream itself also had a buffer of its own and it wasn't empty. So I would _want_ the spec for available_data() (new lousy name) to talk about data available without blocking, allowing the implementation to use data in the IO buffer and/or to query the raw stream, etc. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ For those who understand, NO explanation is needed, for those who don't understand, NO explanation will be given! - Davey D decos...@vnet.ibm.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On Sat, 13 Jun 2009 12:33:46 + (UTC) Antoine Pitrou solip...@pitrou.net wrote: This proposal looks reasonable to me. Please note that it's too late for 3.1 anyway - we're in release candidate phase. Once you have a patch, you can post it on the bug tracker. Thanks I will do that. Sometime in the next couple of weeks. Gratefully Frederick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: It seems like whenever I want to do some kind of opportunistic but non-blocking stuff with a remote service Do you actually do this with buffered streams? I find it's better to steer well clear of buffered I/O objects when doing non-blocking stuff, because they don't play well with other things like select(). Anyhow, I wouldn't be opposed to having a way of looking into the buffer, but it should be a separate API -- preferably with a better name than peek0(), which gives no clue at all about what it does differently from peek(). -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 16Jun2009 11:21, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Cameron Simpson wrote: It seems like whenever I want to do some kind of opportunistic but non-blocking stuff with a remote service Do you actually do this with buffered streams? Sure, in C, python and perl quite happily. In some circumstances. Provided you're careful about when to fflush() it can all go quite smoothly. It's certainly not applicable to everything. I find it's better to steer well clear of buffered I/O objects when doing non-blocking stuff, because they don't play well with other things like select(). Ah, the non-blockingness. Well that's the rub. I normally avoid non-blocking requirements by using threads, so that the thread gathering from the stream can block. Then the consumer can poll a Queue from the worker thread, etc. I really don't like select(/poll/epoll etc) much; aside from select's scaling issues with lots of files (hence poll/epoll) there are high performance things where having an event loop using select is a good way to go, but I generally prefer using threads and/or generators to make the code clear (single flow of control, single task for the block of code, etc) if there's no reason not to. Anyhow, I wouldn't be opposed to having a way of looking into the buffer, but it should be a separate API -- preferably with a better name than peek0(), which gives no clue at all about what it does differently from peek(). Indeed, though arguably read1() is a lousy name too, on the same basis. My itch is that peek() _feels_ like it should be look into the buffer but actually can block and/or change the buffer. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ You can't wait for inspiration. You have to go after it with a club. - Jack London ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: On 16Jun2009 11:21, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Cameron Simpson wrote: It seems like whenever I want to do some kind of opportunistic but non-blocking stuff with a remote service Do you actually do this with buffered streams? Sure, in C, python and perl quite happily. In some circumstances. Provided you're careful about when to fflush() it can all go quite smoothly. It's certainly not applicable to everything. I find it's better to steer well clear of buffered I/O objects when doing non-blocking stuff, because they don't play well with other things like select(). Ah, the non-blockingness. Well that's the rub. I normally avoid non-blocking requirements by using threads, so that the thread gathering from the stream can block. Then the consumer can poll a Queue from the worker thread, etc. I really don't like select(/poll/epoll etc) much; aside from select's scaling issues with lots of files (hence poll/epoll) there are high performance things where having an event loop using select is a good way to go, but I generally prefer using threads and/or generators to make the code clear (single flow of control, single task for the block of code, etc) if there's no reason not to. Anyhow, I wouldn't be opposed to having a way of looking into the buffer, but it should be a separate API -- preferably with a better name than peek0(), which gives no clue at all about what it does differently from peek(). Indeed, though arguably read1() is a lousy name too, on the same basis. My itch is that peek() _feels_ like it should be look into the buffer but actually can block and/or change the buffer. Can block, but not if you don't want it too. You might just want to see what, if anything, is currently available, up to n bytes. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 16Jun2009 02:18, MRAB pyt...@mrabarnett.plus.com wrote: My itch is that peek() _feels_ like it should be look into the buffer but actually can block and/or change the buffer. Can block, but not if you don't want it too. You might just want to see what, if anything, is currently available, up to n bytes. Am I missing something? In the face of an _empty_ buffer (which I can't tell from outside) how do I prevent peek() blocking? More generally, if I go peek(n) and if n bytes_in_buffer_right_now and the raw stream would block if a raw read is done? My concerns would go away if I could probe the buffer content size; then I could ensure peek(n) chose n = the content size. If that's not enough, my problem - I can choose to read-and-block or go away and come back later. -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ If all around you is darkness and you feel you're contending in vain, then the light at the end of the tunnel is the front of an oncoming train. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
2009/6/14 Cameron Simpson c...@zip.com.au: On 14Jun2009 15:16, I wrote: | Is it possible to access the buffer? I see nothing in the docs. I've just found getvalue() in IOBase. Forget I said anything. It seems to be my day for that kind of post:-( Where are you seeing this? Only BytesIO and StringIO have a getvalue() method. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: For myself, I'd expect more often to want to see if there's stuff in the buffer _without_ doing any raw reads at all. What uses do you have in mind for that? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 14Jun2009 09:21, Benjamin Peterson benja...@python.org wrote: | 2009/6/14 Cameron Simpson c...@zip.com.au: | On 14Jun2009 15:16, I wrote: | | Is it possible to access the buffer? I see nothing in the docs. | | I've just found getvalue() in IOBase. Forget I said anything. | It seems to be my day for that kind of post:-( | | Where are you seeing this? Only BytesIO and StringIO have a getvalue() method. I had thought I'd traced it by class inheritance. But I got BytesIO and IOBase confused. So: no getvalue then. So probably there is a case for peek0(), which never does a raw read. Thoughts? -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ I was gratified to be able to answer promptly and I did. I said I didn't know. - Mark Twain ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 15Jun2009 11:48, Greg Ewing greg.ew...@canterbury.ac.nz wrote: For myself, I'd expect more often to want to see if there's stuff in the buffer _without_ doing any raw reads at all. What uses do you have in mind for that? It seems like whenever I want to do some kind of opportunistic but non-blocking stuff with a remote service (eg something I have a packetising connection to, such as a TCP stream) I want to be able to see if there's more stuff to gather up before issuing a flush operation. And I want to be able to do that in a non-blocking way, much as a Queue has a non-blocking get() method. As an example, I've got the occasional protocol handler where it has to make a remote query. To avoid deadlock, the stream must be flushed after write()ing the query packet. However, flushing on every such packet is horribly wasteful if you know you have a bunch of them (for example, the caller is asking a bunch of questions). It is far far more efficient to write() each packet without flushes, keep the knowledge that a flush is needed, and flush when there's nothing more pending. That way the lower layer has maximum opportunity to pack data into packets. All that presumes another thread reading responses, which is how I generally write this stuff anyway, otherwise a full buffer will deadlock too. So your dispatch thread inner loop looks like this: # single consumer, so Q.empty() implies ok to Q.get() needFlush = False while not Q.empty(): P=Q.get() if P.needFlush: needFlush = True out.write(P.serialise()) if needFlush: out.flush() In this scheme, there _are_ packets that don't need a flush, because nobody is waiting on their response. Anyway, if I were reading from an IO object instead of a Queue I'd want to poll for buffer empty. If there isn't an empty buffer I know there will be a packet worth of data coming immediately and I can pick it up with regular read()s, just as I'm doing with Q.get() above. But if the buffer is empty I can drop out of the pick it all up now loop and flush(). Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ If you take something apart and put it back together again enough times, you will eventually have enough parts left over to build a second one. - Ayse Sercan a...@netcom.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: On 13Jun2009 12:24, Nick Coghlan ncogh...@gmail.com wrote: | I would phrase this suggestion as users having a reasonable expectation | that the following invariant should hold for a buffered stream: | | f.peek(n) == f.read(n) | | Since the latter method will perform as many reads of the underlying | stream as necessary to reach the requested number of bytes (or EOF), | then so should the former. I disagree. If that were that case, why have peek() at all? I realise that it doesn't move the logical position, but it does mean that peek(huge_number) imposes a requirement to grow the stream buffer arbitrarily. A peek that does at most one raw read has the advantage that it can pick up data outside the buffer but lurking in the OS buffer, yet to be obtained. Those data are free, if they're present. (Of course, if they're absent peek() wil still block). Note my suggestion later that if the above invariant were to be adopted then a peek1() method should be added to parallel read1(). However, from what Benjamin has said, a more likely invariant is going to be: preview = f.peek(n) f.read(n).startswith(preview) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Nick Coghlan ncoghlan at gmail.com writes: Since the latter method will perform as many reads of the underlying stream as necessary to reach the requested number of bytes (or EOF), then so should the former. How do you propose to implement this while staying compatible with 1) unseekable raw streams 2) the expectation that peek() doesn't advance the logical file pointer? Note that the current behaviour I get from Python 3.1 is for it to return the *entire* buffer, no matter what number I pass to it: [...] That's an outright bug - I've promoted an existing issue about this [1] to a release blocker and sent it to Benjamin to have another look at. The original docstring for peek() says: Returns buffered bytes without advancing the position. The argument indicates a desired minimal number of bytes; we do at most one raw read to satisfy it. We never return more than self.buffer_size. In that light, I'm not sure it's a bug -- although it can certainly look unexpected at first sight. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Frederick Reeve cylix at solace.info writes: peek(n): If n is less than 0, None, or not set; return buffer contents with out advancing stream position. If the buffer is empty read a full chunk and return the buffer. Otherwise return exactly n bytes up to _chunk size_(not contents) with out advancing the stream position. If the buffer contents is less than n, buffer an additional chunk from the raw stream before hand. If EOF is encountered during any raw read then return as much as we can up to n. (maybe I should write that in code form??) This proposal looks reasonable to me. Please note that it's too late for 3.1 anyway - we're in release candidate phase. Once you have a patch, you can post it on the bug tracker. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Antoine Pitrou wrote: The original docstring for peek() says: ...we do at most one raw read to satisfy it. In that light, I'm not sure it's a bug It may be behaving according to the docs, but is that behaviour useful? Seems to me that if you're asking for n bytes, then it's because you're doing some kind of parsing that requires lookahead, and nothing less than n bytes will do. I think it would be more useful if the at most one raw read part were dropped. That would give it the kind of deterministic behaviour generally expected when dealing with buffered streams. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Greg Ewing greg.ewing at canterbury.ac.nz writes: I think it would be more useful if the at most one raw read part were dropped. That would give it the kind of deterministic behaviour generally expected when dealing with buffered streams. As I already told Nick: please propose an implementation scheme. Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 14Jun2009 12:33, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Antoine Pitrou wrote: The original docstring for peek() says: ...we do at most one raw read to satisfy it. In that light, I'm not sure it's a bug It may be behaving according to the docs, but is that behaviour useful? Seems to me that if you're asking for n bytes, then it's because you're doing some kind of parsing that requires lookahead, and nothing less than n bytes will do. I think it would be more useful if the at most one raw read part were dropped. That would give it the kind of deterministic behaviour generally expected when dealing with buffered streams. Is it possible to access the buffer? I see nothing in the docs. People seem to want peek() to be read() without moving the read offset, which it almost seems to be. Nick and Greg both want it to really be that, and thus do enough raw reads to get n bytes; Nick wants a peek1() like read1(), too. It has a pleasing feel to me, too. But ... For myself, I'd expect more often to want to see if there's stuff in the buffer _without_ doing any raw reads at all. A peek0(n), if you will: Read and return up to n bytes without calling on the raw stream. It feels like peek is trying to span both extremes and doesn't satisfy either really well. If peek gets enhanced to act like read in terms of the amount of data returned, should there not be a facility to examine buffered data without raw reads? Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ Being on a Beemer and not having a wave returned by a Sportster is like having a clipper ship's hailing not returned by an orphaned New Jersey solid waste barge. - OTL ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 14Jun2009 15:16, I wrote: | Is it possible to access the buffer? I see nothing in the docs. I've just found getvalue() in IOBase. Forget I said anything. It seems to be my day for that kind of post:-( -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ These are but a few examples of what can happen when the human mind is employed to learn, to probe, to question as opposed to merely keeping the ears from touching. - rec.humor.funny 90.07.16 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
