Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Travis E. Oliphant wrote: > > Attached is my current draft of the enhanced buffer protocol for Python > 3000. It is basically what has been discussed except for some issues > with non single-segment memory areas (such as a sub-array). > > Comments are welcome. > The latest version of the PEP

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Greg Ewing wrote: > > The buffer API is for the use of C code, and it should > be designed with the convenience of C code in mind. > Using Python data structures unnecessarily seems like > the wrong way to go about that. > > The following alternative would seem to provide most of > the things tha

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Travis E. Oliphant wrote: > > The memory management is the only issue. In fact, the PEP still has the issue of who manages the memory for the format-description string when it is returned. The easiest thing to do is to return a Python String and let reference counting handle the memory managem

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Greg Ewing wrote: > Travis E. Oliphant wrote: > >> typedef char *(*formatbufferproc)(PyObject *view, int *itemsize) >> >> Get the format-string of the memory using the struct-module >> string syntax > > I'm not sure whether a struct-format string would be > the most convenient for

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Greg Ewing
Josiah Carlson wrote: > About the only issue I can see with implementing the mechanism as you > describe is that everything that wants to offer the buffer interface > would need to store its data in a PyArray structure. And that wouldn't be acceptable, because the point of the buffer interface is

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Greg Ewing
Travis E. Oliphant wrote: > typedef char *(*formatbufferproc)(PyObject *view, int *itemsize) > > Get the format-string of the memory using the struct-module > string syntax I'm not sure whether a struct-format string would be the most convenient form for use by C-level code, as i

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Nick Coghlan
Greg Ewing wrote: > Nick Coghlan wrote: >> I'd also like to know what should happen if >> the object's Python refcount drops to zero, but the view count is still >> greater than 0. > > That shouldn't happen, because the code using the view > ought to be responsible for holding a reference to the

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Greg Ewing
Nick Coghlan wrote: > I'd also like to know what should happen if > the object's Python refcount drops to zero, but the view count is still > greater than 0. That shouldn't happen, because the code using the view ought to be responsible for holding a reference to the containing object as long as

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Daniel Stutzbach wrote: > I know I'm joining this discussion late in the game, so I apologize if > my look through the list archives was not sufficiently exhaustive and > this has been proposed and shot down before... No, I don't think you are late. But this discussion has been going on off and

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Daniel Stutzbach
On 2/27/07, Josiah Carlson <[EMAIL PROTECTED]> wrote: > About the only issue I can see with implementing the mechanism as you > describe is that everything that wants to offer the buffer interface > would need to store its data in a PyArray structure. Bytes, unicode, > array.array, mmap, etc. Mos

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Josiah Carlson
"Daniel Stutzbach" <[EMAIL PROTECTED]> wrote: [snip] > The container cannot reallocate the memory unless the reference > counter on the PyArray is exactly 1. Alternatively, some objects could opt to create a new PyArray of sufficient size, copy data as necessary, and leave all previous views to p

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Nick Coghlan
Daniel Stutzbach wrote: > When an array-like container allocates memory, it allocates a PyArray > to store the actual data in. When a caller request a view, the > container increments the PyArray's reference counter and returns a > pointer to the PyArray. The caller is responsible for decrementin

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Daniel Stutzbach
I know I'm joining this discussion late in the game, so I apologize if my look through the list archives was not sufficiently exhaustive and this has been proposed and shot down before... What if the locking mechanism were put into the array's memory instead of the container's memory? If the arra

[Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Attached is my current draft of the enhanced buffer protocol for Python 3000. It is basically what has been discussed except for some issues with non single-segment memory areas (such as a sub-array). Comments are welcome. -Travis Oliphant PEP: Title: Revising the buffer protocol Version:

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Greg Ewing
Guido van Rossum wrote: > On 2/27/07, Greg Ewing <[EMAIL PROTECTED]> wrote: > > > What about \r? > > Mac OS 9 has been dead and unsupported for many years now. Even if the Python code isn't running on MacOS 9, it might want to write a file that will be read by a MacOS 9 system. -- Greg _

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
On 2/27/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > On 2/27/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > On 2/27/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > > > > Therefore, normal code can ignore the possibility, or (to be really > > > robust against someone else messing with the input stre

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
On 2/27/07, Greg Ewing <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > > I'd like to constrain newline to be either \n or \r\n for writing; > > What about \r? Mac OS 9 has been dead and unsupported for many years now. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Greg Ewing
Guido van Rossum wrote: > I'd like to constrain newline to be either \n or \r\n for writing; What about \r? -- Greg ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Jim Jewett
On 2/27/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 2/27/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > > Therefore, normal code can ignore the possibility, or (to be really > > robust against someone else messing with the input stream) add an "if > > result is None: continue" clause to its

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Walter Dörwald
Guido van Rossum wrote: > On 2/27/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > [...] >> The basic principle is that these codecs can encode strings and decode >> bytes in multiple chunks. If you want to encode a unicode string u in >> UTF-16 you can do it in one go: >> s = u.encode("utf-16"

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
On 2/27/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > > On 2/27/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > >> Guido van Rossum wrote: > >> > >> > The encoding/decoding behavior should be no different from that of the > >> > encode() and decode() methods on unicode

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Walter Dörwald
Guido van Rossum wrote: > On 2/27/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: >> Guido van Rossum wrote: >> >> > The encoding/decoding behavior should be no different from that of the >> > encode() and decode() methods on unicode strings and byte arrays. >> >> Except that it must work in increme

Re: [Python-3000] pre-PEP: Enhancing buffer protocol (tp_as_buffer)

2007-02-27 Thread Guido van Rossum
On 2/27/07, Travis E. Oliphant <[EMAIL PROTECTED]> wrote: > My understanding of this locking mechanism would require objects that > wish to use it to keep track of how many views they have "exported" and > refuse to re-allocate memory until the views have all been released. Right. > In my underst

Re: [Python-3000] pre-PEP: Enhancing buffer protocol (tp_as_buffer)

2007-02-27 Thread Travis E. Oliphant
Guido van Rossum wrote: > On 2/26/07, Travis Oliphant <[EMAIL PROTECTED]> wrote: >> Guido van Rossum wrote: >> Great. I have no problem with this. Is your idea of locking the same >> as mine (i.e. a function in the API for release?) > > Right. My understanding of this locking mechanism would re

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
On 2/27/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > > The encoding/decoding behavior should be no different from that of the > > encode() and decode() methods on unicode strings and byte arrays. > > Except that it must work in incremental mode. The new (in 2.5) > inc

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Walter Dörwald
Guido van Rossum wrote: > The encoding/decoding behavior should be no different from that of the > encode() and decode() methods on unicode strings and byte arrays. Except that it must work in incremental mode. The new (in 2.5) incremental codecs should be usable for that. > Certainly no normal

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
On 2/27/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > On 2/27/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > On 2/27/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > > > > Trying to set (non-)blocking may raise an exception on some streams, > > > but that still seems better than having to know the in

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Jim Jewett
On 2/27/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 2/27/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > > Trying to set (non-)blocking may raise an exception on some streams, > > but that still seems better than having to know the internal details > > before you can even ask. > I doubt it -

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
The encoding/decoding behavior should be no different from that of the encode() and decode() methods on unicode strings and byte arrays. Certainly no normalization of diacritics will be done; surrogate handling depends on the encoding and whether the unicode string implementation uses 16 or 32 bit

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
On 2/27/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > On 2/27/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > On 2/27/07, Paul Moore <[EMAIL PROTECTED]> wrote: > > > It *may* even be useful if many of these would support non-blocking > > I/O; we're currently considering adding a standard API for

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Jim Jewett
On 2/27/07, Adam Olsen <[EMAIL PROTECTED]> wrote: > On 2/26/07, Mike Verdone <[EMAIL PROTECTED]> wrote: > > Text I/O > > ... operate on a per-character basis instead of a per-byte basis. > "per-character" needs some clarification. I'm guessing this will only > return entire code points, but the u

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Jim Jewett
On 2/27/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 2/27/07, Paul Moore <[EMAIL PROTECTED]> wrote: > It *may* even be useful if many of these would support non-blocking > I/O; we're currently considering adding a standard API for returning > "EWOULDBLOCK" errors (e.g. return None from rea

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Paul Moore
On 27/02/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 2/27/07, Paul Moore <[EMAIL PROTECTED]> wrote: > [...] > > Documenting the revised open() factory in this PEP would be useful. It > > needs to address encoding issues, so it's not a simple copy of the > > existing open(). > > Check the d

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Guido van Rossum
On 2/27/07, Paul Moore <[EMAIL PROTECTED]> wrote: [...] > Documenting the revised open() factory in this PEP would be useful. It > needs to address encoding issues, so it's not a simple copy of the > existing open(). Check the doc again. I added on at the end. It could use some review. I also adde

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Steven Bethard
On 2/27/07, Paul Moore <[EMAIL PROTECTED]> wrote: > > .__init__(self, buffer, encoding=None, newline=None) > > > >Same as above but if newline is not None use that as the > > newline pattern (for reading and writing), and if newline is not set > > attempt to find the newline pattern fro

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Adam Olsen
On 2/26/07, Mike Verdone <[EMAIL PROTECTED]> wrote: > Text I/O > The text I/O layer provides functions to read and write strings from > streams. Some new features include universal newlines and character > set encoding and decoding. The Text I/O layer is defined by a > TextIOBase abstract base cla

Re: [Python-3000] Draft PEP for New IO system

2007-02-27 Thread Paul Moore
On 26/02/07, Mike Verdone <[EMAIL PROTECTED]> wrote: > Daniel Stutzbach and I have prepared a draft PEP for the new IO system > for Python 3000. This document is, hopefully, true to the info that > Guido wrote on the whiteboards here at PyCon. This is still a draft > and there's quite a few decisio