Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)
Travis Oliphant wrote:
Carl Banks wrote:
My recommendation is, any flag should turn on some circle in the Venn
diagram (it could be a circle I didn't draw--shaped arrays, for
example--but it should be *some* circle).
I don't think your Venn diagram is broad enough and it un-necessarily
limits the use of flags to communicate between consumer and exporter.
We don't have to ram these flags down that point-of-view for them to be
productive.If you have a specific alternative proposal, or specific
criticisms, then I'm very willing to hear them.
Ok, I've thought quite a bit about this, and I have an idea that I think
will be ok with you, and I'll be able to drop my main objection. It's
not a big change, either. The key is to explicitly say whether the flag
allows or requires. But I made a few other changes as well.
First of all, let me define how I'm using the word contiguous: it's a
single buffer with no gaps. So, if you were to do this:
memset(bufinfo-buf,0,bufinfo-len), you would not touch any data that
isn't being exported.
Without further ado, here is my proposal:
--
With no flags, the PyObject_GetBuffer will raise an exception if the
buffer is not direct, contiguous, and one-dimensional. Here are the
flags and how they affect that:
Py_BUF_REQUIRE_WRITABLE - Raise exception if the buffer isn't writable.
Py_BUF_REQUIRE_READONLY - Raise excpetion if the buffer is writable.
Py_BUF_ALLOW_NONCONTIGUOUS - Allow noncontiguous buffers. (This turns
on shape and strides.)
Py_BUF_ALLOW_MULTIDIMENSIONAL - Allow multidimensional buffers. (Also
turns on shape and strides.)
(Neither of the above two flags implies the other.)
Py_BUF_ALLOW_INDIRECT - Allow indirect buffers. Implies
Py_BUF_ALLOW_NONCONTIGUOUS and Py_BUF_ALLOW_MULTIDIMENSIONAL. (Turns on
shape, strides, and suboffsets.)
Py_BUF_REQUIRE_CONTIGUOUS_C_ARRAY or Py_BUF_REQUIRE_ROW_MAJOR - Raise an
exception if the array isn't a contiguous array with in C (row-major)
format.
Py_BUF_REQUIRE_CONTIGUOUS_FORTRAN_ARRAY or Py_BUF_REQUIRE_COLUMN_MAJOR -
Raise an exception if the array isn't a contiguous array with in Fortran
(column-major) format.
Py_BUF_ALLOW_NONCONTIGUOUS, Py_BUF_REQUIRE_CONTIGUOUS_C_ARRAY, and
Py_BUF_REQUIRE_CONTIGUOUS_FORTRAN_ARRAY all conflict with each other,
and an exception should be raised if more than one are set.
(I would go with ROW_MAJOR and COLUMN_MAJOR: even though the terms only
make sense for 2D arrays, I believe the terms are commonly generalized
to other dimensions.)
Possible pseudo-flags:
Py_BUF_SIMPLE = 0;
Py_BUF_ALLOW_STRIDED = Py_BUF_ALLOW_NONCONTIGUOUS
| Py_BUF_ALLOW_MULTIDIMENSIONAL;
--
Now, for each flag, there should be an associated function to test the
condition, given a bufferinfo struct. (Though I suppose they don't
necessarily have to map one-to-one, I'll do that here.)
int PyBufferInfo_IsReadonly(struct bufferinfo*);
int PyBufferInfo_IsWritable(struct bufferinfo*);
int PyBufferInfo_IsContiguous(struct bufferinfo*);
int PyBufferInfo_IsMultidimensional(struct bufferinfo*);
int PyBufferInfo_IsIndirect(struct bufferinfo*);
int PyBufferInfo_IsRowMajor(struct bufferinfo*);
int PyBufferInfo_IsColumnMajor(struct bufferinfo*);
The function PyObject_GetBuffer then has a pretty obvious
implementation. Here is an except:
if ((flags Py_BUF_REQUIRE_READONLY)
!PyBufferInfo_IsReadonly(bufinfo)) {
PyExc_SetString(PyErr_BufferError,buffer not read-only);
return 0;
}
Pretty straightforward, no?
Now, here is a key point: for these functions to work (indeed, for
PyObject_GetBuffer to work at all), you need enough information in
bufinfo to figure it out. The bufferinfo struct should be
self-contained; you should not need to know what flags were passed to
PyObject_GetBuffer in order to know exactly what data you're looking at.
Therefore, format must always be supplied by getbuffer. You cannot tell
if an array is contiguous without the format string. (But see below.)
And even if the consumer isn't asking for a contiguous buffer, it has to
know the item size so it knows what data not to step on.
(This is true even in your own proposal, BTW. If a consumer asks for a
non-strided array in your proposal, PyObject_GetBuffer would have to
know the item size to determine if the array is contiguous.)
--
FAQ:
Q. Why ALLOW_NONCONTIGUOUS and ALLOW_MULTIDIMENSIONAL instead of
ALLOW_STRIDED and ALLOW_SHAPED?
A. It's more useful to the consumer that way. With ALLOW_STRIDED and
ALLOW_SHAPED, there's no way for a consumer to request a general
one-dimensional array (it can only request a non-strided one-dimensional
array), and requesting a SHAPED array but not a STRIDED one can only
return a C-like (row-major) array, although a consumer might reasonably
want a Fortran-like (column-major) array. This approach maps more
directly to the consumer's needs, is more flexible, and still
Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)
Carl Banks wrote: Py_BUF_REQUIRE_READONLY - Raise excpetion if the buffer is writable. Is there a use case for this? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Changes to decimal.py
Tim Peters wrote: can wait a couple months, I'd be happy to own it. A possible saving grace for ln() is that while the mathematical function is one-to-one, I'm working right now in making the old operation to pass the new tests ok. Soon I'll cut a branch to work publicly on this (good idea from Raymond), and I'll be pleased to get your help here. Two months is ok: the updated decimal won't be finished before that. Thank you! -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new subscriber looking for grunt work
Martin v. Löwis wrote: an activity that is always worthwhile is bug and patch review. Pick a patch or a bug report that hasn't seen any feedback (there are, unfortunately, plenty of them), and try to analyse it. Sergio, welcome. As Martin said, bugs and patch revision is a fruitful activity, for Python, and for you. You can use these pages to have a grand vision of which bugs and patchs we have still open and start choosing your first: http://www.taniquetil.com.ar/facundo/py_bugs_00.html http://www.taniquetil.com.ar/facundo/py_patchs_00.html Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_normalization failures across communitybuildbots
On 4/16/07, Raymond Hettinger [EMAIL PROTECTED] wrote: [Collin Winter] This should be fixed in r54844. The problem was that the availability of the urlfetch resource wasn't being checked early enough and so test_support.run_suite() was converting the ResourceDenied exception into a TestError instance. This wasn't showing up on other machines since the urlfetch'd files weren't being cleaned out between test runs. Can you add code to clean them out between runs? I'll see what I can do. Collin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] distutils mixes 32/64bit binaries
On the platforms that can run both 32 and 64bit python, such as
x86_64, distutils builds both 32 and 64 bit libraries in the same
directory such as build/lib.linux-x86_64-2.5.
This can be easily fixed by placing 64 bit libraries in
build/lib64.linux-x86_64-2.5 instead.
On the other hand it may be time to revisit use of os.uname in
distutils' get_platform to determine the target directory. In the
environments that can run python interpreters compiled for different
architectures, distutils' get_platform should return the platform for
which the interpreter was built, not the platform on which it runs.
Similar problem exists with the default install directory
$prefix/lib/python2.5/lib-dynload. 64-bit libraries should probably go
to $prefix/lib/python2.5/lib64-dynload instead.
I've noticed that there was a similar proposal to use $prefix/lib64
for 64bit binary components, but it was quickly rejected:
http://mail.python.org/pipermail/python-dev/2006-November/070044.html
In my view, following LSB spirit would suggest splitting pythonX.Y
tree into platform dependent part (*.so) that would go under
$prefix/lib or $prefix/lib64 and a platform independent part
(*.py{,c,o}) that would go under $prefix/share. This, of course, would
be a much bigger change than the lib64-dynload band aid that I am
proposing here.
PS: As a data point, the R project has made the following changes to
their program to deal with this issue:
o LDFLAGS now defaults to -L/usr/local/lib64 on most Linux
64-bit OSes (but not ia64). The use of lib/lib64 can be
overridden by the new variable LIBnn.
o The default installation directory is now ${prefix}/${LIBnn}/R,
/usr/local/lib64/R on most 64-bit Linux OSes and /usr/local/lib/R
elsewhere.
https://stat.ethz.ch/pipermail/r-announce/2005/000802.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] minidom - new-style classes?
I'm working on minidom's DOM Level 1 compliance, targeting Python 2.6. We have some bugs involving DOM property behavior. For example, setting the nodeValue attribute of an Element is supposed to have no effect. We don't implement this. The right way to implement these quirks is using new-style classes and properties. Right now minidom uses old-style classes and lots of hackery, and it's pretty broken. (Another example--there is an Attr._set_prefix method, but it is *not* called from __setattr__.) Surely nobody is subclassing these classes. You don't subclass DOM interfaces--the DOM doesn't work that way. So this change should be OK. Right? -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom - new-style classes?
On 4/17/07, Jason Orendorff [EMAIL PROTECTED] wrote: Surely nobody is subclassing these classes. You don't subclass DOM interfaces--the DOM doesn't work that way. So this change should be OK. Right? People are definitely subclassing those classes: http://www.google.com/codesearch?hl=enlr=q=class%5Cs%2B.%2B%5C%28.*minidom%5C.btnG=Search Whether any of those uses will break if switched to new-style class, I can't say. Collin Winter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom - new-style classes?
On Tuesday 17 April 2007 22:37, Jason Orendorff wrote: The right way to implement these quirks is using new-style classes and properties. Right now minidom uses old-style classes and lots of hackery, and it's pretty broken. (Another example--there is an Attr._set_prefix method, but it is *not* called from __setattr__.) Yes, it's truly vile. Better than it used to be, but There's also some vague attempt at supporting the Python CORBA IDL mapping, since the W3C DOM specifications use that. I expect the support is incomplete at best. Surely nobody is subclassing these classes. You don't subclass DOM interfaces--the DOM doesn't work that way. So this change should be OK. Right? There are people who've tried building new DOM implementations by subclassing the minidom classes to get most of the behavior. I'm don't know the status of any of these implementations, but changes to these classes have proved difficult due to this and the possibility of breaking pickles (amazed me, that one did!). I'd love to see a sane implementation, using new-style classes and properties, but as long as we've got to support existing applications, we're pretty well hosed as far as xml.dom.minidom is concerned. A new DOM implementation conforming to the W3C recommendations would be nice, but I'd really rather see an XML object model that doesn't suck, but that supports as much of the information found in the W3C DOM as possible. Something based more on the ElementTree API, perhaps. The value of the W3C-approved API has certainly turned out to be more decoy than anything. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom - new-style classes?
Perhaps a rewrite could target 3.0 and 2.6 could use a backported version of this *if* py3k compatibility mode is enabled? I'd love to see at least the 3.0 version cleaned up. --Guido On 4/17/07, Fred L. Drake, Jr. [EMAIL PROTECTED] wrote: On Tuesday 17 April 2007 22:37, Jason Orendorff wrote: The right way to implement these quirks is using new-style classes and properties. Right now minidom uses old-style classes and lots of hackery, and it's pretty broken. (Another example--there is an Attr._set_prefix method, but it is *not* called from __setattr__.) Yes, it's truly vile. Better than it used to be, but There's also some vague attempt at supporting the Python CORBA IDL mapping, since the W3C DOM specifications use that. I expect the support is incomplete at best. Surely nobody is subclassing these classes. You don't subclass DOM interfaces--the DOM doesn't work that way. So this change should be OK. Right? There are people who've tried building new DOM implementations by subclassing the minidom classes to get most of the behavior. I'm don't know the status of any of these implementations, but changes to these classes have proved difficult due to this and the possibility of breaking pickles (amazed me, that one did!). I'd love to see a sane implementation, using new-style classes and properties, but as long as we've got to support existing applications, we're pretty well hosed as far as xml.dom.minidom is concerned. A new DOM implementation conforming to the W3C recommendations would be nice, but I'd really rather see an XML object model that doesn't suck, but that supports as much of the information found in the W3C DOM as possible. Something based more on the ElementTree API, perhaps. The value of the W3C-approved API has certainly turned out to be more decoy than anything. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
