On 04/08/2013 09:59 AM, Sebastian Berg wrote:
On Mon, 2013-04-08 at 08:42 +0200, Dag Sverre Seljebotn wrote:
On 04/06/2013 04:19 PM, Nathaniel Smith wrote:
Hi all,
If you build current numpy master with
NPY_RELAXED_STRIDES_CHECKING=1 python setup.py install
then Cython code using ndarrays starts blowing up, e.g.:
# foo.pyx
def add_one(array):
cdef double[::1] a = array
a[0] += 1.
return array
foo.add_one(np.ascontiguousarray(np.arange(10.)[::100]))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foo.pyx", line 2, in foo.add_one (foo.c:1210)
cdef double[::1] a = array
ValueError: Buffer and memoryview are not contiguous in the same dimension.
The problem (as discussed before) is that Cython has an unnecessarily
strict definition of "contiguous", so NPY_RELAXED_STRIDES_CHECKING=1
pretty much breaks all existing compiled Cython modules.
Our plan is to make NPY_RELAXED_STRIDES_CHECKING=1 into the default
sooner or later, and Cython is a major blocker on this plan. It may
become the default as soon as the 1.8 pre-releases (with the
expectation that we'll probably have to switch back again before the
actual release, but still).
References:
Previous thread:
http://thread.gmane.org/gmane.comp.python.cython.devel/14634
Detailed discussion of the difference between numpy/cython's current
definition of "contiguity", and the correct definition:
http://thread.gmane.org/gmane.comp.python.cython.devel/14634/focus=14640
The PR implementing NPY_RELAXED_STRIDES_CHECKING:
https://github.com/numpy/numpy/pull/3162
Another test case:
https://github.com/numpy/numpy/issues/2956
We're hoping that Cython will also switch soon to the more accurate
check for contiguity. This shouldn't cause any backwards compatibility
problems -- it just means Cython code would make strictly fewer
copies, and error out due to lack of contiguity strictly less often,
even with older numpys. And it seems like a necessary step for getting
this untangled and minimizing user pain. What do you think?
I agree that we should follow NumPy here, but can't see myself having
time to do the change in near future. I don't know about Mark?
I think it is a fairly simple change though if any NumPyers would like
to do it, look at in Cython/Utility/MemoryView_C.c in the function
_pyx_memviewslice_is_contig
looks like it should just be to add a check for shape too.
I guess you have changed your implementation of PEP 3118 too slightly on
the NumPy side? Is this undefined in the PEP or are you now not in
strict adherence to it?
Hi,
maybe I will have a look at that, but not sure if I will manage. The
PEP 3118 is a point, but it does not seem to cover this (probably we
should try to get a clarification into 3118). I am still wondering
whether for buffers that are requested contiguous numpy should set the
strides again, since it cannot hurt. Would that make a difference for
Cython? I expected Cython just got any buffer and then checked the
strides instead of requesting the buffer contiguous and then double
checking.
At least when I implemented
cdef np.ndarray[double, mode='fortran'] arr
that relies on PEP 3118 contiguous-flags and I did no checking myself.
Lots of Cython code does this instead of memoryviews (I still write my
own code that way).
The memory views OTOH does their own checking, but I also see plenty of
references to PyBUF_C_CONTIGUOUS etc. inside
Cython/Utility/MemoryView.pyx, so perhaps it does both. Mark would have
the definitive answer here.
Dag Sverre
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
