On Oct 25, 2009, at 7:33 AM, Stefan Behnel wrote: > Hi all, > > the wiki has an enhancement page on better C array support. > > http://wiki.cython.org/enhancements/arraytypes > > Note that this is not about SIMD operations and whatnot, just about > plain > > int[] x > > The page mentions things like dynamic memory allocation happening > automatically behind the scenes, so that > > cdef int a[runtime_size] > > would locally allocate memory, as would subsequent slicing, I guess. I > wonder if that's not too much magic, and if a much simpler thing > wouldn't > get us far enough already.
I think dynamic memory allocation is the primary advantage of such a type, though the proposed SIMD type would meet these needs as well. > Since we already agreed to get a full-featured SIMD type, what do > you think > about dropping the dynamic memory handling part for plain C arrays, > and > instead just supporting slicing operations on any C pointer type, and > letting them return a Python list? (or a byte string in the special > case of > a char*) > > This would be a rather straight forward thing to support: > > cdef char* s = ... > py_s = s[:30] > > It would translate directly to a call to PyString_FromStringAndSize(), > which is a rather often requested feature IMO. All other base types > would > return a runtime-constructed list instead: > > cdef int* x = ... > py_int_list = x[4:20:-1] > > This would generate a loop that creates Python int values from the > slice > and put them into a list, so it's basically a short form for this: > > py_int_list = [ int(x[i]) for i from 20 >= i > 4 by -1 ] > > I suggest returning a list and not a tuple here, as a list can be > modified > later on, which is a likely thing to happen. Having to manually copy a > tuple into a new list just to be able to modify it, is a rather > inefficient > requirement. While turning them into a list may be useful, we should consider if this will be backwards incompatible with conversion to a SIMD/memory type. What could be more safely (and explicitly) done is cdef int* x py_int_list = list(x[4:20:-1]) or py_int_list = tuple(x[4:20:-1]) with conversion to object still left for future debate. > Using this in a loop context would also be easy to optimise, i.e. > > cdef int* x = ... > cdef Py_ssize_t length = ... > cdef int i > > for i in x[:length]: > ... > > would be implemented as a C for loop from 0 to length-1, and pointer > index > access to get a value for i in each step (or maybe a running pointer > variable, not sure what's better in C code). I see no reason why we couldn't already optimize such a loop, even for non-python compatible types i. I'd like to note that stack-allocated memory is a very non-pythonic thing, and I've seen a lot of Cython users tripped up by it, so I'm not sure how much we should encourage it/make it easy (though optimizing it when we know it's safe could be valuable to do). - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
