On Sun, Oct 25, 2009 at 11:33 AM, Stefan Behnel <[email protected]> wrote: > Hi all, > > the wiki has an enhancement page on better C array support. > > http://wiki.cython.org/enhancements/arraytypes > > Note that this is not about SIMD operations and whatnot, just about plain > > int[] x > > The page mentions things like dynamic memory allocation happening > automatically behind the scenes, so that > > cdef int a[runtime_size] > > would locally allocate memory, as would subsequent slicing, I guess. I > wonder if that's not too much magic, and if a much simpler thing wouldn't > get us far enough already. >
Stefan, I think you are missing a point here... Some of as do need automatic allocation of plain C arrays, because we want them for passing to a C function for fill-in data, or because we want to use it as temporary for intermediate computation. > > Since we already agreed to get a full-featured SIMD type, what do you think > about dropping the dynamic memory handling part for plain C arrays, > Well, I still think that automatic memory allocation for C arrays would be a REALLY nice feature... > and > instead just supporting slicing operations on any C pointer type, and > letting them return a Python list? (or a byte string in the special case of > a char*) > I'm definitely +1 about all this. But I insist: Memory allocation and fancy slicing operations for coercion to PyObject are IMO to different beasts... So you could support the "lhs = rhs[slice]" when lhs is a pyobject, but generate a Cython compile error if lhs is a C pointer/array type (perhaps making exceptions for the case of char[]/char*) until auto mem alloc is implemented. Am I missing something here? > This would be a rather straight forward thing to support: > > cdef char* s = ... > py_s = s[:30] > > It would translate directly to a call to PyString_FromStringAndSize(), > which is a rather often requested feature IMO. All other base types would > return a runtime-constructed list instead: > > cdef int* x = ... > py_int_list = x[4:20:-1] > > This would generate a loop that creates Python int values from the slice > and put them into a list, so it's basically a short form for this: > > py_int_list = [ int(x[i]) for i from 20 >= i > 4 by -1 ] > > I suggest returning a list and not a tuple here, as a list can be modified > later on, which is a likely thing to happen. Having to manually copy a > tuple into a new list just to be able to modify it, is a rather inefficient > requirement. > > Using this in a loop context would also be easy to optimise, i.e. > > cdef int* x = ... > cdef Py_ssize_t length = ... > cdef int i > > for i in x[:length]: > ... > > would be implemented as a C for loop from 0 to length-1, and pointer index > access to get a value for i in each step (or maybe a running pointer > variable, not sure what's better in C code). > > There are a couple of requirements for this. One is that this only works > for Python compatible base types, so e.g. arrays of pointers wouldn't be > supported (unless we coerce them to PyCObject/PyCapsule, but I doubt that > there's a use case for that). Another bit is that we might (not sure yet) > need to know the step size at compile time. I'm also not sure yet how this > fits with the future SIMD type. > > Anyway, what do you think about this? > > Stefan > _______________________________________________ > Cython-dev mailing list > [email protected] > http://codespeak.net/mailman/listinfo/cython-dev > -- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
