On Jun 26, 2008, at 5:03 PM, Jay Parlar wrote: > On 6/26/08, Simon Burton <[EMAIL PROTECTED]> wrote: >>> Within that extension class, I'll have to populate the internal >>> structure of the list with my float values. Or is it because I >>> can use >>> the Python/C API calls to populate, instead of calling the >>> Python-visible append() that will give me the performance? >> >> >> You don't have to create/store python objects in your custom list. >> Perhaps you could even alias the storage in your queue, and have >> _zero_ copying. >> >> It really depends where your bottleneck is, and what for/how >> often you are >> using this get method. If all the work is being done at the >> python level >> then i don't think any cython trickery will help. It's up to you >> to figure it out. > > We're putting data into the queue 1000 times per second, so that needs > to be fast. And we're pulling data back out a few times per second, > but pulling out large chunks, so that too needs to be fast. This is > essentially the core stuff happening in the tight while loop of our > program.
If the producer/consumer of the data is in Python, you are better off (I believe) storing them tuples of floats in a Python list than trying to wrap/unwrap them. To see significant speedups you would probably have to to write the code that pushes/pulls data from the queue to process and produce native C doubles, and interact with the queue using cdef functions. Having a fast queue isn't going to gain you much if the inner loop is pushing/pulling via Python objects via Python functions. > And what do you mean by aliasing the storage? When I picture that, I > think of storing the Python objects in the queue, and then creating > lists with pointers back to the objects. > > I've never worked with Python at the C API level before. I've been > programming Python for over 7 years, and C for longer, but never had > to mix them, so please forgive my ignorance :) Note that Cython automatically uses the PyFloat_* function on assignment to/from a double and PyList_Append (when doing L.append) so there's little advantage to trying to use the C/API directly. - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
