Matthew Brett wrote:
> I am running code that creates a dictionary, fills it with lists of
> tuples created from C integers, and returns the dictionary, like this:
>
> def afunc():
> cdef int i=1
> cdef int j=2
> cdef int k=3
> cdef int m=4
> cdef int n=5
>
> d = {}
> val1 = (i, j)
> val2 = (k, m)
> d[m] = [val1, val2]
> return d
>
>
> The code above does indeed return the dictionary I was expecting:
>
> {5: [(1, 2), (3, 4)]}
>
> however, in a more complex routine, I am doing a similar thing, but
> returning many thousands of lists of tuples, and the routine grinds to
> a halt.
Did you check that your machine has enough memory? Depending on how "many
thousands" we are talking about, it may start swapping when its memory
fills up. Also, make sure you are not creating reference cycles somewhere
in your structure that may require cyclic garbage collection.
> Looking at the generated C code, I wonder if I am running into a
> reference counting problem
That's rather unlikely (although you didn't state which version of Cython
you are using).
> because the tuple I'm creating seems to be decref'ed.
Don't worry, that's normal. When Cython creates them, they get stored in
temporary variables that hold the reference. Adding them to the dict will
let the dict add another reference, and when the temporary reference is
taken out of scope, Cython must DECREF it.
> I've tried adding Py_INCREFs to the list, tuple, and python copies of
> the integers, but this made no difference to the larger routine.
Please don't do that. That will cause your code to leak memory (which in
turn will lead to higher memory consumption and more likely swapping).
> I realize I'm not understanding on-the-fly python object creation /
> destruction in Cython, and was wondering where I should go to look to
> understand it better.
It works more or less as in CPython, just with a sequence of C calls
instead of a lengthy interpreter eval loop.
Stefan
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
