On Jan 7, 2010, at 8:59 AM, Matthew wrote: > Well I guess I outsmarted myself on this one. After implementing my > object hasher using dict, > my code slowed down by nearly a factor of 10 LOL. > > The hashing code itself shows up as a minimal contribution to the > overall time in the profiler. > However it does require a little extra logic in the initializer and > a copy of a few pointers. > > Basically I copy all the C device pointers to a new cython class > which then get's saved in a dictionary upon the deallocation > of a matrix object. If that object exists in the dictionary it is > given to the allocator instead of allocating a new block. > > I'm wondering just what kind of overhead Im up against with regards > to the python initialization of my cython defined class? > This almost suggests that the overhead problem is actually in python/ > cython and not just in my C code.
This could probably be discovered via some profiling. If this is the case, take a look at http://trac.cython.org/cython_trac/ticket/238 (the modern version of the former PY_NEW trick). Maybe you're passing lots of (keyword?) arguments? > > -Matt > > Robert Bradshaw <rober...@...> writes: > >> >> On Jan 7, 2010, at 12:05 AM, Stefan Behnel wrote: >> >>> Matthew Bromberg, 06.01.2010 21:50: >>>> How does tuple or list compare speed wise with dict? >>> >>> Like apples and oranges, basically. >> >> If you're trying to index into it with an int, especially a c int, >> lists and tuples will be much faster. >> >>>> Ultimately I have to hash into my list using size information. >>> >>> Any specific reason why you /can't/ use a dict? >> >> Which will likely be just as fast, if not faster, than hashing into a >> list manually yourself. >> >>>> This also still does not address my confusion with regards to how >>>> to >>>> capture a python object before it get's destroyed. >>> >>> As long as there is a reference to it (e.g. in the hash table), it >>> won't >>> get deallocated. So: use a Python list for your hash table, stop >>> caring >>> about ref-counts and it will just work. >> >> +1. Ideally, you should never have to worry about reference counts >> when working with Cython at all. >> >> I'm still not quite sure exactly what you're trying to do, but if >> it's >> creating and deleting thousands of these objects a second and that's >> killing you (the actual allocation/deallocation, not the >> initialization) then what you might want to do is something like >> >> http://hg.sagemath.org/sage-main/file/21efb0b3fc47/sage/rings/real_double.pyx#l2260 >> >> which is a bit hackish and will probably need to be adapted to your >> specific case. If initializing is expensive, than you can probably >> keep around a pool of initalized pointers/buffers/whatever, and have >> the object creation just set/unset these fields (much cleaner). >> >> - Robert >> > _______________________________________________ > Cython-dev mailing list > [email protected] > http://codespeak.net/mailman/listinfo/cython-dev _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
