Alan Runyan asked me to post my notes from my attempt to get ZODB running on PyPy. It's very much an experimental work in progress (that I got distracted from), but hopefully this is at least useful to anyone else who wants to attempt the same thing.
I first tried building ZODB trunk (with C extensions). Ran into the following issues: * PyPy is missing an implementation of the _Py_ForgetReference macro which is used by ZODB's persistent/cPickleCache.c * The check_argument_cmp check in BTrees/objectkeymacros.h fails for the root object of the database on PyPy * I got a PyPy exception re wrap_objobjargspec which appeared to be some issue with using the __setitem__ slot of an extension type At this point I gave up on trying to build the C extensions, and instead did the following: 1. Started with Jim's python-btrees branch (http://svn.zope.org/repos/main/ZODB/branches/jim-python-btrees) 2. Replaced the 'persistent' package with a checkout of Tres' pure-Python persistent implementation (http://svn.zope.org/repos/main/persistent/trunk/persistent) 3. Set exts = [] in setup.py to disable all C extensions when installing. Next I worked around some issues in the pure-Python branches: * BTrees.___BTree.fsBucket class was missing toString and fromString methods, which I implemented. * persistent.picklecache.PickleCache class was missing update_object_size_estimation method (and indeed the whole limit-cache-size-by-bytes feature). I added it as a no-op stub. * persistent.picklecache.PickleCache.__setitem__ was raising a KeyError for a duplicate oid even when trying to set the same object already stored under that key * persistent.TimeStamp did not give the Python implementation, which I fixed by an import of persistent.timestamp + module alias if persistent.TimeStamp (the C module) is missing * persistent.pyPersistence.Persistent's __new__ needs to accept *args and **kw At this point I was able to start up the Pyramid ZODB scaffold and add an object to the DB root. Next I tried to get zodbshootout running as a benchmark. I didn't get this working, as I got stuck on an issue with the pure-Python PersistentMapping getting committed without its object state (appeared to be something to do with failing to get unghosted before it gets committed). Along the way, I noticed a couple incompatibilities with PyPy because PyPy implements the cPickle module as an alias to the pure-Python pickle module. This leads to some problems due to slight semantic differences between cPickle and pickle. In particular, cPickle.Pickler accepts a lone protocol argument; pickle.Pickler does not (leading to an error in ZEO.zrpc.marshal.encode). And pickle.Unpickler does not have a noload method, which ZODB.serialize.referencesf uses to obtain the referenced oids from a pickle without the overhead of unpickling. That's as far as I got. Hope this is helpful to the next guy. :) cheers, David ---------- David Glick Web Developer davidgl...@groundwire.org 206.286.1235x32 Online tools and strategies for the environmental movement. Sign up for our newsletter: http://www.groundwire.org/email-capture _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev