-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 09/24/2011 08:02 PM, David Glick wrote: > Alan Runyan asked me to post my notes from my attempt to get ZODB > running on PyPy. It's very much an experimental work in progress > (that I got distracted from), but hopefully this is at least useful > to anyone else who wants to attempt the same thing. > > I first tried building ZODB trunk (with C extensions). Ran into the > following issues: * PyPy is missing an implementation of the > _Py_ForgetReference macro which is used by ZODB's > persistent/cPickleCache.c * The check_argument_cmp check in > BTrees/objectkeymacros.h fails for the root object of the database on > PyPy * I got a PyPy exception re wrap_objobjargspec which appeared to > be some issue with using the __setitem__ slot of an extension type > > At this point I gave up on trying to build the C extensions, and > instead did the following: 1. Started with Jim's python-btrees branch > (http://svn.zope.org/repos/main/ZODB/branches/jim-python-btrees) 2. > Replaced the 'persistent' package with a checkout of Tres' > pure-Python persistent implementation > (http://svn.zope.org/repos/main/persistent/trunk/persistent) 3. Set > exts = [] in setup.py to disable all C extensions when installing. > > Next I worked around some issues in the pure-Python branches: * > BTrees.___BTree.fsBucket class was missing toString and fromString > methods, which I implemented. * persistent.picklecache.PickleCache > class was missing update_object_size_estimation method (and indeed > the whole limit-cache-size-by-bytes feature). I added it as a no-op > stub. * persistent.picklecache.PickleCache.__setitem__ was raising a > KeyError for a duplicate oid even when trying to set the same object > already stored under that key * persistent.TimeStamp did not give the > Python implementation, which I fixed by an import of > persistent.timestamp + module alias if persistent.TimeStamp (the C > module) is missing * persistent.pyPersistence.Persistent's __new__ > needs to accept *args and **kw > > At this point I was able to start up the Pyramid ZODB scaffold and > add an object to the DB root. > > Next I tried to get zodbshootout running as a benchmark. I didn't get > this working, as I got stuck on an issue with the pure-Python > PersistentMapping getting committed without its object state > (appeared to be something to do with failing to get unghosted before > it gets committed). Along the way, I noticed a couple > incompatibilities with PyPy because PyPy implements the cPickle > module as an alias to the pure-Python pickle module. This leads to > some problems due to slight semantic differences between cPickle and > pickle. In particular, cPickle.Pickler accepts a lone protocol > argument; pickle.Pickler does not (leading to an error in > ZEO.zrpc.marshal.encode). And pickle.Unpickler does not have a noload > method, which ZODB.serialize.referencesf uses to obtain the > referenced oids from a pickle without the overhead of unpickling. > > That's as far as I got. Hope this is helpful to the next guy. :)
Cool, great work. Are the changes you needed to make available as patchers, or checked in? Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk5/RgAACgkQ+gerLs4ltQ7NmgCg2n4GT1wQWxPaRNWCguOtAb8q OQ4AoKcbDGep0ig5ZL/x+/m4hNeAVyuq =KtIZ -----END PGP SIGNATURE----- _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
