Hi everyone, as you might have seen from my checkins on Friday, I released mongopersist. In short, it implements a persistence backend in MongoDB (that, with a little bit of effort, could be used as a ZODB replacement in Zope):
http://pypi.python.org/pypi/mongopersist Some background and motivation: Many years ago, Jim told me that the persistence code could be used to implement any storage backend. But until now I never had the need to confirm that statement. The ZODB and its various storages served me just fine. But then I decided to use MongoDB as storage for my latest project. (I just recently used storm with PostGreSQL for a project and was once again annoyed about the impedance mismatch between the stiff relational and flexible object worlds.) It was out of question that I just wanted to use pymongo straight up, since it makes admin interfaces just very painful to write. I first tried p01.mongo, but unfortunately it tries to re-implement persistence with some of the most annoying ORM-style features (such as explicit attribute declarations). So, I came to the conclusion I wanted to build an abstraction layer myself and see how well it would work. An ORM-style solution was out of question. I consider attribute-type declarations ORM-style, since it prohibits me to extend objects in arbitrary ways (think annotations, sub-classing, etc). I then looked into implementing a ZODB storage. Unfortunately, the ZODB really wants to deal with pickles and I wanted to store objects in a way that the data could be used without Python. Thus, implementing a storage mechanism on top persistence seemed like the best choice. It was not too hard, since ZODB's storage implementations provided good examples. Here are some of the main features: * Full persistent data manager implementation making storing objects seamless. * Data manager can be used to load and dump objects directly. * Automatic and explicit specification of the mongo collection to use. * Store multiple object types in the same collection. This is particularly useful, if you have sub-classes of an object type. * Storage of non-persistent objects. Pretty much everything that can be pickled can be stored. There is some work left to do to support all pickle hooks and object constructors. * Plugins to provide custom serializers and de-serializers for objects. This is useful for objects whose reduced state is not suitable for Mongo storage, such as datetime.date. * Control over document granularity. In the ZODB, any persistent object causes a separate pickle/document. In mongopersist you can choose whether a persistent object is stored as part of its parent object or not. (The Mongo guys make a big deal about choosing document granularity.) * Cross database support, i.e. full DBRef object references are used everywhere. * Optional write-conflict detection. * Simple Mapping implementation that represents a collection. * Optional Zope Support: Multiple container implementations for various use cases. The main container supports one collection for multiple instances by keeping a parent reference. The container itself can be connected to a ZODB or mongopersist. The container also provides full access to the MongoDB query API, filtering by parent automatically and supporting raw or object results. Before you jump on this right away, here are some features that are missing: * BTreeContainer instances will probably store, but they would not provide sensible output in Mongo. Custom hooks need to be written to handle BTreeContainers sensibly. * Since the pickle module does not cleanly separate reducing an object and creating a pickle string, mongopersist must effectively re-implement the pickle API. I have implemented the most common use cases, but some pickle hooks and handling of classes with __new__ constructors are still missing. * Cyclic references of non-persistent objects is not supported. That said, we are about to head into production with mongopersist. If you need any help or want those above two problems addressed, feel free to contact me. So was Jim right? Absolutely! Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter" _______________________________________________ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
