Hanno Schlichting wrote: > Shane Hathaway wrote: >> I have just created a patch for ZODB that makes the object serialization >> format pluggable, meaning that it should now be possible to write Python >> code that causes ZODB to store in formats other than Python pickles. > > Cool :) > >> In addition to making the data format pluggable, the patch puts most of >> ZODB's dependencies on the cPickle module in one place, so now if we >> decide to improve our usage of the cPickle module, we can make that >> change in one place rather than several. > > I'm wondering if this would be a good opportunity to include a version > marker in addition to the protocol format to the API's?
Maybe. Starting with pickle protocol version 2, pickles do start with a version header. > The ZODB still uses version one of the pickle protocol throughout and so > far it has been complicated and cumbersome to change this in any way. Is it cumbersome? I tried changing to pickle protocol v2 while making this patch. Only 4 of the 3000+ ZODB tests failed. One of them failed because we have some pickle introspection code that does not yet understand protocol 2; that should be easy to fix. Another failed because it seems to depend on the exact length of a generated pickle, which is a bit silly. I didn't look at the rest of the test failures because I assume they are all similarly superficial. > While the documentation of Protocol Buffers mention that they try to be > stable and avoid incompatible versions, I don't trust any standard to be > so generic that it can avoid incompatible changes over a period of many > years. The patch I made might cover this. After this patch, ZODB will look for the serializer format name in curly braces at the beginning of each serialized object. That format name could include a version number if necessary. BTW, I've learned that Google's protocol buffers lack the concept of a mapping, and I think that might be a major problem for my current project. We might switch to a different language agnostic format like Thrift or JSON. Shane _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
