On Wed, Apr 28, 2010 at 11:43 AM, Hanno Schlichting <ha...@hannosch.eu> wrote: > On Wed, Apr 28, 2010 at 5:11 PM, Jim Fulton <j...@zope.com> wrote: >> Do you know of specific benefits you expect from protocol 2? Any >> specific reasons >> you think it would be better in practice? > > I have just seen some ongoing work on pickles in recent times, for > example from the Python 2.7 what's new: > > - The pickle and cPickle modules now automatically intern the strings > used for attribute names, reducing memory usage of the objects > resulting from unpickling. (Contributed by Jake McGuire; issue 5084.)
I can't see why this should be protocol specific. > - The cPickle module now special-cases dictionaries, nearly halving > the time required to pickle them. (Contributed by Collin Winter; issue > 5670.) That's odd. cPickle lready special-cased dictionaries. > Unless I've misread the code, these changes only apply to protocol > two. We should double check that. I'll take a closer look. > And then there's the old claims of pep 307 stating that pickling > new-style classes would be more efficient. Which doesn't apply to persistent object since they're handled differently. > Finally Python 3 introduces pickle protocol version 3, which deals > explicitly with the new bytes type. There's more changes in Python 3 > and the pickle format, so that's a separate project. But it suggested > to me, that the pickle format isn't quite as "dead" anymore as it used > to be. I really think Python 3 is a separate topic. It's likely that there will be many things to confront. :( > >> I've avoided going to protocol 2 for two reasons: >> >> - It wasn't clear we'd get a benefit without deeper changes. >> Those deeper changed might be of value, but only if we're >> careful about how we make them. >> >> In particular, we could replace class names in pickles >> if we has a registry mapping ints to class names. >> This could provide a number of benefits beyond >> smaller pickles, but it needs some thought to get right. > > Right. I'm not particular interested in the pickle class registry. > Having a hard dependency between code filling the registry and the > actual data has all sorts of implications. I don't really want to go > there myself. It has some positive implications if you get it right: - Smaller pickles - Easier class renaming - Potentially greater security Getting it right is almost certainly a bigger project than anyone wants to deal with right now. ... >> I suggest doing some realistic experiments to look at the impact of the >> change. >> >> - Convert an interesting Zope 2 database from protocol 1 to protocol 2. >> How does this affect database size? >> >> - Do some sort of write and read benchmarks using the 2 protocols to >> see if there's a meaningful benefit. > > Ok, thanks. That gives me enough direction to work on some specific > benchmarks. Cool. BTW, you might want to (try :) to search the list archives. I think someone did some experiments a while back. Jim -- Jim Fulton _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
