Re: [ZODB-Dev] historicalRevision for subobjects
Christian Heimes wrote at 2005-5-4 22:14 +0200:
>Dieter Maurer wrote:
>> It is possible (by means of a historical connection)
>> and I posted relevant code some time ago.
>> Search the archive for "HistoryJar".
>
>Sorry I can't find the code. Neither with historical jar zodb nor
>historical jar dieter googles something useful for me.
Maybe, Google does not index attachments.
Here is the code again.
--
Dieter
# $Id: History.py,v 1.8 2004/01/02 16:47:58 dieter Exp $
'''History extensions.'''
from OFS.History import HystoryJar, Historian, Historical
from struct import pack, unpack
from cStringIO import StringIO
from cPickle import Unpickler
from ZODB.Connection import Connection
from ZODB.POSException import POSKeyError
from ZODB.TimeStamp import TimeStamp
from sys import maxint
# Code mostly stolen from "OFS.History"
def historicalRevision(self,serial):
'''return the state of *self* current at *serial* and set "_p_jar"
up in a way that all subobject access has the same property.'''
jar= HistoryJar(self._p_jar,serial)
rev= jar[self._p_oid]
parent= getattr(self,'aq_parent',None)
of= getattr(rev,'__of__',None)
if parent is not None and of is not None: rev= of(parent)
return rev
class HistoryJar(HystoryJar):
'''a relaying connection selecting an object current at a serial (representing a time).'''
def __init__(self,base,serial):
HystoryJar.__init__(self,base)
self._jar= base; self._serial= serial
self._cache= {}
def __getitem__(self,oid):
'''return the state for *oid* current for *_serial*.'''
# determine which history records are available
# ATT: this information should be cached
# even better, when ZODB would support loading an object valid at serial
jar= self._jar; obj= jar[oid]
rev= obj.__class__.__basicnew__()
rev._p_jar= self
rev._p_oid= oid
self.setstate(rev)
return rev
def setstate(self,obj):
serial= self._serial; jar= self._jar; oid= obj._p_oid
S= jar._storage
filter= lambda t, serial= serial: t['serial'] <= serial # this might need to become a picklable object, once ZEO supports 'filter'.
if _hasSmartHistory(S):
ts= S.history(oid,None,1, filter= filter, )
else:
hf= HistoryFetcher(jar,oid,None,last=1, filter=filter)
ts= hf.next()
if not ts: raise POSKeyError, (oid,serial)
t= ts[0]
oserial= t['serial']
state= self.oldstate(obj,oserial)
obj._p_serial= oserial
obj.__setstate__(state)
ht= TimeStamp(unpack('8s',serial)[0]).timeTime()
try: obj._p_historical= ht
except AttributeError:
# this may not be possible, e.g. for __dict__ less instances (such as "BTrees"
pass
obj._p_changed= 0
# take over methods from "Connection", because we do not derive
# from this class, we must use "im_func".
oldstate= Connection.oldstate.im_func
_persistent_load= Connection._persistent_load.im_func
class Historian(Historian):
'''deliver historical revisions.'''
def __getitem__(self,key):
self= self.aq_parent
serial=_decodeSerial(key)
# Note: we can not use the shortcut below, because
# we would loose history information for subobjects
# which may be newer.
# if serial == self._p_serial: return self
return historicalRevision(self,serial)
class Historical(Historical):
HistoricalRevisions= Historian()
# provide information on whether or not this is a historical version
def isHistorical(self):
'''false (in fact 'None'), if *self* is not historical;
otherwise, the time for which *self* has been requested.'''
return getattr(self,'_p_historical',None)
def getHistoricalSubpath(self):
'''returns an URL subpath to reference a historical version corresponding to this time.'''
ht= self.isHistorical()
if ht is None: return ''
return '/HistoricalRevisions/' + _encodeSerial(self._p_serial)
# must override to use our "historicalRevision"
def manage_historicalComparison(self, REQUEST, keys=[]):
"Compare two selected revisions"
if not keys:
raise HistorySelectionError, (
"No historical revision was selected.")
if len(keys) > 2:
raise HistorySelectionError, (
"Only two historical revision can be compared")
rev1=historicalRevision(self, _decodeSerial(keys[-1]))
if len(keys)==2:
rev2=historicalRevision(self, _decodeSerial(keys[0]))
else:
rev2=self
return self.manage_historyCompare(rev1, rev2, REQUEST)
class HistoryFetcher:
'''auxiliary class to incrementally fetch history.'''
_curr= 0
_inc= 1
_complete= 0
def __init__(self,jar,oid,version=None,first=0,last=None,filter=None):
'''prepare fetching historical records for *oid* in *version*.
Tries to find records satisfying *filter*. Records *first*
through *last* are returned.
'''
S= jar._storage
history= S.history
if _hasSmartHistory(S): se
Re: [ZODB-Dev] historicalRevision for subobjects
Dieter Maurer wrote: It is possible (by means of a historical connection) and I posted relevant code some time ago. Search the archive for "HistoryJar". Sorry I can't find the code. Neither with historical jar zodb nor historical jar dieter googles something useful for me. Christian ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] historicalRevision for subobjects
[Christian Heimes] >>> I had the idea of getting a snapshot of the entire ZODB for a given >>> transaction serial number but I don't know how to get it. Is this >>> possible? ... [Dieter Maurer] > It is possible (by means of a historical connection) and I posted > relevant code some time ago. Search the archive for "HistoryJar". Sorry, Google gets 0 hits on HistoryJar. Maybe this is what you have in mind (on the Zope-CMF list a couple years ago)? http://mail.zope.org/pipermail/zope-cmf/2003-March/018073.html ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] historicalRevision for subobjects
Tim Peters wrote at 2005-5-2 13:37 -0400: > ... >> I had the idea of getting a snapshot of the entire ZODB for a given >> transaction serial number but I don't know how to get it. Is this >> possible? > >Sorry, don't know. It is possible (by means of a historical connection) and I posted relevant code some time ago. Search the archive for "HistoryJar". -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] historicalRevision for subobjects
[Christian Heimes] > ... > Christian Theune and me have implemented a diffed history feature for > Archetypes based content types. It's part of ATContentTypes and is meant > to help reviewers to see the differences between different revisions of > an object. The code is using the method OFS.History.historicalRevision + > db().oldstate() to iterate over the ZODB revisions of the object. > > The code was working very well because we were using AttributeStorage. AS > stores the data of Archetype fields as a simple attribute on the object. > For security and performance reasons I have implemented annotation > storage that is using Zope3 style attribute annotations to store the data > in an OOBTree attached to the object. With AnnotationStorage the history > isn't working any more because historicalRevision() returns only an older > version of the object but *not* older versions of its subobjects like the > btree. Even trying to get the different revisions of the oobtree didn't > help. The symptoms were the same. I assume I have to get the revisions of > the btree bucket or choose a different path. Yes, under the covers "a BTree" is an arbitrarily large graph (rooted DAG -- directed and acyclic) of BTree and Bucket objects. Each such object in a BTree graph has its own history (each is a distinct "first-class" persistent object in its own right). The object most app code thinks of as being "the BTree" is just the root of this arbitrarily large object graph, and the other objects in the graph generally aren't directly visible to application code. It's common as mud, e.g., that adding a new key to a BTree doesn't change its root object; then fetching an older revision of the root will still point to the current revision of the bucket to which that key was added. Indeed, BTrees are designed to make this so: if every key addition mutated the root object, any two transactions adding to a single BTree concurrently would suffer a write conflict (as does indeed happen if using, e.g., a PersistentMapping or PersistentList instead). > I had the idea of getting a snapshot of the entire ZODB for a given > transaction serial number but I don't know how to get it. Is this > possible? Sorry, don't know. I don't expect a reasonable way exists in ZODB 3.2. It may be possible to trick ZODB 3.3/3.4's MVCC machinery into delivering a consistent past view (provided it hasn't been packed away!), but if so it's not an exposed mechanism. You might want to stare at the code involving Connection._txn_time. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] historicalRevision for subobjects
Hey :) I need some guidance from ZODB gurus. Christian Theune and me have implemented a diffed history feature for Archetypes based content types. It's part of ATContentTypes and is meant to help reviewers to see the differences between different revisions of an object. The code is using the method OFS.History.historicalRevision + db().oldstate() to iterate over the ZODB revisions of the object. The code was working very well because we were using AttributeStorage. AS stores the data of Archetype fields as a simple attribute on the object. For security and performance reasons I have implemented annotation storage that is using Zope3 style attribute annotations to store the data in an OOBTree attached to the object. With AnnotationStorage the history isn't working any more because historicalRevision() returns only an older version of the object but *not* older versions of its subobjects like the btree. Even trying to get the different revisions of the oobtree didn't help. The symptoms were the same. I assume I have to get the revisions of the btree bucket or choose a different path. I had the idea of getting a snapshot of the entire ZODB for a given transaction serial number but I don't know how to get it. Is this possible? The code for the feature including some tries is in: http://svn.plone.org/collective/ATContentTypes/branches/tiran-ann-history-fix/lib/historyaware.py Christian (Tiran) ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
