Hi all - I'm one of the unfortunates who managed to break a Data.fs when migrating a ZEO backend to new hardware. Unfortunately, I missed the 'CRITICAL' error logged by ZOE (aside: is there a "fail_on_critical" option somewhere?) and ended up with transaction ids that parse as timestamps in the year 4732 (or there abouts). This caused surprisingly few issues for our Zope/Plone install. Of course ZODB itself, didn't care, it just started using incremental tids. The biggest impact was that packing had no effect any more. (Versions are Zope 2.9.10, ZODB 3.6.3)
Long story short, I dug into the code for FileStorage, and patched copyTransactionsFrom to detect and correct future timestamps. This approach was inspired by the existing code that detects and corrects out-of-order transactions, as well as the FileStorage __init__ code that detects future timestamps. I've attached a patch. I've successfully used this in a small python script to correct my problem, reading one FileStorage and writing a new one. I thought I should send this here, for comment. Is this something that should go into mainline ZODB? Ross -- Ross Reedstrom, Ph.D. reeds...@rice.edu Systems Engineer & Admin, Research Scientist phone: 713-348-6166 The Connexions Project http://cnx.org fax: 713-348-3665 Rice University MS-375, Houston, TX 77005 GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E F888 D3AE 810E 88F0 BEDE
--- BaseStorage.py.orig 2009-05-28 12:17:31.000000000 -0500 +++ BaseStorage.py 2009-05-28 12:51:34.000000000 -0500 @@ -414,16 +414,24 @@ print ('Time stamps back in order %s' % (t)) ok=1 + t=TimeStamp(tid) + t_now = time.time() + t_now = TimeStamp(*time.gmtime(t_now)[:5] + (t_now % 60,)) + if t > t_now: + print ('Time stamp from the future, resetting: %s' % (t)) + tid = None + if verbose: print _ts self.tpc_begin(transaction, tid, transaction.status) + tid=self._tid for r in transaction: oid=r.oid if verbose: print oid_repr(oid), r.version, len(r.data) if restoring: - self.restore(oid, r.tid, r.data, r.version, + self.restore(oid, tid, r.data, r.version, r.data_txn, transaction) else: pre=preget(oid, None)
_______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev