Hi all -
I'm one of the unfortunates who managed to break a Data.fs when
migrating a ZEO backend to new hardware. Unfortunately, I missed the
'CRITICAL' error logged by ZOE (aside: is there a "fail_on_critical"
option somewhere?) and ended up with transaction ids that parse as
timestamps in the year 4732 (or there abouts). This caused surprisingly
few issues for our Zope/Plone install. Of course ZODB itself, didn't
care, it just started using incremental tids. The biggest impact was
that packing had no effect any more. (Versions are Zope 2.9.10, ZODB

Long story short, I dug into the code for FileStorage, and patched
copyTransactionsFrom to detect and correct future timestamps. This
approach was inspired by the existing code that detects and corrects
out-of-order transactions, as well as the FileStorage __init__ code 
that detects future timestamps. I've attached a patch.

I've successfully used this in a small python script to correct my
problem, reading one FileStorage and writing a new one. I thought I
should send this here, for comment. Is this something that should go
into mainline ZODB?

Ross Reedstrom, Ph.D.                                 reeds...@rice.edu
Systems Engineer & Admin, Research Scientist        phone: 713-348-6166
The Connexions Project      http://cnx.org            fax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E  F888 D3AE 810E 88F0 BEDE
--- BaseStorage.py.orig	2009-05-28 12:17:31.000000000 -0500
+++ BaseStorage.py	2009-05-28 12:51:34.000000000 -0500
@@ -414,16 +414,24 @@
                         print ('Time stamps back in order %s' % (t))
+            t=TimeStamp(tid)
+            t_now = time.time()
+            t_now = TimeStamp(*time.gmtime(t_now)[:5] + (t_now % 60,))
+            if t > t_now:
+                print ('Time stamp from the future, resetting: %s' % (t))
+                tid = None
             if verbose:
                 print _ts
             self.tpc_begin(transaction, tid, transaction.status)
+            tid=self._tid
             for r in transaction:
                 if verbose:
                     print oid_repr(oid), r.version, len(r.data)
                 if restoring:
-                    self.restore(oid, r.tid, r.data, r.version,
+                    self.restore(oid, tid, r.data, r.version,
                                  r.data_txn, transaction)
                     pre=preget(oid, None)
For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to