Hi all,

We recently started testing with newer versions of ZODB3 and RelStorage, 
specifically ZODB3 3.11.0a1 together with the current release of RelStorage 
1.5.1. The updated ZODB3 release brings in the newly-independent release of 
persistent (currently 4.0.6), and together there seems to be a small 
incompatibility resulting in the following errors at commit time:

> File 
> 'python2.7/site-packages/transaction-1.4.1-py2.7.egg/transaction/_transaction.py',
>  line 394 in _commitResources
>    rm.tpc_vote(self)
> File 'python2.7/site-packages/ZODB-4.0.0a4-py2.7.egg/ZODB/Connection.py', 
> line 781 in tpc_vote
>    s = vote(transaction)
> File 'python2.7/site-packages/relstorage/storage.py', line 789 in tpc_vote
>    return self._vote()
> File 'python2.7/site-packages/relstorage/storage.py', line 823 in _vote
>    self._prepare_tid()
> File 'python2.7/site-packages/relstorage/storage.py', line 705 in _prepare_tid
>    tid_int = u64(tid)
> File 'python2.7/site-packages/ZODB-4.0.0a4-py2.7.egg/ZODB/utils.py', line 82 
> in u64
>    return unpack(">Q", v)[0]
> error: unpack requires a string argument of length 8

In `relstorage.storage.RelStorage._prepare_tid`, a 
`persistent.timestamp.TimeStamp` is used to get the `tid` value. Instead of 
using `TimeStamp.raw()`, however, RelStorage uses `repr(TimeStamp)`. 

Prior to persistent 4.0.5, `repr(TimeStamp)` was equivalent to 
`TimeStamp.raw()`, but now `repr(TimeStamp)` actually returns a 
platform-specific representation, using the appropriate escaping (commit 
e692af8281466fa309aae9273864039dcb287383 for the C version). The net result is 
that instead of `repr(TimeStamp)` returning an 8-byte string like 
b'\x01\x01\x01\x01\x01\x01\x01\x01' it returns a 25-byte string like 
b"'\\x01\\x01\\x01\\x01\\x01\\x01\\x01\\x01'" (on python2.7) and RelStorage's 
assumption is broken, leading to the above error.

I only spotted two uses of this assumption in RelStrorage, the above-mentioned 
`_prepare_tid`, plus `pack`. The following simple patch to change those places 
to use `raw` makes our own internal tests (python2.7, MySQL) pass.

RelStorage's setup.py requires ZODB3 >= 3.7.0. Based on what's at 
http://svn.zope.org/ZODB3/branches/3.7/src/persistent/, it looks like the 
`TimeStamp.raw` method has been available since at least ZODB 3.7, so it looks 
like this should be safe to use on all supported versions (but I haven't tried 
to test that).

--- storage.py  (revision 130026)
+++ storage.py  (working copy)
@@ -738,7 +738,7 @@
        now = time.time()
        stamp = TimeStamp(*(time.gmtime(now)[:5] + (now % 60,)))
        stamp = stamp.laterThan(TimeStamp(p64(last_tid)))
-        tid = repr(stamp)
+        tid = stamp.raw()

        tid_int = u64(tid)
        adapter.txncontrol.add_transaction(cursor, tid_int, user, desc, ext)
@@ -1138,8 +1138,7 @@
                if not skip_prepack:
                    # Find the latest commit before or at the pack time.
-                    pack_point = repr(
-                        TimeStamp(*time.gmtime(t)[:5] + (t % 60,)))
+                    pack_point = TimeStamp(*time.gmtime(t)[:5] + (t % 
                    tid_int = adapter.packundo.choose_pack_transaction(
                    if tid_int is None:

I hope this helps.

For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to