We have an area of our product that can, depending on certain conditions, 
produce lots of rapid transactions all at once. We’re getting many reports of 
POSKeyErrors at sites where the transaction volume is higher than others. They 
appear to coincide (in many cases, at least) with zodbpack runs.

After some investigation, it appears that it’s a timing issue. If an object is 
unreferenced during the pre_pack() stage, it will be marked for GC. If it then 
becomes referenced before the actual DELETE FROM is executed, it will be 
deleted nonetheless and POSKeyErrors will result.

Now, I don’t know how the object is unreferenced and referenced in separate 
transactions; as far as I can tell, there are no two-transaction moves or 
anything like that happening. Nonetheless, it’s entirely possible we can solve 
this by tracking down the code that sets up the race condition.

But is there any lower-level way around the race condition? A good amount of 
our application is pluggable by the end user, and I’d like to avoid vague 
technical warnings about transaction boundaries if possible :) If we could 
verify that no POSKeyErrors will result from the DELETE before running it, that 
would be much simpler.

I should also mention that running zodbpack with days=1 isn’t an 
across-the-board solution for us. We have some customers for whom rapid growth 
of the database is an issue, and they pack more frequently.


For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to