Re: [ZODB-Dev] Problems with ReadConflictError
Andrew McLean wrote: Any advice gratefully received. I'd suggested moving to a ZODB version new enough to have MVCC support, which will likely make your problem go away... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Problems with ReadConflictError
Chris Withers wrote: Andrew McLean wrote: Any advice gratefully received. I'd suggested moving to a ZODB version new enough to have MVCC support, which will likely make your problem go away... Chris I'm using v3.6. Unless I'm missing something, it's the most recent stable version. Since my first post I tried another approach. Namely creating (and using) a new connection instance in the thread that raised the ReadConflictError exception. It seems to work, but I would like to understand this a bit better. class Downloader(threading.Thread): Downloader class to download urls def __init__(self, id, taskQueue, resultQueue, db): self.id = id self.__taskQueue = taskQueue self.__resultQueue = resultQueue self.__db = db self.connect() threading.Thread.__init__(self) def connect(self): self.__results = self.__db.open().root()['done'] def run(self): while 1: # Get task from the queue task = self.__taskQueue.get() if (task[1] == 1) or (not self.done(task)): # ... self.__resultQueue.put((task, result)) def done(self, task): while 1: try: return self.__results.has_key(task) except ReadConflictError: print *** ReadConflictError raised: Generate new connection to database *** self.connect() ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Problems with ReadConflictError
Andrew McLean wrote at 2006-8-21 19:44 +0100: I'm having a problem with my first multi-threaded application using ZODB (v3.6). The program basically uses an OOBTree to record whether tasks have been completed. The key is the task identifier. There are multiple Downloader threads that need to read the OOBTree (to check if a task has been completed). There is a single ProcessResults thread that writes to the OOBTree (when a task is completed). Each thread uses the same DB instance, but uses it's own connection instance. ... The problem is that the Downloader thread occasionally throws a ReadConfictError exception. That's surprising as ReadConflictError have been almost eliminated (due to MVCC). I also have a Packer thread that periodically packs the database. The packer might be responsible for the ReadConflictErrors: When the ZODB tries to read an object that was modified since the transaction start by another connection, it instead loads the object state as it was when the transaction startet (this is MVCC). If this state is no longer available (because packing deleted it), then you still get a ReadConflictError. Try not to pack to the current time but retain some history (for MVCC). I had thought that I could just wait for a bit and then retry the read, but that didn't help. You need to abort the transaction -- before anything changes. After the abort, you should get a new chance to read your object (but of cause all modification in the aborted transaction are lost). -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
