Hi all
I'm writing a multithread app using ZODB/ZEO for persistence. The app consists in a main app which start the connection with ZEO with ClientStorage, launch several threads to gather, parse and fill the DB with data from the network, and, just for testing purposes now, a loop printing the data in the DB every 5 seconds. Each parser store its data in a PersistentList in conn.dbroot() and commit the changes. The problem is... I can't see the committed changes from the main thread, unless I do a commit there too, which seems strange. Not only that, but sometimes I get an assertion error on ZEO/cache.py. This is the first time I'm using ZODB/ZEO for multithreaded app, so maybe I'm doing something really stupid and did not notice. Another problem not related to this is the size and how fast the DB grows. I'm think this is related to undo features, so I would like to know if it's possible to disable it or something else. These are the relevant parts of my code. The code for commiting data on each thread: ### for x in xrange(self._commit_attempts): try: del self.reg_db[:] self.reg_db.extend(regs) transaction.get().commit() except ConflictError, msg: self.logger.debug( 'ConflictError:%s pause a while and retry'%msg ) transaction.get().abort() time.sleep(0.5) else: logger.debug('Commited succesfully') return logger.critical('Commit failed') ### The test loop: ### while 1: for name, regs in dbroot.items(): logger.debug("DB list for '%s' with %s regs"%(name, len(regs))) time.sleep(5) # I only get updates with this commit here transaction.get().commit() ### This is the traceback for the assertion error (line 86 is the commit mentioned above): Traceback (most recent call last): File "test_runparsers.py", line 48, in ? main() File "../../runparsers.py", line 86, in main transaction.get().commit() File "/usr/lib/python2.4/site-packages/transaction/_transaction.py", line 380, in commit self._saveCommitishError() # This raises! File "/usr/lib/python2.4/site-packages/transaction/_transaction.py", line 378, in commit self._commitResources() File "/usr/lib/python2.4/site-packages/transaction/_transaction.py", line 441, in _commitResources rm.tpc_finish(self) File "/usr/lib/python2.4/site-packages/ZODB/Connection.py", line 679, in tpc_finish self._storage.tpc_finish(transaction, callback) File "/usr/lib/python2.4/site-packages/ZEO/ClientStorage.py", line 974, in tpc_finish self._update_cache(tid) File "/usr/lib/python2.4/site-packages/ZEO/ClientStorage.py", line 999, in _update_cache self._cache.invalidate(oid, version, tid) File "/usr/lib/python2.4/site-packages/ZEO/cache.py", line 367, in invalidate assert tid is not None and cur_tid < tid AssertionError And the traceback for the exception ocurred at the server when I get the assertion error on the app Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/ZEO/zrpc/connection.py", line 421, in handle_request ret = meth(*args) File "/usr/lib/python2.4/site-packages/ZEO/StorageServer.py", line 248, in loadEx return self.storage.loadEx(oid, version) File "/usr/lib/python2.4/site-packages/ZODB/FileStorage/FileStorage.py", line 523, in loadEx pos = self._lookup_pos(oid) File "/usr/lib/python2.4/site-packages/ZODB/FileStorage/FileStorage.py", line 514, in _lookup_pos raise POSKeyError(oid) POSKeyError: 0x00 Thanks for any help -- Pedro Werneck _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev