Hi all
I'm writing a multithread app using ZODB/ZEO for persistence. The app
consists in a main app which start the connection with ZEO with
ClientStorage, launch several threads to gather, parse and fill the DB
with data from the network, and, just for testing purposes now, a loop
printing the data in the DB every 5 seconds. Each parser store its data
in a PersistentList in conn.dbroot() and commit the changes.
The problem is... I can't see the committed changes from the main
thread, unless I do a commit there too, which seems strange. Not only
that, but sometimes I get an assertion error on ZEO/cache.py.
This is the first time I'm using ZODB/ZEO for multithreaded app, so
maybe I'm doing something really stupid and did not notice.
Another problem not related to this is the size and how fast the DB
grows. I'm think this is related to undo features, so I would like to
know if it's possible to disable it or something else.
These are the relevant parts of my code. The code for commiting data on
each thread:
###
for x in xrange(self._commit_attempts):
try:
del self.reg_db[:]
self.reg_db.extend(regs)
transaction.get().commit()
except ConflictError, msg:
self.logger.debug(
'ConflictError:%s pause a while and retry'%msg
)
transaction.get().abort()
time.sleep(0.5)
else:
logger.debug('Commited succesfully')
return
logger.critical('Commit failed')
###
The test loop:
###
while 1:
for name, regs in dbroot.items():
logger.debug(DB list for '%s' with %s regs%(name,
len(regs)))
time.sleep(5)
# I only get updates with this commit here
transaction.get().commit()
###
This is the traceback for the assertion error (line 86 is the commit
mentioned above):
Traceback (most recent call last):
File test_runparsers.py, line 48, in ?
main()
File ../../runparsers.py, line 86, in main
transaction.get().commit()
File /usr/lib/python2.4/site-packages/transaction/_transaction.py,
line 380, in commit
self._saveCommitishError() # This raises!
File /usr/lib/python2.4/site-packages/transaction/_transaction.py,
line 378, in commit
self._commitResources()
File /usr/lib/python2.4/site-packages/transaction/_transaction.py,
line 441, in _commitResources
rm.tpc_finish(self)
File /usr/lib/python2.4/site-packages/ZODB/Connection.py, line 679,
in tpc_finish
self._storage.tpc_finish(transaction, callback)
File /usr/lib/python2.4/site-packages/ZEO/ClientStorage.py, line
974, in tpc_finish
self._update_cache(tid)
File /usr/lib/python2.4/site-packages/ZEO/ClientStorage.py, line
999, in _update_cache
self._cache.invalidate(oid, version, tid)
File /usr/lib/python2.4/site-packages/ZEO/cache.py, line 367, in
invalidate
assert tid is not None and cur_tid tid
AssertionError
And the traceback for the exception ocurred at the server when I get
the assertion error on the app
Traceback (most recent call last):
File /usr/lib/python2.4/site-packages/ZEO/zrpc/connection.py, line
421, in handle_request
ret = meth(*args)
File /usr/lib/python2.4/site-packages/ZEO/StorageServer.py, line
248, in loadEx
return self.storage.loadEx(oid, version)
File
/usr/lib/python2.4/site-packages/ZODB/FileStorage/FileStorage.py, line
523, in loadEx
pos = self._lookup_pos(oid)
File
/usr/lib/python2.4/site-packages/ZODB/FileStorage/FileStorage.py, line
514, in _lookup_pos
raise POSKeyError(oid)
POSKeyError: 0x00
Thanks for any help
--
Pedro Werneck
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
