Hi, I am trying to use PP with ZODB.
Parallel Python (www.parallelpython.com) provides works as a job server which starts processes, thus providing a mechanism for the parallel execution of python code on SMP machines or clusters. Each job basically calls a function and then executes it (module dependencies are passed as an import list for each job). For concurrent writing to the DB I had to find a way to have each of these processes create their own connections. However, PP, passes variables to the job-processes as pickles, and the db object is not picklable so it cannot be passed to PP jobs. The solution I am trying is to offload the db handling code to a separate module that can be added to the import list of a PP job: dbserve.py Code: import ZODB import ZEO.ClientStorage class Data: def __init__(self): #setting up ZEO server #os.system('runzeo -a localhost:8008 -f SN.fs &') # setup database storage = ZEO.ClientStorage.ClientStorage(('localhost',8008)) self.db = ZODB.DB(storage) def getConn(self): conn = self.db.open() return conn.root() That seems to be a solution, but fails because now each process tries to create a ClientStorage object from which to derive a connection. and the server won't accept it.... An error has occured during the function execution Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/ppworker.py", line 91, in run __result = __f(*__args) File "<string>", line 4, in cenario File "<string>", line 10, in Reps File "/home/flavio/Documents/Papers/VacBelief/src/dbserve.py", line 14, in __init__ self.db = ZODB.DB(storage) File "/usr/lib/python2.5/site-packages/ZODB3-3.8.0-py2.5-linux-i686.egg/ZODB/DB.py", line 270, in __init__ storage.load(z64, '') File "/usr/lib/python2.5/site-packages/ZEO/ClientStorage.py", line 757, in load p, s, v, pv, sv = self._server.zeoLoad(oid) File "/usr/lib/python2.5/site-packages/ZEO/ServerStub.py", line 82, in zeoLoad return self.rpc.call('zeoLoad', oid) File "/usr/lib/python2.5/site-packages/ZEO/zrpc/connection.py", line 484, in call r_flags, r_args = self.wait(msgid) File "/usr/lib/python2.5/site-packages/ZEO/zrpc/connection.py", line 574, in wait raise DisconnectedError() Anyone has any experience with ZODB and PP? any sugestions? In summary: I need to expose a DB Object to multiple processes so that they can derive their own connections But this object cannot passed at job-creation time because they are not pickleable, and they apparently cannot be created by each job because I cannot have multiple ClientStorage objects targetting the same phisical zeo server (is this a real limitation?) thanks in advance, Flávio C. Coelho _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev