Hi everyone,
I'm evaluating PyTables as a backend for data storage in large
astronomical surveys. So far the results have been fantastic, except for
the following issue: if a table is opened and iterated on from within a
subprocess instantiated with multiprocessing.Process, the program hangs.
This test case reproduces the problem with pytables 2.2 on Python 2.6
and 2.7:
===========================
#!/usr/bin/env python
import tables
import multiprocessing as mp
def _worker(qout = None):
fn = "testcase.h5"
fp = tables.openFile(fn)
print "About to load: ", fn
rows = fp.root.catalog.where('(ra > 0)')
print "Got the iterator, about to iterate"
row = next(rows)
print "Succeeded in one iteration\n"
fp.close()
if qout is not None:
qout.put("Done")
if __name__ == "__main__":
print "**** Running from main process:"
_worker()
print "**** Running from subprocess:"
qout = mp.Queue()
ps = mp.Process(target=_worker, args=(qout,))
ps.daemon = True
ps.start()
print qout.get();
===========================
The output:
============================
[mju...@nebel skysurvey]$ ./db.py
**** Running from main process:
About to load: testcase.h5
Got the iterator, about to iterate
Succeeded in one iteration
**** Running from subprocess:
About to load: testcase.h5
Got the iterator, about to iterate
[[[ ... and here the process just hangs ]]]
============================
The testcase.h5 file can be found on
http://nebel.rc.fas.harvard.edu/mjuric/pytables/ , but I suspect the
problem is not file related (it happens with nearly every one I tried).
Is this a known issue (and is there a workaround), or am I doing
something wrong?
Regards,
--
Mario Juric,
Hubble Fellow, Harvard-Smithsonian Center for Astrophysics
Web : http://www.sns.ias.edu/~mjuric/
Phone : +1 617 744-9003 PGP: ~mjuric/crypto/public.key
E-mail reading/answering policy: http://majuric.org/email/
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
