My test code (modified from Mario's),
=============================================
#!/usr/bin/env python
from services.tables_settings import composite_h5_filename
import tables
import multiprocessing as mp
def _worker(qout = None):
fn = composite_h5_filename
fp = tables.openFile(fn)
print "About to load: ", fn
rows = fp.root.t092010.where('(origWac <5.9) & (origWac>5.8)')
count = 0
for row in rows:
count +=1
print count
fp.close()
if qout is not None:
qout.put("Done")
if __name__ == "__main__":
print "**** Running from main process:"
_worker()
print "**** Running from subprocess:"
qout = mp.Queue()
ps = mp.Process(target=_worker, args=(qout,))
ps.daemon = True
ps.start()
print qout.get();
=============================================
The test works for me if the where condition only returns a small number of
rows (~40), but fails for larger queries (14000 rows). In these cases, the
query will will finish on the main process, but the forked process hangs
after iterating through about half the total rows on the fork. Admittedly,
I don't know much about multiprocessing in python, and this hang might not
be caused by pytables.
Relevant versions,
tables.__version__
Out[2]: '2.2pro'
numexpr.__version__
Out[4]: '1.4.1'
and python 2.6.6.
-Ben
------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
