On 05/06/2013 3:30pm, Jay L. wrote:
I am working with the multiprocessing module attempting to get parallel
reads working. Currently, if I open the file in each child process and
read from an offset I achieve concurrency. If I attempt to open the
file in the parent (via a class) and pass the class object to the
children I get an overflow error.
If you inherit the parent process's fd, then (if I remember correctly)
all the processes will share the same file offset: seeking in one
process will affect the file offset in *all* processes. It's not
surprising that this causes problems, although I can't give a precise
explanation of what is happening.
BTW, the exception object returned to the main process will not have a
traceback attached to it. To see a full traceback you could catch,
print and reraise the exception in the worker process:
def get_object(row):
try:
return file.get(row)
except:
# print exception with traceback then reraise
sys.excepthook(**sys.exc_info())
raise
_______________________________________________
concurrency-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/concurrency-sig
