Antoine Pitrou added the comment:
Thomas, thanks for the heads up. I would suggest something like the following
patch to multiprocessing.Pool:
$ git diff
diff --git a/Lib/multiprocessing/queues.py b/Lib/multiprocessing/queues.py
index 7f77837..ebbb360 100644
--- a/Lib/multiprocessing/queues.py
+++ b/Lib/multiprocessing/queues.py
@@ -260,8 +260,16 @@ class Queue(object):
info('error in queue thread: %s', e)
return
else:
- import traceback
- traceback.print_exc()
+ self._on_queue_thread_error(e)
+
+ def _on_queue_thread_error(self, e):
+ """
+ Private API called when feeding data in the background thread
+ raises an exception. For overriding by concurrent.futures.
+ """
+ import traceback
+ traceback.print_exc()
+
_sentinel = object()
Then you can write your own Queue subclass in concurrent.futures to handle that
error and clean up/restart whatever needs to be cleaned up or restarted. What
do you think?
----------
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue30414>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
