Swen Wenzel added the comment:

I have another use case.
The Docs use the producer-consumer pattern as a usage example.
I'm also using this pattern but apparently the consumers are not that stable 
during development phase.

So if one of your consumers dies during execution of its task, before it can 
say 'task done', then you will have a deadlock.
You could of course avoid this by adding a finally block which will claim the 
task is done even if there was some error or exception but then you will lie to 
the producer!
And suppose you only have one consumer and there are still tasks waiting (which 
is the case for my application), then you'll still have a deadlock since nobody 
is there to execute the remaining tasks.
This could be solved by putting the exception handling within the consumer's 
mainloop like this:

Consumer(Thread):
    def __init__(self, queue):
        self.queue = queue

    def run():
        while True:
            task = self.queue.get()
            try:
                # execute task
            except:
                # handle exception (hopefully) without reraising one
            finally:
                self.queue.task_done()

This way, however, the producer won't notice any issues unless the consumer's 
exception handler sets a flag or puts the exception into a collection that can 
be checked by the producer.

But even that is no solution if the consumer executes a task with an endless 
loop or runs into a deadlock itself.

I would propose to throw an exception if queue.Queue.join() returns because of 
a timeout since then you can investigate the cause within the exception handler 
and do not have to check for the consumer's status after each join(). But this 
is microoptimization so I would also be satisfied with the same solution as for 
threading.Thread.join().

----------
nosy: +Swen.Wenzel

_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue9634>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to