Isn't this a job for threading.BoundedSpemaphore()? On 3/14/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > FWIW, I've been working on a way to simplify the use of queues with daemon > consumer threads > > Sometimes, I launch one or more consumer threads that wait for a task to > enter a > queue and then work on the task. A recurring problem is that I sometimes need > to > know if all of the tasks have been completed so I can exit or do something > with > the result. > > If each thread only does a single task, I can use t.join() to wait until the > task is done. However, if the thread stays alive and waits for more Queue > entries, then there doesn't seem to be a good way to tell when all the > processing is done. > > So, the idea is to create a subclass of Queue that increments a counter when > objects are enqueued, that provides a method for worker threads to decrement > the > counter when the work is done, and offers a blocking join() method that waits > until the counter is zero > > # main thread > q = TaskQueue() > for t in worker_threads(): > t.start() > for task in tasklist: > q.put(task) # increments the counter and enqueues a > task > q.join() # all of the tasks are done (counter is > zero) > do_work_on_results() > > > > # worker thread > while 1: > task = q.get() # task is popped but the counter is > unchanged > do_work(task) > q.decrement() # now the counter gets reduced > > > The idea is still in its infancy (no implementation and it hasn't been tried > in > real-world code) but I would like to get some feedback. If it works out, I'll > post a recipe to ASPN and see how it goes. > > > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org >
-- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
