[Guido]
> >> Apart from trying to guess the API without reading the docs (:-), what
> >> are the use cases for using put/get with a timeout? I have a feeling
> >> it's not that common.
[Josiah Carlson]
> > With timeout=0, a shared connection/resource pool (perhaps DB, etc., I
> > use one in the tuple space implementation I have for connections to the
> > tuple space).
[Tim Peters]
> Passing timeout=0 is goofy: use {get,put}_nowait() instead. There's
> no difference in semantics.
I understand this, as do many others who use it. However, having both
manually and automatically tuned timeouts myself in certain applications,
the timeout=0 case is useful. Uncommon? Likely, I've not yet seen any
examples of anyone using this particular timeout method at koders.com .
> > Note that technically speaking, Queue.Queue from Pythons
> > prior to 2.4 is broken: get_nowait() may not get an object even if the
> > Queue is full, this is caused by "elif not self.esema.acquire(0):" being
> > called for non-blocking requests. Tim did more than simplify the
> > structure by rewriting it, he fixed this bug.
>
> I don't agree it was a bug, but I did get fatally weary of arguing
> with people who insisted it was ;-) It's certainly easier to explain
> (and the code is easier to read) now.
When getting an object from a non-empty queue fails because some other
thread already had the lock, and it is a fair assumption that the other
thread will release the lock within the next context switch...
Because I still develop on Python 2.3 (I need to support a commercial
codebase made with 2.3), I was working around it by using the timeout
parameter:
try:
connection = connection_queue.get(timeout=.000001)
except Queue.Empty:
connection = make_new_connection()
With only get_nowait() calls, by the time I hit 3-4 threads, it was
failing to pick up connections even when there were hundreds in the
queue, and I quickly ran into the file handle limit for my platform, not
to mention that the server I was connecting to used asynchronous sockets
and select, which died at the 513th incoming socket.
I have since copied the implementation of 2.4's queue into certain
portions of code which make use of get_nowait() and its variants
(handline the deque reference as necessary).
Any time one needs to work around a "not buggy feature" with some
claimed "unnecessary feature", it tends to smell less than pristine to
my nose.
> > With block=True, timeout=None, worker threads pulling from a work-to-do
> > queue, and even a thread which handles the output of those threads via
> > a result queue.
>
> Guido understands use cases for blocking and non-blocking put/get, and
> Queue always supported those possibilities. The timeout argument got
> added later, and it's not really clear _why_ it was added. timeout=0
> isn't a sane use case (because the same effect can be gotten with
> non-blocking put/get).
def t():
try:
#thread state setup...
while not QUIT:
try:
work = q.get(timeout=5)
except Queue.Empty:
continue
#handle work
finally:
#thread state cleanup...
Could the above be daemonized? Certainly, but then the thread state
wouldn't be cleaned up. If you can provide me with a way of doing the
above with equivalent behavior, using only get_nowait() and get(), then
put it in the documentation. If not, then I'd say that the timeout
argument is a necessarily useful feature.
[Guido]
> But one lesson we can learn from sockets (or perhaps the reason why
> people kept asking for timeout=0 to be "fixed" :) is that timeout=0 is
> just a different way to spell blocking=False. The socket module makes
> sure that the socket ends up in exactly the same state no matter which
> API is used; and in fact the setblocking() API is redundant.
This would suggest to me that at least for sockets, setblocking() could
be deprecated, as could the block parameter in Queue. I wouldn't vote
for either deprecation, but it would seem to make more sense than to
remove the timeout arguments from both.
- Josiah
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
