[issue17985] multiprocessing Queue.qsize() and Queue.empty() with different results
Richard Oudkerk added the comment: This is really a documentation issue. The doc fix for #18277 covers this. -- components: +Library (Lib) -Extension Modules resolution: - wont fix stage: - committed/rejected status: open - closed type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17985] multiprocessing Queue.qsize() and Queue.empty() with different results
New submission from Andre Dias: The problem is that Queue.empty() is True even if Queue.qsize()0! #!/usr/bin/python from multiprocessing import Queue numbers=Queue() for i in range (0,10): numbers.put(i) if numbers.qsize()0 and numbers.empty(): print BUG?! -- components: Extension Modules messages: 189302 nosy: aod priority: normal severity: normal status: open title: multiprocessing Queue.qsize() and Queue.empty() with different results versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17985] multiprocessing Queue.qsize() and Queue.empty() with different results
Richard Oudkerk added the comment: From the docs: qsize() Return the approximate size of the queue. Because of multithreading/multiprocessing semantics, this number is not reliable. Adding a short sleep before calling qsize() and empty() should make things appear to work. But really, there are no good reasons for using qsize() except for debugging. The same applies to queue.Queue. -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17985] multiprocessing Queue.qsize() and Queue.empty() with different results
Andre Dias added the comment: But qsize() is working. what is not working is empty() 2013/5/15 Richard Oudkerk rep...@bugs.python.org Richard Oudkerk added the comment: From the docs: qsize() Return the approximate size of the queue. Because of multithreading/multiprocessing semantics, this number is not reliable. Adding a short sleep before calling qsize() and empty() should make things appear to work. But really, there are no good reasons for using qsize() except for debugging. The same applies to queue.Queue. -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17985] multiprocessing Queue.qsize() and Queue.empty() with different results
Richard Oudkerk added the comment: On 15/05/2013 10:25pm, Andre Dias wrote: But qsize() is working. what is not working is empty() empty() returns False when there is data in the underlying pipe. But the data does not enter the pipe until a background thread has written it to the pipe. This should not cause any problems. Using Queue.empty() is always subject to races. It is just that the multiprocessing version has an additional type of race compared to the normal one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17985] multiprocessing Queue.qsize() and Queue.empty() with different results
Andre Dias added the comment: RIchard, But the example program has no races, no threads, nothing. empty() is returning TRUE even though qsize() is 0 (which actually is) And it happens almost every time I run that small example. I had read the module doc, and I know its an unreliable method, but man, the example program is too simple to fail Truth is I decided qsize() is more reliable and im using it in my programs, but man empty() problem is so ridiculous that I decided to submit here 2013/5/15 Richard Oudkerk rep...@bugs.python.org Richard Oudkerk added the comment: On 15/05/2013 10:25pm, Andre Dias wrote: But qsize() is working. what is not working is empty() empty() returns False when there is data in the underlying pipe. But the data does not enter the pipe until a background thread has written it to the pipe. This should not cause any problems. Using Queue.empty() is always subject to races. It is just that the multiprocessing version has an additional type of race compared to the normal one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17985] multiprocessing Queue.qsize() and Queue.empty() with different results
Richard Oudkerk added the comment: On 15/05/2013 11:33pm, Andre Dias wrote: But the example program has no races, no threads, nothing. empty() is returning TRUE even though qsize() is 0 (which actually is) And it happens almost every time I run that small example. I had read the module doc, and I know its an unreliable method, but man, the example program is too simple to fail Truth is I decided qsize() is more reliable and im using it in my programs, but man empty() problem is so ridiculous that I decided to submit here If you assume everything is single-threaded then you would assume that this is too simple to fail. But there *is* more than thread involved. empty() and qsize() are basically redundant. You would almost certainly be better off using get_noblock()/put_noblock() and catching Empty/Full. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17985 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
