Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
On Thursday, December 11, 2014 4:53:04 AM UTC, iMath wrote: I think the user interface shouldn't be freezed when using concurrent.futures.ThreadPoolExecutor here,as it executes asynchronously , but it doesn't meet my expectations,anyone can explain why ? any other solutions here to not let user interface freezed? code is here http://stackoverflow.com/questions/27393533/user-interface-freezed-when-using-concurrent-futures-threadpoolexecutor It looks to me that what you are doing is sharing a single core between your GUI and your processing. Threading isn't usually a good approach to Python concurrency that is CPU-bound. Simply changing ThreadPoolExecutor to ProcessPoolExecutor will improve performance, but will still allow the UI to freeze. The approach I use for concurrency in GUI applications is for the GUI to run in the main thread (the default, and there's no choice) and to create a manager thread that does almost no work. This means that the GUI thread gets almost all the processing on the core on which it runs. Whenever there is work to do it is passed to the manager thread which immediately gives it to someone else. The someone else is a process pool. This ensures that the GUI doesn't freeze even in the face of lots of processing (because the processing is done in one or more separate processes). In my book Python in Practice, http://www.qtrac.eu/pipbook.html there's an example of how to do this in chapter 4. The example uses Tkinter but I use exactly the same approach in PyQt and PySide. This chapter also discusses many issues related to Python concurrency. -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
On 12/18/2014 04:16 AM, Mark Summerfield wrote: It looks to me that what you are doing is sharing a single core between your GUI and your processing. Threading isn't usually a good approach to Python concurrency that is CPU-bound. Except that his code was not CPU-bound to begin with. His real problem is that his callback is starting *and* waiting for all the threads to do their work without returning to the main loop, thus blocking the GUI. As for the threads, they are I/O bound--he's simply trying to do concurrent HTTP downloads. So blocking in the GIL is not the issue here. In fact, in lieu of using proper asynchronous I/O, threading of some kind is probably not a bad solution here, but he has to do it within the Qt Framework, using signals to notify the GUI when a thread is finished, or using a synchronization primitive such as a queue, as recommended by your book in fact. -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
Chris Angelico ros...@gmail.com: And I don't remember how Java did things, except that I struggled to find basic fundamental primitives like semaphores, and had to use synchronized functions/objects instead. Java now has a diverse set of synchornization facilities, but the builtin object synchronization produces excellent idioms and is usually preferred (by me). Java also has rigorously defined its multithreaded data model (the Happens-Before Relation). Java's threads have two major problems: * The classic I/O forced you to dedicate a thread for each communication context (connection) because there was no multiplexing facility and because the sockets were blocking (and buffered IIRC). The thread proliferation caused serious scalability issues. Latter-day Java's NIO framework addresses this shortcoming to a great degree, but even that is a surprisingly tricky beast to program for -- it suffers from builtin race conditions. * There is no way to interrupt a thread -- except in Solaris! You can mark a thread for interruption and there is an associated exception but they are not guaranteed to be provided by JVM. And then there's the inherent problems of thread programming: * Deadlocks. * Missing synchronization. which in practice are just too hard for mortals. I've seen it. I've been complicit. Now, when it comes to Python, asyncio is its answer to Java's NIO. It seeks to provide a cross-platform Way to Life, Universe and Everything. A commendable objective. Unfortunately, I find the coroutine approach artificial and unintuitive in practice. Select.epoll(EPOLLET) plus a timer implementation easily beats it (as long as you can limit yourself to linux). Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
在 2014年12月11日星期四UTC+8下午1时25分41秒，Michael Torrie写道： On 12/10/2014 09:52 PM, iMath wrote: I think the user interface shouldn't be freezed when using concurrent.futures.ThreadPoolExecutor here,as it executes asynchronously , but it doesn't meet my expectations,anyone can explain why ? any other solutions here to not let user interface freezed? code is here http://stackoverflow.com/questions/27393533/user-interface-freezed-when-using-concurrent-futures-threadpoolexecutor In most any GUI framework, regardless of your use of threads, your callbacks must return control to the main loop immediately (whether an on-click or an on-timer event), or the GUI *will* freeze. You are spawning threads to download the urls, then sitting there waiting for them to finish in the callback. Of course the GUI will freeze; your callback is blocking it. What you should be doing is spawning the threads to do the download, then have the threads (using a proper QThread mechanism or some other semaphore mechanism) raise a signal to indicate they are done, which you will then catch in the main loop as a normal callback. And really when it comes to I/O and GUIs, asynchronous calls are always better than threads. Not sure what Qt has in the way of asynchronous i/o calls, and I'm not familiar enough with the various async i/o frameworks in Python to speak to that, though I did use Python-Twisted once... very powerful once you get your head wrapped around it. Look through the list archives because in the last 3 or 4 weeks I and another Python user talked proper thread use with PyQt or PySide, with code examples. when it comes to I/O and GUIs, asynchronous calls are always better than threads. I cannot grasp your meaning here, IMO, asynchronous calls are done by using threads. -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
On 12/11/2014 08:20 PM, iMath wrote: 在 2014年12月11日星期四UTC+8下午1时25分41秒，Michael Torrie写道： On 12/10/2014 09:52 PM, iMath wrote: when it comes to I/O and GUIs, asynchronous calls are always better than threads. I cannot grasp your meaning here, IMO, asynchronous calls are done by using threads. Not in your code. All your I/O calls are synchronous; they block until they are finished. But if you mean that calling synchronous calls while in a thread has a similar effect to an asynchronous call, you are correct. But the way you're waiting for the threads to finish, you're blocking your gui. An asynchronous API lets you start long-running I/O calls and define a function that is automatically called upon completion. In other words it's event-driven. Qt may provide everything you need already in an asynchronous form. Check the docs. And use google. Here's a link I found from a google search that illustrates how to fetch a url in Qt using an asynchronous method. You could fire off as many of these as you want, then just wait for signals. http://qt-project.org/wiki/Download_Data_from_URL -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
Michael Torrie torr...@gmail.com: An asynchronous API lets you start long-running I/O calls and define a function that is automatically called upon completion. In other words it's event-driven. Qt may provide everything you need already in an asynchronous form. GUI developers have been doing event-driven programming for decades. That's an excellent preparation for network programming as well. Unfortunately, the minds of a generation of programmers were contaminated by the thread craze of the 1990's (Java and Windows NT). Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
On Fri, Dec 12, 2014 at 6:18 PM, Marko Rauhamaa ma...@pacujo.net wrote: GUI developers have been doing event-driven programming for decades. That's an excellent preparation for network programming as well. Unfortunately, the minds of a generation of programmers were contaminated by the thread craze of the 1990's (Java and Windows NT). Threads predate Java and NT, and with a much MUCH better implementation, on OS/2. I don't consider my mind to have been *contaminated* by the OS/2 threading model, which worked superbly. Like all models, it has its limitations, but it's one worth getting your head around IMO. Knowing when to use threads, when to use processes, and when to keep everything in a single thread with some kind of dispatch loop (and *which* dispatch loop) is a valuable skill. But yes, the Windows threading model does have a number of annoying flaws. And I don't remember how Java did things, except that I struggled to find basic fundamental primitives like semaphores, and had to use synchronized functions/objects instead. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
I think the user interface shouldn't be freezed when using concurrent.futures.ThreadPoolExecutor here,as it executes asynchronously , but it doesn't meet my expectations,anyone can explain why ? any other solutions here to not let user interface freezed? code is here http://stackoverflow.com/questions/27393533/user-interface-freezed-when-using-concurrent-futures-threadpoolexecutor -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt: user interface freezed when using concurrent.futures.ThreadPoolExecutor
On 12/10/2014 09:52 PM, iMath wrote: I think the user interface shouldn't be freezed when using concurrent.futures.ThreadPoolExecutor here,as it executes asynchronously , but it doesn't meet my expectations,anyone can explain why ? any other solutions here to not let user interface freezed? code is here http://stackoverflow.com/questions/27393533/user-interface-freezed-when-using-concurrent-futures-threadpoolexecutor In most any GUI framework, regardless of your use of threads, your callbacks must return control to the main loop immediately (whether an on-click or an on-timer event), or the GUI *will* freeze. You are spawning threads to download the urls, then sitting there waiting for them to finish in the callback. Of course the GUI will freeze; your callback is blocking it. What you should be doing is spawning the threads to do the download, then have the threads (using a proper QThread mechanism or some other semaphore mechanism) raise a signal to indicate they are done, which you will then catch in the main loop as a normal callback. And really when it comes to I/O and GUIs, asynchronous calls are always better than threads. Not sure what Qt has in the way of asynchronous i/o calls, and I'm not familiar enough with the various async i/o frameworks in Python to speak to that, though I did use Python-Twisted once... very powerful once you get your head wrapped around it. Look through the list archives because in the last 3 or 4 weeks I and another Python user talked proper thread use with PyQt or PySide, with code examples. -- https://mail.python.org/mailman/listinfo/python-list
