Then I suppose a natural follow up question would be, what is the point of QThread being exposed in PySide? Qt already offers QtConcurrent to handy asynchronous tasks.
On Tue, Dec 10, 2019 at 7:16 AM Jason H <jh...@gmx.com> wrote: > Not really. > > Maybe you need to flip your application in-side out? Make it a C++ program > with all the threads it needs and just embed a python interpreter for your > processing functions? Not sure how you'll efficiently move data between the > two sides though. For OpenCV (c++), it's lucky that Qt and OpenCV support a > few same memory formats so that you can just take the data as an opaque > pointer. > > As long as you want to code your processing of data in Python, you'll have > the GIL, with will have this problem. C++ Qt can deal with it with no > problem. > > Maybe you could use some Async python (Python 3 only) and integrate Qt > event loop with the python aync event loop and maybe do it that way? But > I'm not a fan of Python async (Qt does it better, and the event loop > integration isn't offically supported. Python's event loop is nvery new > and API isn't stable.) > > *Sent:* Monday, December 09, 2019 at 6:14 PM > *From:* "Ognyan Moore" <ognyan.mo...@gmail.com> > *To:* "Jason H" <jh...@gmx.com> > *Cc:* "Israel Brewster" <ijbrews...@alaska.edu>, pyside@qt-project.org > *Subject:* Re: [PySide] PySide Digest, Vol 83, Issue 13 > In my case, I'm trying to do very rapid spectrogram calculations, ....lots > of them, they each take ~40-100ms but the fact that I do these calculations > very rapidly makes it problematic for them occurring in the GUI thread. > These calculations are triggered by GUI events in pyqtgraph (user adjusts a > plot window in some way); and I attempt to redraw a spectrogram calculation > in as close to real-time as possible. Now, I can make it so that the > function running on a separate thread _never_ has to access some reference > in the main thread. Clearly there must be some way to spawn up a longish > numpy task that does not effect the GUI. > > > > On Mon, Dec 9, 2019 at 3:07 PM Jason H <jh...@gmx.com> wrote: > >> Before anyone points it out, I'm not suggesting Qt is the problem ;-) >> Just that if it is a data-transform-heavy task shuffeling between Qt and >> Python might be ok for display and let the server handle the processing. >> >> >> >> *Sent:* Monday, December 09, 2019 at 3:47 PM >> *From:* "Israel Brewster" <ijbrews...@alaska.edu> >> *To:* "Jason H" <jh...@gmx.com> >> *Cc:* "Ognyan Moore" <ognyan.mo...@gmail.com>, pyside@qt-project.org >> *Subject:* Re: [PySide] PySide Digest, Vol 83, Issue 13 >> >> On Dec 9, 2019, at 11:34 AM, Jason H <jh...@gmx.com> wrote: >> >> "The Pandas data frames are pure Python classes, so they are not easy to >> construct from C/C++, but the underlying data storage of the individual >> columns (see class Series source) >> <https://github.com/pydata/pandas/blob/master/pandas/core/series.py#L88> is >> numpy 1D array. Numpy has a nice C API >> <http://docs.scipy.org/doc/numpy-1.10.0/reference/c-api.html> and you >> can construct the numpy arrays from C and then pass it to your Python >> code." - >> https://stackoverflow.com/questions/11607387/is-there-a-c-c-api-for-python-pandas >> >> https://docs.scipy.org/doc/numpy-1.10.0/reference/c-api.html >> >> Ugh, it's not pretty. But I bet you could wrap the C api or target the >> Python API directly. >> >> If it were me though, having no idea what you are actually doing, I would >> problably write a json-RPC server in python and just relegate Qt to a >> pretty UI that makes json-RPC requests. >> >> >> Interesting thought. Separate out the UI and the backend in completely >> separate processes that don’t interfere with each other. Definitely >> something to consider, although given that I need to graph the data, it >> might still result in a lot of data needing to be shuffled around. >> >> As far as what I’m actually doing, it’s a data processing app that pulls >> in data from gas data collection flights, allows the user to select a bunch >> of different areas of interest, processes the data, and creates pretty >> graphs. >> >> --- >> Israel Brewster >> Software Engineer >> Alaska Volcano Observatory >> Geophysical Institute - UAF >> 2156 Koyukuk Drive >> Fairbanks AK 99775-7320 >> Work: 907-474-5172 >> cell: 907-328-9145 >> >> >> The other me would just figure out a way to bypass pandas and work out >> how to do it entirely in C++ (Eigen?) except for the storage format. IIRC, >> Numpy arrays are just c arrays at the lowest level (don't quote me on that) >> >> >> >> >> *Sent:* Monday, December 09, 2019 at 1:56 PM >> *From:* "Israel Brewster" <ijbrews...@alaska.edu> >> *To:* "Jason H" <jh...@gmx.com> >> *Cc:* "Ognyan Moore" <ognyan.mo...@gmail.com>, pyside@qt-project.org >> *Subject:* Re: [PySide] PySide Digest, Vol 83, Issue 13 >> >> On Dec 9, 2019, at 9:37 AM, Jason H <jh...@gmx.com> wrote: >> >> That article does not take into account that the QThread logic is in >> python. That is the problem >> The problem is the processing function is in Python. Moving it to be C++ >> only would work fine. Python would invoke a compiled function that would >> start the multithreaded processing and complete immediately, then the Qt >> event loop could be used via signals/slots to inform completion. >> >> >> Exactly. If the function could be moved to be C++ only, there wouldn’t be >> an issue. If the function could be executed in a different process rather >> than a thread, there wouldn’t be an issue. Unfortunately, the sheer volume >> of data involved makes moving the data between python and anything else a >> non-starter - the function *has* to be able to modify pandas data frames in >> the calling class directly - unless there is some shortcut that could make >> it work based on the fact that the data is in pandas data frame, and at its >> core pandas/numpy *is* C++ already. >> >> --- >> Israel Brewster >> Software Engineer >> Alaska Volcano Observatory >> Geophysical Institute - UAF >> 2156 Koyukuk Drive >> Fairbanks AK 99775-7320 >> Work: 907-474-5172 >> cell: 907-328-9145 >> >> >> >> >> >> *Sent:* Monday, December 09, 2019 at 1:05 PM >> *From:* "Ognyan Moore" <ognyan.mo...@gmail.com> >> *To:* pyside@qt-project.org >> *Subject:* Re: [PySide] PySide Digest, Vol 83, Issue 13 >> I think it would be great if we can have some guidance regarding >> multi-threading with CPU heavy tasks, not just IO bound tasks. Say I >> wanted to multi-thread a function >> >> >> def cpu_intensive_function(a: int, b: int, c: int) -> np.ndarray: >> return some_numpy_function_calls(a, b, c) >> >> >> How would I run this function in such a way that the computation does not >> hang the GUI thread? I have attempted to follow the guidance on this blog >> post here: >> https://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/ >> . >> Where I utilize QObject.moveToThread(some_thread) as that article suggests, >> the function still executes in the GUI thread. >> >> Some caveats here being that I am making an attempt to have no references >> to mutable objects in the GUI thread and the thread doing the computation. >> >> Thanks! >> Ogi >> >> >> On Mon, Dec 9, 2019 at 3:00 AM <pyside-requ...@qt-project.org> wrote: >> >>> Send PySide mailing list submissions to >>> pyside@qt-project.org >>> >>> To subscribe or unsubscribe via the World Wide Web, visit >>> https://lists.qt-project.org/listinfo/pyside >>> or, via email, send a message with subject or body 'help' to >>> pyside-requ...@qt-project.org >>> >>> You can reach the person managing the list at >>> pyside-ow...@qt-project.org >>> >>> When replying, please edit your Subject line so it is more specific >>> than "Re: Contents of PySide digest..." >>> >>> >>> Today's Topics: >>> >>> 1. Re: Keeping GUI responsive (Sivan Greenberg) >>> >>> >>> ---------------------------------------------------------------------- >>> >>> Message: 1 >>> Date: Sun, 8 Dec 2019 17:54:10 +0200 >>> From: Sivan Greenberg <si...@omniqueue.com> >>> To: Israel Brewster <ijbrews...@alaska.edu> >>> Cc: pyside@qt-project.org >>> Subject: Re: [PySide] Keeping GUI responsive >>> Message-ID: >>> < >>> cajncf-av+xygpo6gv+kr-neifnedtiwg3gyylyg16njqjbr...@mail.gmail.com> >>> Content-Type: text/plain; charset="utf-8" >>> >>> Hi Israel, >>> >>> If your non main function is mostly IO bound, you could try implement >>> your >>> processing function with an interface that allows for polling of results >>> using co-routines which is very slim and very robust (I used that in the >>> past for several PyQt & PySide small apps but also for a full blown >>> project >>> for Ubuntu back in 2006[0] , though with PyGTK - but principle is same) >>> and >>> will allow your GUI main loop to be in charge, even if you ever need to >>> use >>> a spawned shell command or program! (see links below) >>> >>> It seems that nowadays, there are even ready made libs for that (this >>> one >>> seems very interesting): https://github.com/reclosedev/async_gui and >>> also >>> supports abstraction over multiprocessing which would support you even if >>> the the subtask is indeed CPU intensive (like data cleansing , validation >>> etc). >>> >>> If you're using python 3 , the whole coroutines interface has been >>> greatly >>> simplified, and could be easier to use, however the first guide will >>> explain in deep and show how to achieve cooperative multitasking using >>> plain Python 2.x interface: >>> - >>> >>> http://masnun.com/2015/11/13/python-generators-coroutines-native-coroutines-and-async-await.html >>> - >>> >>> https://medium.com/velotio-perspectives/an-introduction-to-asynchronous-programming-in-python-af0189a88bbb >>> >>> HTH, >>> >>> -Sivan >>> >>> [0]: >>> >>> http://www.ubuntugeek.com/hubackup-backup-application-for-ubuntu-home-users.html >>> >>> On Thu, Dec 5, 2019 at 1:41 AM Israel Brewster <ijbrews...@alaska.edu> >>> wrote: >>> >>> > I know this is a FAQ, however I haven’t been able to make any of the >>> > standard answers work for me. Here’s the situation: >>> > >>> > - using PySide2 5.12.2 >>> > - I have an object (QMainWindow subclass) that contains most of the >>> code >>> > for my application >>> > - One of the functions that runs in response to user input takes >>> around 2 >>> > seconds to run. The GUI obviously freezes during this time (BAD!) >>> > - Said function needs to access and modify several large variables >>> (pandas >>> > data frames) from the main object >>> > >>> > So here’s the problem: If I run this function as a separate (python) >>> > thread, that doesn’t help - the GUI is still frozen. I’m thinking this >>> is >>> > due to the GIL, but I could be wrong about that. Running under the >>> > multiprocessing module, however, doesn’t appear to be an option due to >>> the >>> > number and size of the data structures that the function needs to >>> modify, >>> > and if I try just to see what happens, the process actually crashes. >>> > >>> > So what are my options here? How can I keep the GUI responsive while >>> this >>> > function runs, without being able to spin it off as a separate >>> process? Or >>> > is the only option going to be to completely rip apart the function >>> and try >>> > to re-build it in such a way that it can, somehow, still access the >>> memory >>> > from the main thread, while doing the processing in a separate >>> function? >>> > --- >>> > Israel Brewster >>> > Software Engineer >>> > Alaska Volcano Observatory >>> > Geophysical Institute - UAF >>> > 2156 Koyukuk Drive >>> > Fairbanks AK 99775-7320 >>> > Work: 907-474-5172 >>> > cell: 907-328-9145 >>> > >>> > _______________________________________________ >>> > PySide mailing list >>> > PySide@qt-project.org >>> > https://lists.qt-project.org/listinfo/pyside >>> > >>> >>> >>> -- >>> -Sivan >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: < >>> http://lists.qt-project.org/pipermail/pyside/attachments/20191208/fc69df57/attachment-0001.html >>> > >>> >>> ------------------------------ >>> >>> Subject: Digest Footer >>> >>> _______________________________________________ >>> PySide mailing list >>> PySide@qt-project.org >>> https://lists.qt-project.org/listinfo/pyside >>> >>> >>> ------------------------------ >>> >>> End of PySide Digest, Vol 83, Issue 13 >>> ************************************** >> >> _______________________________________________ PySide mailing list >> PySide@qt-project.org https://lists.qt-project.org/listinfo/pyside >> _______________________________________________ >> PySide mailing list >> PySide@qt-project.org >> https://lists.qt-project.org/listinfo/pyside >> >>
_______________________________________________ PySide mailing list PySide@qt-project.org https://lists.qt-project.org/listinfo/pyside
