Thank you all for your thoughts and responses. I think I will use QThreads (instead of threading.thread for Python) if I choose threads over multiprocessing. Now, after looking at some of the posts, I see that memory is not shared in multiprocessing, whereas threads have a shared memory. I have like 10 plots to be continuously plotting real time data, and that data would be read from a file that is being written by another Python script on the disk. Yet, whether or not I should switch to pyqtgraph is still a question as I was able to do real time plotting of a million full data points by threading with Tkinter when I last used matplotlib.And that is what I need now as well, however, using PyQt gives me advantage of possibly putting in QThreads to call out pyqtsignals which would be otherwise complex with just Python threads. Since I am rethinking my approach while using PyQt, I am also thinking whether I should use threads or not and just use multiprocessing (which is not something I am experienced with). I know Martin already answered about multiprocessing vs threading but I am still confused whether what I did earlier with Tkinter GUIs is correct or should I use multiprocessing now for PyQt GUIs. My software needs to get the data read continuously from a file on disk, and plot it continuously (real time) with 10 plot figures. I understand that multiprocessing is for heavy number crunching, but why would it not be a good candidate for data fetching from the disk (which could be one process) parallely to other 10 plot processes? My old approach consisted of data fetching as a thread and plotting as another thread different from the main thread. But, I didn't know multiprocessing capabilities then and that it can distribute the processing load to multiple cores. So, I am wondering if I can deploy multiprocessing for data fetching processes and data plotting processes, and there may be additional processes added later that have some mathematical calculations but not complicated ones (meaning scaling data vector or addition and subtraction of data vectors), OR if that is not correct use of multiprocessing and that I should stick with threading instead? Another reason why I am procrastinating to try one of the two, and research first, is because I am making a software that should run real time forever until interrupted, and that plots should be able to plot data of let's say 3 days ago where a day full data is like 10 million plot points (unlike some real time GUIs which would work as real time but for sometime only).Of Course I can read the data from disk if user needs to plot a 3 day or a week old data, but I don't know how plausible that is, as that comes with a overhead of read time for data, so I instead store data in a numpy array of length 10 million (beyond which software really struggles). At the same time, having all the week-long data in memory (using numpy or whatever) isn't possible or maybe also inefficient. Any thoughts on this? Your thoughts would be greatly appreciated. Thanks again!
On Thu, Oct 13, 2022 at 1:38 PM Ognyan Moore <[email protected]> wrote: > I just want to add that the real advantage of QThreads (when talking about > pyqtgraph) is that you can do CPU-heavy processing in a non-GUI thread, so > the GUI doesn't (temporarily) freeze/lag as number crunching is happening. > I use a QThread to do some log-based calculations here, and I use the > Signal/Slot mechanism to relay the information I want pyqtgraph to plot: > > > https://github.com/j9ac9k/barney/blob/main/src/barney/controllers/SubPlotControllers/SpectrogramController.py > > The calculation is based on an interactive element in the GUI, so when I > first had this setup, when the user would adjust a linear region item, the > GUI would temporarily freeze while I was doing a log calculation on a > ~500x2000 numpy array. Moving to a QThread meant the resulting plot was a > little late to update vs. the others, but the GUI didn't lag or freeze. > > Adding QThreads does add complexity, and I hope to roll an example in the > pyqtgraph example library on how to do this; but the general advice is > don't go down that route unless you have an actual need. > > On Thu, Oct 13, 2022 at 5:27 AM Edmondo Giovannozzi < > [email protected]> wrote: > >> Personally I want speed in a GUI application. So I always use pyqtgraph, >> even sometime modifying the pyqtgraph source in order to have what I need. >> It is well written, so modifying it is easy. >> If a calculation is slowing me down I can transform it in a Fortran or C >> code and use ctypes to link it to Python. >> >> Il Mer 12 Ott 2022, 23:36 Martin Chase <[email protected]> ha >> scritto: >> >>> Heya, >>> >>> There isn't any performance difference between QThreads and python >>> Threads; that's mostly a difference of how you want to integrate with Qt >>> (QThreads fit into the rest of the Qt lifecycle better). Otherwise, the >>> question that I ask to determine thread v. multiprocess is: will my worker >>> mostly be sitting around waiting? If that's the case (e.g. waiting on i/o), >>> then using a thread is great. Otherwise, if the worker needs to do a bunch >>> of heavy work before the data is ready to display (e.g. it's the output of >>> a difficult analysis), put that work on a different core. >>> >>> Only matplotlib is as good at plotting as matplotlib; it's the best >>> python plotting library. If that's your minimum bar, I agree that you'll be >>> writing everything yourself from scratch. PyQtGraph is built for >>> performance, though, so you're going to need the best of both libraries if >>> that's your use-case. >>> >>> Good luck, >>> - Martin >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "pyqtgraph" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/pyqtgraph/CAD_p8v3668bj82zYmWi%2Bns_DfHK4JHAbwvb8oSmitU-RkArFpA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/pyqtgraph/CAD_p8v3668bj82zYmWi%2Bns_DfHK4JHAbwvb8oSmitU-RkArFpA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "pyqtgraph" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/pyqtgraph/CACwj3kUb7toyty5sAE4vP6mV3YsRxQ%2BjGMxRh-hecLFvy9w2DA%40mail.gmail.com >> <https://groups.google.com/d/msgid/pyqtgraph/CACwj3kUb7toyty5sAE4vP6mV3YsRxQ%2BjGMxRh-hecLFvy9w2DA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "pyqtgraph" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/pyqtgraph/CA%2BnduTFi5wBSuqB9_g43HC_yYhwVSunEf7qz4Km4fjUMLnLbXw%40mail.gmail.com > <https://groups.google.com/d/msgid/pyqtgraph/CA%2BnduTFi5wBSuqB9_g43HC_yYhwVSunEf7qz4Km4fjUMLnLbXw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "pyqtgraph" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/CAKhctBFBTGF5%3DUmYiPD9trwZDh-mPkykjG4ZpJP1%3D1Rk8MFzNw%40mail.gmail.com.
