> On May 26, 2021, at 11:37 AM, Ognyan Moore <[email protected]> wrote: > > Completely missed your original post with the geospatial data bit, my > apologies!
No worries! > Side note, if there is some standard downsampling method involving geospatial > data that you think would be useful to have in the library, please let me > know. I’m thinking that might be more specialized than would be appropriate. What I use when I need to “downsample” (or, really just resample, whether it is a downsample or not just depends on the parameters used) is the pyresample library, specifically in my case the bilinear interpolation for swath data functions (https://pyresample.readthedocs.io/en/latest/swath.html#pyresample-bilinear <https://pyresample.readthedocs.io/en/latest/swath.html#pyresample-bilinear>). The process generally takes several seconds for me, depending on the grid size I choose for the result, so I don’t know that it would integrate well into a “live” downsampling algorithm. > > For a sanity check, I would suggest running examples/ScatterPlotSpeedTest.py, > and modify the parameters there (it uses a parameter tree so you can do this > in the GUI) to most match your use-case (number of points and so on). There > is a zooming and panning mode you can adjust. Throwing 150k points at it > definitely took a toll, it's going at < 1fps. That sounds about right, maybe a bit slower on my machine (which is a 2013 model, so if it works well here, it should work well about anywhere). Functional, but a bit painful when trying to pan/zoom to a specific feature. > We also recently filed an issue that would significantly speed up scatter > plot code here: https://bugreports.qt.io/browse/PYSIDE-1572 > <https://bugreports.qt.io/browse/PYSIDE-1572> If you think this issue would > assist you, I would encourage you to chime in there encouraging the PySide > developers to adopt this feature. Done. Looks interesting, although I might think that using pandas data frames or xarray datasets (both of which use numpy ndarrays under the hood) might be a bit simpler than “simple” multi-dimensional numpy arrays due to the named column features of those libraries making it easy to separate out x, y, brushes, symbols, etc vs a simple multi-dimensional numpy array. Just my 2¢ there though, not having looked at the implementation maybe just using straight ndarrays would be a better option for all I know :-) BTW, might I say that PyQtGraph is an excellent library? I love how it is built upon the Qt classes, just adding features - it makes it so easy for me to hack on it when needed, or bring in additional “base” Qt functionality whenever it doesn’t do exactly what I want it to, rather than being forced to try to figure out how to “force” PyQtGraph itself to bend to my will. Makes it easy to use the best of both worlds - PyQtGraph, and Qt itself. Thanks for all the hard work! --- Israel Brewster Software Engineer Alaska Volcano Observatory Geophysical Institute - UAF 2156 Koyukuk Drive Fairbanks AK 99775-7320 Work: 907-474-5172 cell: 907-328-9145 > > > > On Wed, May 26, 2021 at 12:28 PM Israel Brewster <[email protected] > <mailto:[email protected]>> wrote: >> On May 26, 2021, at 11:12 AM, Ognyan Moore <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Israel, >> >> I think Martin was pointing out that pyqtgraph offers downsampling methods >> <https://pyqtgraph.readthedocs.io/en/latest/graphicsItems/plotdataitem.html#pyqtgraph.PlotDataItem.setDownsampling>which >> adjust sampling based on the pixel size of your display. The "peak" >> downsample method draws a vertical line between the lowest and highest value >> within the range of values that would normally fit onto one pixel. > > Sure, and for a basic x-y plot, like a time series, where you have a row of > dots along the x-axis, this would work fine. For geospatial data like I have, > this would be a total mess - there is no line to connect the dots. Rather, > the dots fill the field, and if you remove any of them, then the field will > not be filled at any zoom level. Trying to bridge the gap with a line > wouldn’t work > >> The problem with this method is that it assumes all data to be uniformly >> spaced in the x-axis, which of course may not be the case. > > Right, and for geospatial data this is not only not the case, but any given X > will have hundreds of data points that cover that X value. > > Point being that while I’m sure the downsampling methods are good and very > useful in general, for my geospatial data it simply doesn’t work. > --- > Israel Brewster > Software Engineer > Alaska Volcano Observatory > Geophysical Institute - UAF > 2156 Koyukuk Drive > Fairbanks AK 99775-7320 > Work: 907-474-5172 > cell: 907-328-9145 > >> The nice thing about this downsampling method is that it will re-sample >> for you as your zoom level changes. >> >> Ogi >> >> On Wed, May 26, 2021 at 11:05 AM Israel Brewster <[email protected] >> <mailto:[email protected]>> wrote: >> Thanks for the suggestions! >> >>> On May 25, 2021, at 6:24 PM, Martin Chase <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hey Israel, >>> >>> A couple of thoughts: >>> >>> Downsampling should be able to make things faster. Look at the >>> "optimization" section in >>> https://pyqtgraph.readthedocs.io/en/latest/graphicsItems/plotdataitem.html >>> <https://pyqtgraph.readthedocs.io/en/latest/graphicsItems/plotdataitem.html> >> I don’t think downsampling is appropriate for my dataset. Since the points >> are arranged spatially, any *simple* downsampling would result in holes in >> the data coverage. I could, of course *resample* to a larger grid size, but >> that would either loose resolution when zoomed in - which we can’t afford, >> given that the features we are looking for often are only one or two grid >> cells in size - or require periodic re-resampling depending on the zoom, >> which doubtless would be more computationally expensive than simply >> re-drawing the dataset, not to mention complicated to implement. >> >>> QGraphicsItems can be cached, which might help with panning. Docs >>> https://doc.qt.io/qt-5/qgraphicsitem.html#CacheMode-enum >>> <https://doc.qt.io/qt-5/qgraphicsitem.html#CacheMode-enum> >> It seems like this should work, based on the description - in fact, it >> sounds like it should do essentially what I was suggesting, rendering once >> to a offscreen pixmap and then simply using that rendered version. >> Unfortunately, I spent some time playing around with the setCacheMode >> function as well as the QPixmapCache.setCacheLimit function that the >> setCacheMode function mentioned, with no noticeable effect. Which suggests >> to me that I might be doing something wrong. >> >> That said, I also spent a bit more time pursuing my original thought of >> rendering the plot to a single pixmap, then just displaying that single >> image. As it turned out, once I took a step back from PyQtGraph, >> implementing this solution using the base Qt classes/functions turned out to >> be surprisingly simple - simplified, no doubt, by the fact that I was >> already supplying the symbol brushes to the plot command as a list of >> QPainterPaths. So I was able to simply take the x and y coordinates, and use >> a QPainter to directly paint the QPainterPaths into a QPixmap, which could >> then be added to my plot as a single item. I *did* have to reduce the >> quality of the rendering a bit to fit things into memory - so if you zoom in >> far enough, the individual data cells have rough edges - but other than >> that, it seems to work fine - and performance goes from painfully sluggish >> on my old machine to silky smooth. >> >> There may be other issues with this approach, but so far I haven’t run into >> any, at least with my specific application. More testing remains. >> >> Thanks again! >> --- >> Israel Brewster >> Software Engineer >> Alaska Volcano Observatory >> Geophysical Institute - UAF >> 2156 Koyukuk Drive >> Fairbanks AK 99775-7320 >> Work: 907-474-5172 >> cell: 907-328-9145 >> >>> >>> Others may have more ideas? Let us know if any of that helps! >>> >>> - Martin >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "pyqtgraph" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] >>> <mailto:[email protected]>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/pyqtgraph/CAD_p8v0iE_jYahUx7o2Oz-u%3DEor8ELyEJK1z53i-3YNvtK-dYw%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/pyqtgraph/CAD_p8v0iE_jYahUx7o2Oz-u%3DEor8ELyEJK1z53i-3YNvtK-dYw%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "pyqtgraph" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/pyqtgraph/233ABF09-0EE0-4AC9-90AD-6BB2586D2AD7%40alaska.edu >> >> <https://groups.google.com/d/msgid/pyqtgraph/233ABF09-0EE0-4AC9-90AD-6BB2586D2AD7%40alaska.edu?utm_medium=email&utm_source=footer>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "pyqtgraph" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/pyqtgraph/CA%2BnduTGJACsL9ezspJWhBSjtUuXc2Q_LdB-7EeYoH%2B5fKraGOQ%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/pyqtgraph/CA%2BnduTGJACsL9ezspJWhBSjtUuXc2Q_LdB-7EeYoH%2B5fKraGOQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "pyqtgraph" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/pyqtgraph/0726F04E-3304-47CB-AD98-7CCB0ED755E6%40alaska.edu > > <https://groups.google.com/d/msgid/pyqtgraph/0726F04E-3304-47CB-AD98-7CCB0ED755E6%40alaska.edu?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "pyqtgraph" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/pyqtgraph/CA%2BnduTHjUZ1M1ikx%3D6Vo3nMpCKA0rj4SNfX2HH5nB%3DzyK6toXA%40mail.gmail.com > > <https://groups.google.com/d/msgid/pyqtgraph/CA%2BnduTHjUZ1M1ikx%3D6Vo3nMpCKA0rj4SNfX2HH5nB%3DzyK6toXA%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "pyqtgraph" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/CF71FFE9-707F-428E-8C02-2A09BB226353%40alaska.edu.
