On 20/04/18 20:13, C Hamilton wrote: > Thanks Ben, > > I will give that a try. I am finding that the Interpolation algorithm is > taking forever on a small data set.
I am not surprised. It iterates over every individual point in your dataset, then uses slow floating point computations to try and find other data points within a threshold distance (probably involving square root computations, which are among the most inefficient things to compute). Then it needs to compute a spatially weighted interpolator of some kind, discard the result and start all over at the next data point. By comparison, r.fill.stats pre-computes a set of (less accurate, but who cares in this case of noisy and dense data?) spatial weights in a fixed interpolation window and then just "slides" that over the rasterized data using nothing but (fast) basic arithmetic operations. > > Doesn't the Postgres PDAL extension handle point data like this? Would > there be a benefit of putting the data in PostGIS? I doubt it. That would give you the overhead of a DBMS in addition to the sluggishness of common interpolators. If you want to experiment more with algorithms that interpolate from vector points, then you should make sure to build a spatial index. Whether that happens inside a database or in a file system should make little difference for the effective running time. But fastest option is to read the data sequentially from a file, rasterizing them, then running r.fill.stats on the rasterized result. At least that's what my experiments with that type of data amounted to. That's a "quick and dirty" interpolation, but "dirty" is also an adequate description of the nature of magnetometry data, so it's a good fit. Cheers, Ben > > Thanks, > > Calvin > > On Fri, Apr 20, 2018 at 1:10 PM, Benjamin Ducke <[email protected] > <mailto:[email protected]>> wrote: > > Hi Calvin, > > On 20/04/18 18:46, Richard McDonnell wrote: > > Calvin, > > > > Under Processing Toolbox, Type interpolation into the search bar, the > > tool you want can be found there. > > None of the typical GIS interpolation tools will be > of much help here, since they have all been optimized > to work with relatively sparse data. They will be > too inefficient for high-resolution geomagnetic data. > > You data is already dense, with some gaps. So you need > something that is fast and efficient for small > interpolation radii, and will not produce too many > artefacts. > > If you can use the GRASS7 plug-in from within QGIS, > than this is a good processing path: > > 1. Rasterize your data points, using a cell size > small enough so that you don't have too much information > loss (i.e. not too many occurrences of more than one data > point per cell). This might take a little trial and error > to find good settings. You should end up with a raster > where you have gaps of max. 1-3 empty cells between cells > with measurement data. > > Now you need to fill these relatively small gaps. > > 2. Use r.fill.stats from current GRASS 7.4: > > https://grass.osgeo.org/grass74/manuals/r.fill.stats.html > <https://grass.osgeo.org/grass74/manuals/r.fill.stats.html> > > ... because that has been designed exactly for the > purpose. Set the interpolation radius as small as > possible, so that it just manages to fill the gaps. > With a small radius, r.fill.stats is very fast and > memory efficient, even on very large raster datasets. > > Note that r.fill.stats is part of GRASS 7 only since > version 7.4. If your QGIS ships with an earlier > version, then you might have to update your GRASS 7 > binaries manually. > > Best, > > Ben > > > Regards, > > > > Richard. > > > > On 20/04/2018 16:42, C Hamilton wrote: > >> I don't know how many of you have worked with archaeological > >> magnetormetry data, but I have been asked to help with magnetometry > >> data of the ancient Mound Builders in North America. I am looking for > >> some suggestion on how to best turn the irregular magnetic data points > >> into an image. > >> > >> I see an Interpolation plugin that perhaps is the solution, but I also > >> see that it is not part of QGIS 3. Is this because it has not been > >> ported yet, or is not going to be porter, or is there a better > solution? > >> > >> I don't know if any of the Lidar tools would work for this or not. > >> LASTools is a commercial product and the locked version only works > >> with small data sets and adds noise unless you purchase a license. At > >> least that is how it was the last time I used it. > >> > >> Thanks for any suggestions. > >> > >> Calvin > >> > >> > >> _______________________________________________ > >> Qgis-user mailing list > >> [email protected] <mailto:[email protected]> > >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > <https://lists.osgeo.org/mailman/listinfo/qgis-user> > >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > <https://lists.osgeo.org/mailman/listinfo/qgis-user> > > > > -- > > *_Richard McDonnell_* > > *GIS Specialist PgD GIS AssocSCSI* > > *OPW FRM Data Management* > > *52 Stephens Green, Dublin 2.* > > TEL: 01 6476543 > > > > <http://www.opw.ie/> > > > > > > > > Email Disclaimer: http://www.opw.ie/en/disclaimer/ > <http://www.opw.ie/en/disclaimer/> > > > > > > _______________________________________________ > > Qgis-user mailing list > > [email protected] <mailto:[email protected]> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > <https://lists.osgeo.org/mailman/listinfo/qgis-user> > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > <https://lists.osgeo.org/mailman/listinfo/qgis-user> > > > > > > -- > Dr. Benjamin Ducke > Deutsches Archäologisches Institut (DAI) > Zentrale Berlin, IT-Referat > * Projekt "Stunde Null" * > _______________________________________________ > Qgis-user mailing list > [email protected] <mailto:[email protected]> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > <https://lists.osgeo.org/mailman/listinfo/qgis-user> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > <https://lists.osgeo.org/mailman/listinfo/qgis-user> > > -- Dr. Benjamin Ducke Deutsches Archäologisches Institut (DAI) Zentrale Berlin, IT-Referat * Projekt "Stunde Null" * _______________________________________________ Qgis-user mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-user Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
