[GRASS-user] Recommendations for v.surf.rst parameters for huge region
Hello list, I am trying to interpolate raster values for a single massive dataset that represents dozens of multibeam bathymetry surveys over a few decades. The region is pretty big: all of Eastern Canada; at 100m resolution there are ~ 464M cells. The raster data has a wide range of completeness; in some areas, there is near 100% coverage - these areas were surveyed with modern sonars and 100% overlap between survey lines. In other areas, the data is very sparse, with ~1km or more between tracklines. These areas would represent surveys from the 70's to 90's using singlebeam echosounders. Firstly, would v.surf.rst be the best module for a massive interpolation job like this? If not, could you recommend what would be the optimal method? If v.surf.rst is the right module to use, I was wondering if anyone could help with what parameters to use for an area this size, at least as a starting point. I have read the manual several times, but I still don't have a good intuition for how parameters like npmin, segmax, dmin, dmax, smooth all work together. At the moment, I have a script written that accepts a user-supplied number of random positions all over the input raster. For each random point, I obtain the east and north coordinates with v.to.db, feed these to g.region, and grow the region around the point in all four cardinal directions by some value like 10,000m to create an analysis window around each point. I create a polygon vector of this region with v.in.region, and use this polygon to clip my vectorized raster bathymetry with v.clip, and then do v.surf.rst on this clipped vector. I have no other way of knowing what v.surf.rst parameters to use other than trial and error, so I have a 4-level nested 'for' loop written to basically traverse through all permutations of the parameters within the ranges I chosen. So for example, I am exploring all combinations of parameters within these ranges: Tension: 10, 20, 40, 80, 160 npmin: 100, 200, 400, 800, 1000 smooth: 1, 2, 4, 8, 16, 32 With 7 random points selected around the full region, this script will produce a few hundred maps per point, and quite a long time to run. And even more time to look at the results. I know this isn't the best approach. I am looking for help to find some workable set of parameters to use for the entire dataset from other users who have more experience using this module. Thanks, -- Eric Patton Marine Geoscience Technologist Geological Survey of Canada (Atlantic) Natural Resources Canada Bedford Institute of Oceanography 1 Challenger Drive, Dartmouth NS, Canada ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.random segfault
On 20/10/16 12:10PM, Maris Nartiss wrote: > > Hello Eric, > it is hard to say if I hit your bug or another one, but I managed to > reproduce crash and already reported it to the bug tracker: > https://github.com/OSGeo/grass/issues/1024 > > The problem seems to be with specifying parameter "restrict". If you > really need it, just wait for a fix as it should be quite fast (famous > last words). > > Māris. Thanks, Maris, much appreciated. I'll keep an eye on the status of it. ~ Eric. ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] r.param.scale issue
Your remarks Māris are very interesting. I am actually having trouble with my CPU (AMD Ryzen 7 3800X) and debian stable: I should try to test several debian backports to have the full temperature sensors support (not yet supported with 4.19) but am not especially at ease with these quite dodgy manipulation... Thank you for your support, Vincent. Le vendredi 16 octobre 2020 à 11:39 +0300, Maris Nartiss a écrit : > Valgrind leak check is needed only for developers as it tracks memory > consumption not its errors (although in this case it allowed to > notice > OpenMP presence by shear luck). As your valgrind run is clean, this > is > a dead end. > > On my system I also have a OpenMP and pthreads configuration. > Unfortunately tracking OpenMP issues is really hard as it might need > precise thread/process count and timing to reproduce the bug :-( > Besides it could also be not our bug. > > Please do test carefully your hardware to rule out bad thermal > management / faulty RAM as your hardware is brand new and thus > manufacturing defects can not be ruled out yet. > > Also state your CPU and GCC. > > I did not see any problems with Intel(R) Xeon(R) CPU E3-1241 v3 and > gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04). > > Māris. > > ceturtd., 2020. g. 15. okt., plkst. 11:13 — lietotājs Vincent Bain > () rakstīja: > > Thanks Māris for your advice, > > > > it sounds dreadful to me to have to deal with these hardware > > issues... > > I installed valgrind and ran r.param.scale. > > > > Attached are the outputs of the command, first run without any > > argument, then with --leak-check=full option. Of course it's nearly > > "doubledutch" to me! > > > > Vincent. ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] r.param.scale issue
Vincent, On Thu, Oct 15, 2020 at 6:39 PM Vincent Bain wrote: > > Markus, Māris, and others who feel concerned, > > following on with this post, I am wondering if either something is > wrong with my openMP install (then it's not the place to bother anyone > with it) or if there may be something wrong with r.param.scale, and > possibly more generally with the way GRASS deals with openMP. > > Besides GRASS GIS I use OpenMP directives from within Clawpack fortran > code, safely. Note that r.param.scale itself doesn't use OpenMP. It it pulled in from GMATHLIB: # find OpenMP support: ag OMPLIB | grep Makefile | cut -d':' -f1 | sort lib/gmath/Makefile lib/gmath/Makefile lib/gmath/test/Makefile lib/gpde/Makefile lib/gpde/Makefile lib/gpde/test/Makefile lib/rst/interp_float/Makefile lib/temporal/lib/Makefile raster/r.proj/Makefile raster/r.series.accumulate/Makefile raster/r.sim/r.sim.sediment/Makefile raster/r.sim/r.sim.water/Makefile raster/r.sim/simlib/Makefile raster/r.sun/Makefile vector/v.surf.rst/Makefile Hence, I believe that GMATHLIB is the test candidate here. Hence, Markus ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.random segfault
Hello Eric, it is hard to say if I hit your bug or another one, but I managed to reproduce crash and already reported it to the bug tracker: https://github.com/OSGeo/grass/issues/1024 The problem seems to be with specifying parameter "restrict". If you really need it, just wait for a fix as it should be quite fast (famous last words). Māris. ceturtd., 2020. g. 15. okt., plkst. 22:11 — lietotājs Eric Patton () rakstīja: > > Hi guys, > > I have encountered an error using v.random. It segfaults whenever I try to > create some random points. > > > gdb v.random > (gdb) run output=test_points npoints=6 restrict=test column=Depth --v --o > Starting program: /usr/local/grass78/bin/v.random output=test_points > npoints=6 restrict=test column=Depth --v --o > warning: the debug information found in "/lib64/ld-2.27.so" does not match > "/lib64/ld-linux-x86-64.so.2" (CRC mismatch). > > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > WARNING: Vector map already exists and will be overwritten > WARNING: Table linked to vector map does not > exist > Using native format > Generating points... >0% > Program received signal SIGSEGV, Segmentation fault. > 0x8c21 in main () > (gdb) bt > #0 0x8c21 in main () > > Is there anything else I can try? І'm not sure what's going on with the > debugging symbols, but it shouldn't be causing the program to crash, right? > > My system: > > g.version -re > GRASS 7.8.4 (2020) > libgis revision: 2020-10-13T18:27:00+00:00 > libgis date: 2020-10-12T13:00:00-03:00 > PROJ: 7.0.0 > GDAL/OGR: 3.0.4 > GEOS: 3.8.0 > SQLite: 3.22.0 > > uname -a > Linux L-102377 5.4.0-51-generic #56~18.04.1-Ubuntu SMP Tue Oct 6 09:47:18 UTC > 2020 x86_64 x86_64 x86_64 GNU/Linux > > Thanks, > > -- > Eric Patton > Marine Geoscience Technologist > Geological Survey of Canada (Atlantic) > Natural Resources Canada > Bedford Institute of Oceanography > 1 Challenger Drive, Dartmouth NS, Canada > > ___ > grass-user mailing list > grass-user@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] r.param.scale issue
Valgrind leak check is needed only for developers as it tracks memory consumption not its errors (although in this case it allowed to notice OpenMP presence by shear luck). As your valgrind run is clean, this is a dead end. On my system I also have a OpenMP and pthreads configuration. Unfortunately tracking OpenMP issues is really hard as it might need precise thread/process count and timing to reproduce the bug :-( Besides it could also be not our bug. Please do test carefully your hardware to rule out bad thermal management / faulty RAM as your hardware is brand new and thus manufacturing defects can not be ruled out yet. Also state your CPU and GCC. I did not see any problems with Intel(R) Xeon(R) CPU E3-1241 v3 and gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04). Māris. ceturtd., 2020. g. 15. okt., plkst. 11:13 — lietotājs Vincent Bain () rakstīja: > > Thanks Māris for your advice, > > it sounds dreadful to me to have to deal with these hardware issues... > I installed valgrind and ran r.param.scale. > > Attached are the outputs of the command, first run without any > argument, then with --leak-check=full option. Of course it's nearly > "doubledutch" to me! > > Vincent. ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
