Re: [GRASS-user] Estimating import time for large data file
On Fri, 29 May 2020, Moritz Lennert wrote: grass --help is your friend ;-) Moritz, Good to know; I never tried this. grass -c epsg:4326 GISDBASE/LOCATION replace GISDBASE and LOCATION with the respective names -c can also be used with a file: -c /path/to/georeferenced/file Thanks for the reminder. Another, untested, path for possibly speeding up import: use ogr2ogr with the -simplify option to first simplify the geometry, if that is possible in your case. Would be interesting to benchmark v.in.ogr alone against ogr2ogr -simply + v.in.ogr. I'll consider doing this test. When I run a shell script, or the command for a long running process, I use the 'time' tool with it. Regards, Rich ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Estimating import time for large data file
On 29/05/20 14:37, Rich Shepard wrote: On Fri, 29 May 2020, Moritz Lennert wrote: Importing such a large file can take a lot of time because of the cleaning in order to get it into GRASS GIS' topological format. Moritz, I assumed it would taka a while, but it seemed to be stuck. Perhaps because that polygon was large, complex, and needed a lot of cleaning. I think it really depends on the specific file in terms of geometry complexity and cleanliness, so difficult to pre-determine. I regularly launch large import jobs overnight. Then that's what I'll do tonight: I'll start it running in screen before I log off and let it do it's thing for as long as it wants. You should see an information about its progress in the different cleaning steps (note that some cleaning is repeated). I was using the GUI because I needed to create a new location and have forgotten how to create a new location and mapset from the command line. grass --help is your friend ;-) grass -c epsg:4326 GISDBASE/LOCATION replace GISDBASE and LOCATION with the respective names -c can also be used with a file: -c /path/to/georeferenced/file Another, untested, path for possibly speeding up import: use ogr2ogr with the -simplify option to first simplify the geometry, if that is possible in your case. Would be interesting to benchmark v.in.ogr alone against ogr2ogr -simply + v.in.ogr. Moritz ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Estimating import time for large data file
On Fri, 29 May 2020, Moritz Lennert wrote: Importing such a large file can take a lot of time because of the cleaning in order to get it into GRASS GIS' topological format. Moritz, I assumed it would taka a while, but it seemed to be stuck. Perhaps because that polygon was large, complex, and needed a lot of cleaning. I think it really depends on the specific file in terms of geometry complexity and cleanliness, so difficult to pre-determine. I regularly launch large import jobs overnight. Then that's what I'll do tonight: I'll start it running in screen before I log off and let it do it's thing for as long as it wants. You should see an information about its progress in the different cleaning steps (note that some cleaning is repeated). I was using the GUI because I needed to create a new location and have forgotten how to create a new location and mapset from the command line. If you are sure you do not want to do any polygon cleaning during import, you can use the -c flag. But you will have to handle the result with care. Actually, I much prefer to let GRASS clean as it goes. Thanks very much for the knowledge. Stay well, Rich ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
[GRASS-user] r.surf.area
Hi GRASS members Suppose you compute the "r.surf.area"(1) to some raster file and find that the 3d area is smaller than your 2d area? What could be the problem? Note: The exercise was done with a 5 m cell size raster file, with 37 km2 watershed with a slope of 0,0341 m/m, within 7.8 version of GRASS GIS 7.8 console in a 3.12.3 QGIS [with this method the difference is higher (12 ha, comparing with ArcGIS)] and outside QGIS, in GRASS GIS GUI Thanks in advance (1) https://grass.osgeo.org/grass78/manuals/r.surf.area.html Cumprimentos, *Valter Albino -* Geógrafo Físico, M.Sc. Modelação H / Riscos ambientais / OT www.valteralbino.wixsite.com/hydrodynamics ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Estimating import time for large data file
On 28/05/20 17:25, Rich Shepard wrote: I'm trying to import (using v.in.ogr) a rather large file (1.22G *.shp and 4.5M *.dbf). I see a lot of system disk reads, low cpu usage, but after a half-hour it appears to have stopped processing. Importing such a large file can take a lot of time because of the cleaning in order to get it into GRASS GIS' topological format. I'm running Slackware-14.2/x86_64 on a desktop with an AMD Ryzen7 2700 CPU (8 cores/16 threads) and 32G DDR4 memory. GRASS-7.9.dev is configured to enable largefile and use openmp. 1. Can I do more to facilitate import of this large, statewide wetlands data set? 2. Is there a way to estimate the time import would take? If so, I'll start it using screen and let it run in the background, overnight if necessary. I think it really depends on the specific file in terms of geometry complexity and cleanliness, so difficult to pre-determine. I regularly launch large import jobs overnight. 3. Is there a realtime progress monitor that informs me grass is chewing on the import or stuck somewhere? The GUI layer manager shows the map needs cleaning for many (most? all?) polygons when it eventually is imported, but that's to be addressed later. You should see an information about its progress in the different cleaning steps (note that some cleaning is repeated). If you are sure you do not want to do any polygon cleaning during import, you can use the -c flag. But you will have to handle the result with care. Moritz ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.what.rast - solved
On 29/05/20 10:09, Micha Silver wrote: On 29/05/2020 9:42, Uwe Fischer wrote: Hello list, thanks to Micha Silver and Helmut Kudrnovsky, the problem ist solved. I did not set g.region prior to using v.what.rast. Seems to be a typical non-Grass-Professional mistake. Now it works fine. But to be honest, I do not really understand what it is good for. When the raster image and the points to be updated share the same Grass location and CRS, why do I have to set a region in addition? Glad you got it worked out. The only thing I would add to Stefan's explanation is a pointer to the GRASS wiki on region settings: https://grasswiki.osgeo.org/wiki/Computational_region Although I am a huge fan and promotor of the computational region concept, I actually agree with Uwe that it is debatable whether in the specific cas of v.what.rast this should be the default behavior. It is a bit counter-intuitive. In any case, it should be clearly and explicitly documented in the man page. Uwe, would you mind posting an issue on this in github [1] ? Moritz [1] https://github.com/OSGeo/grass/issues ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.what.rast - solved
On 29/05/2020 9:42, Uwe Fischer wrote: Hello list, thanks to Micha Silver and Helmut Kudrnovsky, the problem ist solved. I did not set g.region prior to using v.what.rast. Seems to be a typical non-Grass-Professional mistake. Now it works fine. But to be honest, I do not really understand what it is good for. When the raster image and the points to be updated share the same Grass location and CRS, why do I have to set a region in addition? Glad you got it worked out. The only thing I would add to Stefan's explanation is a pointer to the GRASS wiki on region settings: https://grasswiki.osgeo.org/wiki/Computational_region Regards, Micha Best regards and thanks again, Uwe ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user -- Micha Silver Ben Gurion Univ. Sde Boker, Remote Sensing Lab cell: +972-523-665918 https://orcid.org/-0002-1128-1325 ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.what.rast - solved
Hei Uwe, It is unfortunately not obvious from the very beginning, but actually one of the nice features of GRASS GIS. No need to "clip" any data. You can easily operate on any subset of your data and all data will be cleanly aligned even if the extent of on layer changes Once you are used to it you will enjoy it. Cheers Stefan From: grass-user On Behalf Of Uwe Fischer Sent: fredag 29. mai 2020 08:42 To: grass-user@lists.osgeo.org Subject: [GRASS-user] v.what.rast - solved Hello list, thanks to Micha Silver and Helmut Kudrnovsky, the problem ist solved. I did not set g.region prior to using v.what.rast. Seems to be a typical non-Grass-Professional mistake. Now it works fine. But to be honest, I do not really understand what it is good for. When the raster image and the points to be updated share the same Grass location and CRS, why do I have to set a region in addition? Best regards and thanks again, Uwe ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
