Re: [GRASS-user] Large Grass Raster Notes
On Sun, Jan 18, 2015 at 1:00 AM, Jeshua Lacock jes...@3dtopo.com wrote: Greetings, In the event my experience working with rather large GRASS rasters may be useful, I thought I would share it. The following notes were compiled running the GRASS 7.0.0beta4-141230 package on Mac OS X 10.10.1. The system is a 6-core Xeon with 32GB of RAM, running in full 64-bit mode. 1. Using an r.external virtual mosaic as the input proved impossible trying to run r.resample on it on trying to produce a nearly teracell raster (I ended up breaking sub-tiles). I closely followed the instructions on the Wiki. 2. On smaller sub-tiles of said image, r.resample was insanely slow even with the external imagery. You may check/benchmark the other resampling modules, some of them may be more modern (ideally faster): http://grass.osgeo.org/grass70/manuals/keywords.html#resample In case you try, please share the outcome. cheers, Markus ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] SQLITE db locking problem
On Sun, Jan 18, 2015 at 3:23 PM, Mark Wynter m...@dimensionaledge.com wrote: Hi everyone I’m having issues with SQLITE as the db driver - I encounter this BUSY warning. Only occurs when patching a particular map, in this case “temp_nbt_cleaned which is the largest of the datasets being patched. If I work with smaller datasets, it works fine. Pre-patching, SQLITE DB size is only 170MB. I’ve come across various threads reporting something similar with db locking. But couldn’t find a resolution. I’m running grass70 on Centos65, hosted on AWS.. SQLite is version 3.6.23.1 I thought the official SQLite version for Centos65 (and Centos66) is 3.6.20. Could there be a conflict between the official version and your version 3.6.23.1? This is the error message. GRASS 7.0.0svn (nodeclean):/var/tmp v.patch -e input=temp_t_cleaned,temp_b_cleaned,temp_nbt_cleaned out=temp_patched Patching vector map temp_t_cleaned... Patching vector map temp_b_cleaned... Patching vector map temp_nbt_cleaned... WARNING: Busy SQLITE db, already waiting for 10 seconds… WARNING: Busy SQLITE db, already waiting for 20 seconds… etc etc I can not reproduce the problem on Fedora 20 with SQLite 3.8.7.4 and Scientific Linux 6.6 with SQLite 3.6.20. Are you using default db connection settings for SQLite or is the GRASS SQLite database in a non-standard location? Markus M I encounter no problems patching the maps, including “temp_nbt_cleaned if I use PG db driver, or DBF driver. building topology for vector map temp_patched@PERMANENT... Registering primitives... 1860504 primitives registered 9027638 vertices registered Building areas... 100% 0 areas built 0 isles built Attaching islands... Attaching centroids... 100% Number of nodes: 1457611 Number of primitives: 1860504 Number of points: 0 Number of lines: 1860504 Number of boundaries: 0 Number of centroids: 0 Number of areas: 0 Number of isles: 0 Intersections at borders will have to be snapped Lines common between files will have to be edited The header information also may have to be edited v.patch complete. 3 vector maps patched I’m more than comfortable using PG in the backend, but I find v.out.postgis too slow when exporting the resultant dataset back to postgis (possibly because its simultaneously trying to read and write to the same db). I haven’t tested whether v.out.ogr quicker than v.out.postgis. Any assistance resolving the SQLite issue would be greatly appreciated. Kind regards Mark ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] grass-user Digest, Vol 105, Issue 36
-0001.html -- Message: 2 Date: Tue, 20 Jan 2015 17:35:48 +0100 From: Markus Neteler nete...@osgeo.org To: Jeshua Lacock jes...@3dtopo.com Cc: GRASS user list grass-user@lists.osgeo.org Subject: Re: [GRASS-user] Large Grass Raster Notes Message-ID: CALFmHhuKm1= xmcvdkova2vgznwvc_zax7plh7gxsqebbnvx...@mail.gmail.com Content-Type: text/plain; charset=UTF-8 On Sun, Jan 18, 2015 at 1:00 AM, Jeshua Lacock jes...@3dtopo.com wrote: Greetings, In the event my experience working with rather large GRASS rasters may be useful, I thought I would share it. The following notes were compiled running the GRASS 7.0.0beta4-141230 package on Mac OS X 10.10.1. The system is a 6-core Xeon with 32GB of RAM, running in full 64-bit mode. 1. Using an r.external virtual mosaic as the input proved impossible trying to run r.resample on it on trying to produce a nearly teracell raster (I ended up breaking sub-tiles). I closely followed the instructions on the Wiki. 2. On smaller sub-tiles of said image, r.resample was insanely slow even with the external imagery. You may check/benchmark the other resampling modules, some of them may be more modern (ideally faster): http://grass.osgeo.org/grass70/manuals/keywords.html#resample In case you try, please share the outcome. cheers, Markus -- Message: 3 Date: Tue, 20 Jan 2015 14:04:05 -0500 From: Vaclav Petras wenzesl...@gmail.com To: Jeshua Lacock jes...@3dtopo.com Cc: GRASS user list grass-user@lists.osgeo.org Subject: Re: [GRASS-user] Large Grass Raster Notes Message-ID: cabo5uvsfknmc3o+-g+kdk-yj4-29yhblzvrytfeqrol9vgp...@mail.gmail.com Content-Type: text/plain; charset=utf-8 On Sun, Jan 18, 2015 at 1:03 PM, Jeshua Lacock jes...@3dtopo.com wrote: Actually, I misspoke, the wildcard will not work with long lists either as it will crash with some kind argument too long message as well. This is strange, the wild card is interpreted inside the module, so the argument list shouldn't be too long. If you don't quite the pattern the shell can do the expansion but this is not what you want. Correct: g.remove ... pattern=landsat_* Incorrect: g.remove ... pattern=landsat_* The later will transform the command line to: g.remove ... pattern=landsat_01 landsat_02 landsat_03 landsat_04 ... in case you are in directory with files or directories named landsat_01, landsat_02, landsat_03, ... What is the message you get? And what is the command return code (do echo $? to find out)? -- next part -- An HTML attachment was scrubbed... URL: http://lists.osgeo.org/pipermail/grass-user/attachments/20150120/6678f2d5/attachment-0001.html -- ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user End of grass-user Digest, Vol 105, Issue 36 *** -- Desire can be of greater import than utility; experience more important than closure. ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
On Sun, Jan 18, 2015 at 1:03 PM, Jeshua Lacock jes...@3dtopo.com wrote: Actually, I misspoke, the wildcard will not work with long lists either as it will crash with some kind argument too long message as well. This is strange, the wild card is interpreted inside the module, so the argument list shouldn't be too long. If you don't quite the pattern the shell can do the expansion but this is not what you want. Correct: g.remove ... pattern=landsat_* Incorrect: g.remove ... pattern=landsat_* The later will transform the command line to: g.remove ... pattern=landsat_01 landsat_02 landsat_03 landsat_04 ... in case you are in directory with files or directories named landsat_01, landsat_02, landsat_03, ... What is the message you get? And what is the command return code (do echo $? to find out)? ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
