Re: [GRASS-user] Large Grass Raster Notes
Jeshua, On Fri, Jan 23, 2015 at 3:22 AM, Vaclav Petras wenzesl...@gmail.com wrote: On Thu, Jan 22, 2015 at 9:10 PM, Jeshua Lacock jes...@3dtopo.com wrote: Since that is indeed the case, it would be super nice to add a pattern flag to r.patch and v.patch. It would. I'm not sure if there is a ticket for it. Please open an enhancement ticket for this in trac: http://trac.osgeo.org/grass/ - New ticket (requires login) thanks Markus Note that some other modules accept file with names which gives different possibilities than pattern option, e.g. usage with temporal framework. ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
On Jan 20, 2015, at 12:04 PM, Vaclav Petras wenzesl...@gmail.com wrote: Correct: g.remove ... pattern=landsat_* Incorrect: g.remove ... pattern=landsat_* The later will transform the command line to: g.remove ... pattern=landsat_01 landsat_02 landsat_03 landsat_04 ... So I just checked, and indeed I hadn’t used the quotes when it didn’t work, so thank you very much for the pointer! Since that is indeed the case, it would be super nice to add a pattern flag to r.patch and v.patch. Best, Jeshua Lacock Founder/Engineer 3DTOPO Incorporated http://3DTOPO.com Phone: 208.462.4171 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
On Thu, Jan 22, 2015 at 9:10 PM, Jeshua Lacock jes...@3dtopo.com wrote: Since that is indeed the case, it would be super nice to add a pattern flag to r.patch and v.patch. It would. I'm not sure if there is a ticket for it. Note that some other modules accept file with names which gives different possibilities than pattern option, e.g. usage with temporal framework. ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
On Sun, Jan 18, 2015 at 1:00 AM, Jeshua Lacock jes...@3dtopo.com wrote: Greetings, In the event my experience working with rather large GRASS rasters may be useful, I thought I would share it. The following notes were compiled running the GRASS 7.0.0beta4-141230 package on Mac OS X 10.10.1. The system is a 6-core Xeon with 32GB of RAM, running in full 64-bit mode. 1. Using an r.external virtual mosaic as the input proved impossible trying to run r.resample on it on trying to produce a nearly teracell raster (I ended up breaking sub-tiles). I closely followed the instructions on the Wiki. 2. On smaller sub-tiles of said image, r.resample was insanely slow even with the external imagery. You may check/benchmark the other resampling modules, some of them may be more modern (ideally faster): http://grass.osgeo.org/grass70/manuals/keywords.html#resample In case you try, please share the outcome. cheers, Markus ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
On Sun, Jan 18, 2015 at 1:03 PM, Jeshua Lacock jes...@3dtopo.com wrote: Actually, I misspoke, the wildcard will not work with long lists either as it will crash with some kind argument too long message as well. This is strange, the wild card is interpreted inside the module, so the argument list shouldn't be too long. If you don't quite the pattern the shell can do the expansion but this is not what you want. Correct: g.remove ... pattern=landsat_* Incorrect: g.remove ... pattern=landsat_* The later will transform the command line to: g.remove ... pattern=landsat_01 landsat_02 landsat_03 landsat_04 ... in case you are in directory with files or directories named landsat_01, landsat_02, landsat_03, ... What is the message you get? And what is the command return code (do echo $? to find out)? ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
On Jan 17, 2015, at 8:04 PM, Jeshua Lacock jes...@3dtopo.com wrote: maybe silly question regarding g.remove, have you used the pattern option instead of the name option? Hi, Thank you for your suggestion Anna! Well I didn’t want to use wildcards because one little mistake and — The list was verified first by eye. Besides that, this same potential issue applies to many modules that may not have pattern matching that I might be more comfortable with then essentially rm -R. Actually, I misspoke, the wildcard will not work with long lists either as it will crash with some kind argument too long message as well. Best, Jeshua Lacock Founder/Engineer 3DTOPO Incorporated http://3DTOPO.com Phone: 208.462.4171 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
[GRASS-user] Large Grass Raster Notes
Greetings, In the event my experience working with rather large GRASS rasters may be useful, I thought I would share it. The following notes were compiled running the GRASS 7.0.0beta4-141230 package on Mac OS X 10.10.1. The system is a 6-core Xeon with 32GB of RAM, running in full 64-bit mode. 1. Using an r.external virtual mosaic as the input proved impossible trying to run r.resample on it on trying to produce a nearly teracell raster (I ended up breaking sub-tiles). I closely followed the instructions on the Wiki. 2. On smaller sub-tiles of said image, r.resample was insanely slow even with the external imagery. At first I thought it was a limitation of being overwhelmed with small sector requests and purchased a terabyte SSD, copied the external imagery to the SSD, but that didn’t noticeably help. Throughput was way less than 1mb/sec. 3. g.remove is unusable for long lists. Actually applies to I think any utility that takes a list for input. I wish I could just put it to a file so it could parse it line by line (As I understand the issue it is a limitation of the insanely long shell arguments). If anyone wants me to elaborate on anything, or has pointers how to do things better - please let me know! Cheers, Jeshua Lacock Founder/Engineer 3DTOPO Incorporated http://3DTOPO.com Phone: 208.462.4171 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
Hi, On Sat, Jan 17, 2015 at 7:00 PM, Jeshua Lacock jes...@3dtopo.com wrote: Greetings, In the event my experience working with rather large GRASS rasters may be useful, I thought I would share it. The following notes were compiled running the GRASS 7.0.0beta4-141230 package on Mac OS X 10.10.1. The system is a 6-core Xeon with 32GB of RAM, running in full 64-bit mode. 1. Using an r.external virtual mosaic as the input proved impossible trying to run r.resample on it on trying to produce a nearly teracell raster (I ended up breaking sub-tiles). I closely followed the instructions on the Wiki. 2. On smaller sub-tiles of said image, r.resample was insanely slow even with the external imagery. At first I thought it was a limitation of being overwhelmed with small sector requests and purchased a terabyte SSD, copied the external imagery to the SSD, but that didn’t noticeably help. Throughput was way less than 1mb/sec. 3. g.remove is unusable for long lists. Actually applies to I think any utility that takes a list for input. I wish I could just put it to a file so it could parse it line by line (As I understand the issue it is a limitation of the insanely long shell arguments). maybe silly question regarding g.remove, have you used the pattern option instead of the name option? Best, Anna If anyone wants me to elaborate on anything, or has pointers how to do things better - please let me know! Cheers, Jeshua Lacock Founder/Engineer 3DTOPO Incorporated http://3DTOPO.com Phone: 208.462.4171 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Large Grass Raster Notes
On Jan 17, 2015, at 8:00 PM, Anna Petrášová kratocha...@gmail.com wrote: maybe silly question regarding g.remove, have you used the pattern option instead of the name option? Hi, Thank you for your suggestion Anna! Well I didn’t want to use wildcards because one little mistake and — The list was verified first by eye. Besides that, this same potential issue applies to many modules that may not have pattern matching that I might be more comfortable with then essentially rm -R. Best, Jeshua Lacock Founder/Engineer 3DTOPO Incorporated http://3DTOPO.com Phone: 208.462.4171 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
