Re: [GRASS-dev] new addon: i.cutlines - Creates semantically meaningful tile borders
On 14/08/18 10:13, Moritz Lennert wrote: On 13/08/18 15:30, Markus Neteler wrote: Hi Moritz, On Mon, Aug 13, 2018 at 2:04 PM, Moritz Lennert wrote: On 13/08/18 13:41, Markus Neteler wrote: ... AFAIK, the only moment where i.cutlines potentially reads the whole image would be in the edge detection part. That's why there is the tiling option to avoid just that. I suppose you refer to tile_width=integer Width of tiles for tiled edge detection (pixels) tile_height=integer Height of tiles for tiled edge detection (pixels) ? So, unless I'm forgetting something, you should be able to work on large images. If you have seen this crash the module, please file a bug report. A colleague working with a large dataset > 10e9 pixels) just reported that the module does not crash, but that it seems to take "forever" (he stopped the process after a day). I guess this is in the r.cost phase. An option would be to tile (and parallelize) the entire process which would mean finding cutlines in the individual tiles, making sure that the start and endpoints of these cutlines match the start and endpoints on the neighboring tiles... Any suggestions are welcome. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] new addon: i.cutlines - Creates semantically meaningful tile borders
On 13/08/18 15:30, Markus Neteler wrote: Hi Moritz, On Mon, Aug 13, 2018 at 2:04 PM, Moritz Lennert wrote: On 13/08/18 13:41, Markus Neteler wrote: ... AFAIK, the only moment where i.cutlines potentially reads the whole image would be in the edge detection part. That's why there is the tiling option to avoid just that. I suppose you refer to tile_width=integer Width of tiles for tiled edge detection (pixels) tile_height=integer Height of tiles for tiled edge detection (pixels) ? So, unless I'm forgetting something, you should be able to work on large images. If you have seen this crash the module, please file a bug report. OK, fine, we'll try and report. Note that for versions < 7.5 (I'll have to check whether Pietro's fix was backported to 7.4) I'm not sure either. Actually you backported it: https://trac.osgeo.org/grass/changeset/72541/grass/branches/releasebranch_7_4/lib/python/pygrass So, yes, it should work with 7.4.1. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] new addon: i.cutlines - Creates semantically meaningful tile borders
Hi Moritz, On Mon, Aug 13, 2018 at 2:04 PM, Moritz Lennert wrote: > On 13/08/18 13:41, Markus Neteler wrote: ... > AFAIK, the only moment where i.cutlines potentially reads the whole image > would be in the edge detection part. That's why there is the tiling option > to avoid just that. I suppose you refer to tile_width=integer Width of tiles for tiled edge detection (pixels) tile_height=integer Height of tiles for tiled edge detection (pixels) ? > So, unless I'm forgetting something, you should be able > to work on large images. If you have seen this crash the module, please file > a bug report. OK, fine, we'll try and report. > Note that for versions < 7.5 (I'll have to check whether Pietro's fix was > backported to 7.4) I'm not sure either. > there is a parameter name conflict between i.zc and the > GridModule class used for the tiling, so you'd have to use i.edge. At time we use trunk. thanks, Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] new addon: i.cutlines - Creates semantically meaningful tile borders
Hi Markus, On 13/08/18 13:41, Markus Neteler wrote: Hi Moritz, all, On Tue, Mar 13, 2018 at 4:06 PM, Moritz Lennert wrote: Hi to all, A new addon is available [1]: i.cutlines tiles the images into tiles with irregular borders that avoid cutting through meaningful objects. This allows tiling an image for parallel processing while avoiding border effects. ... [1] https://grass.osgeo.org/grass74/manuals/addons/i.cutlines.html we are currently trying to use this approach on a huge (?) area of 50 gigapixels of aerial images (i.e., the free openNRW 10cm orthophotos) which we have to classify. In future the areas we'll have to process may be even bigger. Right now, as far as I understand, i.cutlines is reading the entire images. This becomes increasingly difficult with bigger areas due to hardware limitations. Do you have a suggestion how to deal with that? Kind of chicken and egg problem? :) AFAIK, the only moment where i.cutlines potentially reads the whole image would be in the edge detection part. That's why there is the tiling option to avoid just that. So, unless I'm forgetting something, you should be able to work on large images. If you have seen this crash the module, please file a bug report. Note that for versions < 7.5 (I'll have to check whether Pietro's fix was backported to 7.4) there is a parameter name conflict between i.zc and the GridModule class used for the tiling, so you'd have to use i.edge. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] new addon: i.cutlines - Creates semantically meaningful tile borders
Hi Moritz, all, On Tue, Mar 13, 2018 at 4:06 PM, Moritz Lennert wrote: > Hi to all, > > A new addon is available [1]: i.cutlines tiles the images into tiles with > irregular borders that avoid cutting through meaningful objects. This allows > tiling an image for parallel processing while avoiding border effects. ... > [1] https://grass.osgeo.org/grass74/manuals/addons/i.cutlines.html we are currently trying to use this approach on a huge (?) area of 50 gigapixels of aerial images (i.e., the free openNRW 10cm orthophotos) which we have to classify. In future the areas we'll have to process may be even bigger. Right now, as far as I understand, i.cutlines is reading the entire images. This becomes increasingly difficult with bigger areas due to hardware limitations. Do you have a suggestion how to deal with that? Kind of chicken and egg problem? :) thanks Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] new addon: i.cutlines - Creates semantically meaningful tile borders
On Tue, Mar 13, 2018 at 4:06 PM, Moritz Lennert wrote: > Hi to all, > > A new addon is available [1]: i.cutlines tiles the images into tiles with > irregular borders that avoid cutting through meaningful objects. This allows > tiling an image for parallel processing while avoiding border effects. > > Enjoy ! Cool! It would also be interesting for post-orthophoto mosaiking. Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] new addon: i.cutlines - Creates semantically meaningful tile borders
Hi to all, A new addon is available [1]: i.cutlines tiles the images into tiles with irregular borders that avoid cutting through meaningful objects. This allows tiling an image for parallel processing while avoiding border effects. Enjoy ! Moritz [1] https://grass.osgeo.org/grass74/manuals/addons/i.cutlines.html ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
