Re: [GRASS-user] GRASS GIS virtual raster data sets
* Markus Metz [2018-06-01 08:24:20 +0200]: On Thu, May 31, 2018 at 10:41 PM, Markus Neteler wrote: On Wed, May 30, 2018 at 4:21 PM, Nikos Alexandris wrote: > Dears, > > I guess there is no GRASS GIS built-in implementation of GDAL's VRT > concept (several related ideas listed > https://trac.osgeo.org/grass/wiki/Grass8Planning). I wish to see that as well (and discussed it a while ago with our dear Markus M... I have an idea about how to implement native GRASS VRT, but no resources to do so... That is great! If I may ask, is time the main requirement? GRASS GIS would gain a lot having the possibility to virtually mosaik raster tiles. > Once past some compilation issues, I am about to try to build a VRT > based on native GRASS GIS raster maps (using GDAL with support for GRASS > GIS raster data), then link to this VRT via `r.external` from inside a > regular Mapset. Interesting, will that not cause loop dependencies? I didn't think about it carefully then. Stefan's test shows that you are right, I guess. My naive thinking was: source images are, each, an inode. A VRT is a list of filenames pointing to inodes. Link via r.external to the VRT which points to the source inodes. Essentially, an indirect way to read GRASS GIS raster maps, from inside a valid GRASS GIS session. I guess I miss important parts of GRASS GIS' raster data model which render the above schema to be of no use. I thought of true internal support for that to avoid loops. > As a side-note, an image without pre-computed statistics stored in its > metadata, will require some time for `r.external` to work. The latter > likely forces this computation. > > Perhaps it is faster to use `gdalinfo`'s '-stats' option to do this. It will not be faster because you would need gdalinfo -mm -stats in order to get the actual values, not an approximation. Sometimes an approximation of the stats based on a subsample is stored in the metadata of a raster. > > I wonder if there are smart ways to build, in GRASS GIS' data base, > a larger virtual raster map out of many smaller ones, without the need > to actually patch the latter ones in a new raster map file. I guess that some C code needs to be touched in lib/raster/ to allow for a VRT-style raster concept. Yes. Maybe we missed the current GSoC train for this idea :)> The tricky part is to get a qualified student for this project. (I wish I could be in this position! :-/) Nikos signature.asc Description: PGP signature ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS GIS virtual raster data sets
Essentially, I have "exactly", I think, the same workflow in mind. So, I am in a path where you have been already. Thank you for sharing this Stefan. - Computing LST images from Landsat 8. - Instead of writing GRASS GIS native raster maps, I will switch to GeoTIFFs via r.external.out - After having built time series for hundreds of WRS2 Paths and Rows combinations, I end up with some aggregated products. - I tiled Europe in 10k^2 squares (in ETRS89) to have less tiles to deal with. Size of images per tile is acceptable (compressed via DEFLATE, ranging between less than 100 and up to 400MB). - I "need" to avoid to patch these tiles in single European-wide raster maps. Thus the need, and the idea, for a "VRT". So, I guess the idea won't work. I will do the attempt to "learn" this by experience. Thank you, Nikos * Stefan Blumentrath [2018-06-01 07:09:08 +]: Hi, Some experience with GDAL VRT and GRASS: Be aware of: https://trac.osgeo.org/grass/ticket/2837 Also note also that esp. for integer rasters, compression of one big raster map can be much better than with lots of smaller tiles [1]. So I guess cases where patching by means of a virtual raster is most beneficial is in intermediate steps of an analysis chain… Cheers Stefan 1: https://www.mail-archive.com/grass-user@lists.osgeo.org/msg33125.html From: grass-user On Behalf Of Markus Metz Sent: fredag 1. juni 2018 08:24 To: Markus Neteler Cc: GRASS user list Subject: Re: [GRASS-user] GRASS GIS virtual raster data sets On Thu, May 31, 2018 at 10:41 PM, Markus Neteler mailto:nete...@osgeo.org>> wrote: On Wed, May 30, 2018 at 4:21 PM, Nikos Alexandris mailto:n...@nikosalexandris.net>> wrote: > Dears, > > I guess there is no GRASS GIS built-in implementation of GDAL's VRT > concept (several related ideas listed > https://trac.osgeo.org/grass/wiki/Grass8Planning). I wish to see that as well (and discussed it a while ago with our dear Markus M... I have an idea about how to implement native GRASS VRT, but no resources to do so... GRASS GIS would gain a lot having the possibility to virtually mosaik raster tiles. > Once past some compilation issues, I am about to try to build a VRT > based on native GRASS GIS raster maps (using GDAL with support for GRASS > GIS raster data), then link to this VRT via `r.external` from inside a > regular Mapset. Interesting, will that not cause loop dependencies? I thought of true internal support for that to avoid loops. > As a side-note, an image without pre-computed statistics stored in its > metadata, will require some time for `r.external` to work. The latter > likely forces this computation. > > Perhaps it is faster to use `gdalinfo`'s '-stats' option to do this. It will not be faster because you would need gdalinfo -mm -stats in order to get the actual values, not an approximation. Sometimes an approximation of the stats based on a subsample is stored in the metadata of a raster. > > I wonder if there are smart ways to build, in GRASS GIS' data base, > a larger virtual raster map out of many smaller ones, without the need > to actually patch the latter ones in a new raster map file. I guess that some C code needs to be touched in lib/raster/ to allow for a VRT-style raster concept. Yes. Maybe we missed the current GSoC train for this idea :)> The tricky part is to get a qualified student for this project. Markus M markusN ___ grass-user mailing list grass-user@lists.osgeo.org<mailto:grass-user@lists.osgeo.org> https://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user -- Nikos Alexandris | Remote Sensing & Geomatics GPG Key Fingerprint 6F9D4506F3CA28380974D31A9053534B693C4FB3 signature.asc Description: PGP signature ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS GIS virtual raster data sets
On Thu, May 31, 2018 at 10:41 PM, Markus Neteler wrote: > > On Wed, May 30, 2018 at 4:21 PM, Nikos Alexandris > wrote: > > Dears, > > > > I guess there is no GRASS GIS built-in implementation of GDAL's VRT > > concept (several related ideas listed > > https://trac.osgeo.org/grass/wiki/Grass8Planning). > > I wish to see that as well (and discussed it a while ago with our dear > Markus M... I have an idea about how to implement native GRASS VRT, but no resources to do so... > > GRASS GIS would gain a lot having the possibility to virtually mosaik > raster tiles. > > > Once past some compilation issues, I am about to try to build a VRT > > based on native GRASS GIS raster maps (using GDAL with support for GRASS > > GIS raster data), then link to this VRT via `r.external` from inside a > > regular Mapset. > > Interesting, will that not cause loop dependencies? > > I thought of true internal support for that to avoid loops. > > > As a side-note, an image without pre-computed statistics stored in its > > metadata, will require some time for `r.external` to work. The latter > > likely forces this computation. > > > > Perhaps it is faster to use `gdalinfo`'s '-stats' option to do this. It will not be faster because you would need gdalinfo -mm -stats in order to get the actual values, not an approximation. Sometimes an approximation of the stats based on a subsample is stored in the metadata of a raster. > > > > I wonder if there are smart ways to build, in GRASS GIS' data base, > > a larger virtual raster map out of many smaller ones, without the need > > to actually patch the latter ones in a new raster map file. > > I guess that some C code needs to be touched in lib/raster/ to allow > for a VRT-style raster concept. Yes. > Maybe we missed the current GSoC train for this idea :)> The tricky part is to get a qualified student for this project. Markus M > markusN > ___ > grass-user mailing list > grass-user@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS GIS virtual raster data sets
On Wed, May 30, 2018 at 4:21 PM, Nikos Alexandris wrote: > Dears, > > I guess there is no GRASS GIS built-in implementation of GDAL's VRT > concept (several related ideas listed > https://trac.osgeo.org/grass/wiki/Grass8Planning). I wish to see that as well (and discussed it a while ago with our dear Markus M... GRASS GIS would gain a lot having the possibility to virtually mosaik raster tiles. > Once past some compilation issues, I am about to try to build a VRT > based on native GRASS GIS raster maps (using GDAL with support for GRASS > GIS raster data), then link to this VRT via `r.external` from inside a > regular Mapset. Interesting, will that not cause loop dependencies? I thought of true internal support for that to avoid loops. > As a side-note, an image without pre-computed statistics stored in its > metadata, will require some time for `r.external` to work. The latter > likely forces this computation. > > Perhaps it is faster to use `gdalinfo`'s '-stats' option to do this. > > I wonder if there are smart ways to build, in GRASS GIS' data base, > a larger virtual raster map out of many smaller ones, without the need > to actually patch the latter ones in a new raster map file. I guess that some C code needs to be touched in lib/raster/ to allow for a VRT-style raster concept. Maybe we missed the current GSoC train for this idea :) markusN ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
[GRASS-user] GRASS GIS virtual raster data sets
Dears, I guess there is no GRASS GIS built-in implementation of GDAL's VRT concept (several related ideas listed https://trac.osgeo.org/grass/wiki/Grass8Planning). Once past some compilation issues, I am about to try to build a VRT based on native GRASS GIS raster maps (using GDAL with support for GRASS GIS raster data), then link to this VRT via `r.external` from inside a regular Mapset. As a side-note, an image without pre-computed statistics stored in its metadata, will require some time for `r.external` to work. The latter likely forces this computation. Perhaps it is faster to use `gdalinfo`'s '-stats' option to do this. I wonder if there are smart ways to build, in GRASS GIS' data base, a larger virtual raster map out of many smaller ones, without the need to actually patch the latter ones in a new raster map file. Nikos signature.asc Description: PGP signature ___ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user
