On Sat, Mar 11, 2017 at 8:53 AM, Nikos Alexandris <[email protected]> wrote: > > Nikos Alexandris > >>> Why does (attempting to) import a 38m pixel resolution GHSL [0] GeoTIFF >>> layer, ie GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0_p1.tif, in GRASS' >>> db progress slow?
because it is a very large raster map: Size is 507904, 647168 > > > (Apologies for cross-posting to gdal-dev) > > Markus Neteler: > >> Can you elaborate a bit more? I have downloaded and checked: >> >> That is 9835059101 bytes in 19885 files or I downloaded the wrong one >> (please post an URL). > > For example <http://ghsl.jrc.ec.europa.eu/ghs_bu.php>, > > see > > GHS_BUILT_LDS1975_GLOBE_R2016A_3857_38 (768MB) GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38 (854MB) GHS_BUILT_LDS2000_GLOBE_R2016A_3857_38 (892MB) GHS_BUILT_LDS2014_GLOBE_R2016A_3857_38 (900MB) > > "3857" is the EPSG code. They are split in two GeoTIFFs (p1, p2) and > there is a VRT along with overviews for it. No overviews for the TIFFs. > > For example: > > GHSL_data_access_v1.3.pdf > GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0.clr > GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0.vrt > GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0.vrt.ovr > GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0_p1.tif > GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0_p2.tif > > > Even trying to clip, with gdal_translate, might create file(s) of > hundreds of GBs. This might be due to missing compression. then use compression. The source tiffs use LZW with blocks of 4096x4096 cells. > > The import of p1 or p2 or of the VRT file in GRASS' data base, via > r.in.gdal/r.import, does not progress at all. Importing GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0_p1.tif with r.in.gdal took 1:31 hours on a laptop with SSD. The resultant cell file was 1.5 GB. Recompressing with BZIP2 took 2:20 hours and the size of the cell file was reduced to a mere 143 MB. > ( > Side question: why is max 2047? What if there is a lot more of RAM? > ) To avoid integer overflow because 2047 is converted to bytes with 2047 * 1024 * 1024. Markus M
_______________________________________________ grass-user mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/grass-user
